I’m teaching our science writing course in the Fall, and I’m also one of the instructors in our teachers’ workshop next month (we still have room for more participants!). And now I’ve found a useful, general, basic paper that I have to hand out.
Motulsky, HJ (2014) Common Misconceptions about Data Analysis and Statistics. JPET 351(1):200-205.
What it’s got is clear, plain English; brevity; covers some ubiquitous errors; will be incredibly useful for our introductory biology students. You should read it, too, for background in basic statistical literacy. Here’s the abstract.
Ideally, any experienced investigator with the right tools should be able to reproduce a finding published in a peer-reviewed biomedical science journal. In fact, however, the reproducibility of a large percentage of published findings has been questioned. Undoubtedly, there are many reasons for this, but one reason may be that investigators fool themselves due to a poor understanding of statistical concepts. In particular, investigators often make these mistakes: 1) P-hacking, which is when you reanalyze a data set in many different ways, or perhaps reanalyze with additional replicates, until you get the result you want; 2) overemphasis on P values rather than on the actual size of the observed effect; 3) overuse of statistical hypothesis testing, and being seduced by the word “significant”; and 4) over-reliance on standard errors, which are often misunderstood.
I can probably open any biomedical journal and find papers that commit all four of those errors.