Suppose someone presents you with some data in the form of numbers in tables. These numbers may have been used as evidence to support some contention. Can you judge whether those numbers are authentic without actually repeating the entire study?
There have been cases in the past where people have reviewed other people’s data and found suspicious numeric patterns that would have been unlikely to occur naturally. One of the famous cases is that involving Cyril Burt’s studies of twins that purportedly showed that genetics played a far greater role in a person’s development than had been previously thought. In 1974, soon after Burt’s death in 1971, Leon Kamin analyzed Burt’s data and found that they were likely not correct because the statistical correlations he reported stayed stable up to the third decimal place, despite being obtained from different sample sizes. The odds of that happening naturally are extremely low. (Not in Our Genes by R. C. Lewontin, Steven Rose, and Leon Kamin (1984) p. 103.) [Read more…]
