I wrote recently about an article by Ed Yong about a new technique that could be of use in catching research papers where the authors have massaged their data to get positive results. Yong has now identified and interviewed the hitherto anonymous developer of this technique. He is social psychologist Uri Simonsohn of the University of Pennsylvania.
In an interview with Yong, Simonsohn explains how he stumbled onto this problem and the basic idea behind his method, which he is planning to publish. He also illuminates a problem, if one can consider it that, in that science operates a lot on trust. Scientists believe the work of other scientists unless there is reason to suspect otherwise and this makes it easier for less scrupulous scientists to sail a little too close to the wind.
What’s systemic is the lack of defences. Social psychology — and science in general — doesn’t have sufficient mechanisms for preventing fraud.
If there’s a tool to detect fake data, I’d like people to know about it so we can take findings that aren’t true out of our journals. And if it becomes clear that fabrication is not an unusual event, it will be easier for journals to require authors to publish all their raw data. It’s extremely hard for fabrication to go undetected if people can look at your data.
A university’s reputation suffers a lot when people fake data, but they don’t have tools for preventing that — journals do. Journals should be embarrassed when they publish fake data, but there’s no stigma. They’re portrayed as the victims, but they’re more like the facilitators, like a government that looks the other way. I’d like journals to take ownership of the problem and start working towards stopping it.
I think that is a very good idea. If the self-correcting nature of science is to be maintained, journals must start policing themselves much more, either by giving closer scrutiny to the papers submitted to them before publication or being more willing to publish contradictory findings of previously published work or even retracting them.
A longer version of Yong’s interview with Simonsohn can be read here.