NY Magazine has the best summary of the LaCour affair. This is the recent case of a study published by a graduate student that showed the effectiveness of gay & lesbian survey-takers of persuading people to support marriage equality…which has since been found to consist of falsified data. The article summarizes how the study was exposed, and there were grounds for suspicion long before the news broke. Reading the protocols and doing a little math, for instance, revealed that this graduate student had apparently paid out about a million dollars for this survey. And it just unraveled from there.
Once the pieces of the puzzle were exposed, the problem was obvious. But why were people reluctant to question the results? This bit was telling:
But even before Broockman, Kalla, and Aronow published their report, LaCour’s results were so impressive that, on their face, they didn’t make sense. Jon Krosnick, a Stanford social psychologist who focuses on attitude change and also works on issues of scientific transparency, says that he hadn’t heard about the study until he was contacted by a “This American Life” producer who described the results to him over the phone. “Gee,” he replied, “that’s very surprising and doesn’t fit with a huge literature of evidence. It doesn’t sound plausible to me.” A few clicks later, Krosnick had pulled up the paper on his computer. “Ah,” he told the producer, “I see Don Green is an author. I trust him completely, so I’m no longer doubtful.” (Some people I spoke to about this case argued that Green, whose name is, after all, on the paper, had failed in his supervisory role. I emailed him to ask whether he thought this was a fair assessment. “Entirely fair,” he responded. “I am deeply embarrassed that I did not suspect and discover the fabrication of the survey data and grateful to the team of researchers who brought it to my attention.” He declined to comment further for this story.)
This isn’t just a problem with political science, or academia, or atheism — it’s people in general. Once a monkey is declared king, all the other monkeys stop thinking.
yes, good advice: “Read the paper, not just the names of the authors, before accepting the paper’s conclusions”. in other words: THINK, don’t just ACCEPT!
EG.
just because the paper presents a conclusion one prefers, read it on its own merits. If the methods it used to reach that conclusion are flawed then tell them to try again, with better methods. Flawed methods don’t disprove the conclusion; invalidation is not disproof. Bad methods don’t support the conclusion; use proper methodologies and see what conclusion results.
On the other paw, seeing the name “LaCour” on a paper will now raise eyebrows and suspicions for a generation (as with Regnerus, Dembski, Fleischmann & Pons, et alia, et alia.
Unfair, but utterly justified.
Oh, LaCour. I was thinking LaClair.
Did Green’s student put his name on the paper without telling him?
Or does he put his name on everything that comes out of his lab, without necessarily reading it first?
Both would be horrible, and both have (with other people) happened in real life. Their consequences, though, would be quite different: in the first case, the student can go look for a new thesis supervisor, and best of luck with that, hah; in the second… *crickets*
I’m relying on my memory here, but I think it’s substantially right; I haven’t had that many days to forget what I’ve read at Retractionwatch, etc.
LaCour was not Green’s student in any sense, and certainly was not in “his lab” (LaCour was/is at UCLA; Green is/will be at Columbia; I don’t even know if Green has a “lab”). Green gave a talk at a conference; LaCour came up afterwards and said he was doing some related work, and would Green like to see what he had? Green said, sure, and LaCour sent him something (presumably already fraudulent…okay, it turns out I have had enough days to forget some of this). One of them (I think LaCour) suggested writing a joint paper. Green signed off on the paper (I don’t think I’ve seen any account of how much of it—e.g., the introduction, or the review of the literature, or the methodology, or the conclusions, any of them in whole [unlikely] or in part [I would hope so, otherwise he really doesn’t have any excuse for putting his name on the thing, even if he did read it!]—Green wrote).
Good Christ! Why are we relying on supposition and memory?! This is a thread about criticizing people for failing to read critically. Read the Retraction Watch story and stop guessing!
Honestly, the part that upsets me the most is where Brookman and Kalla were told to stop making noise about the LaCour study until they had absolute proof. Outright fraud of the sort LaCour perpetrated is rare, but fiddling with your numbers until you get a “cleaner” p value or whatnot is much more common – but the latter is harder to prove unless a full-scale replication is performed.
Found it! The link is almost at the bottom of the long NY Mag article.
Quote from Green:
“Convinced that the results were robust, I helped Michael LaCour write up the findings, especially the parts that had to do with the statistical interpretation of the experimental design. Given that I did not have IRB approval for the study from my home institution, I took care not to analyze any primary data — the datafiles that I analyzed were the same replication datasets that Michael LaCour posted to his website.”
That makes sense.
What upsets me even a bit more are the statements in the NY Mag article which say that people who make such noise have all sorts of cartoonishly evil motivations interpreted into them (like wanting to take down a big name out of some kind of bizarre envy) and end up having trouble getting grants or jobs not just for those reasons but also because correcting other people’s work is considered more lowly than doing something original. Uh, people, somebody has to do it, or we’ll soon all stand on original dungheaps instead of the shoulders of giants.
Self-correcting nature of science?
Interesting Retraction Watch post on LaCour’s response and other recent developments.
Interview with Green in NY Mag.
So preventing a well respected person from making a fool of themselves, is considered too lowly to engage in? Pointing out errors in said paper is being envious of the author’s reputation? (“pointing out errors” is NOT pointing at minor flaws and calling them major errors.) yeah, makes perfect sense, so be it. (NOT).
The really messed up part of this is that idiots – creationists and climate change deniers – will say “See, scientists can’t be trusted” when in fact, this is pretty much the canonical example of science working.
Scientist A produces surprising result. Scientist B tries to replicate result, fails. Scientist A’s result is thrown out. There were some serious human failings there – the effect of the media and doing science by press release, the effect of Green’s popularity etc – but the scientific method itself found the problem and corrected it.
And the consequences for the student who found the error are the ultimate disproof of the idiots’ claims. He’s not going to be banished to the basement for questioning orthodoxy and overturning a popular result, he’s a rockstar for questioning orthodoxy, doing solid science and overturning a popular result.
The difference between him and the idiots is that they think the “questioning orthodoxy” part is enough and they shouldn’t have to do the science part themselves.
Thanks for posting the links, David. I’m on an awful mobile device. The details are a lot more routine than most scientists care to admit. LaCour’s part is clearly unethical, but the rest is a series of small tragedies.
Green deserves a lot of credit for immediately retracting the story as soon as someone brought the problem to his attention, despite the hit to his reputation.
So would Green have done a better job on the Dark Chocolate Diet?
I am still so utterly, utterly impressed with Don Green’s reaction to this. To my mind, that is the absolute epitome of how a proper scientist should respond to such a situation.
Oh yes.
That said, it is the only way to salvage his reputation in the long run. What else could he have done? Defend LaCour by denying the pretty much obvious? That would only have worked for a few days.