If you’re really interested in reading some technical bafflegab, LaCour has posted a 23 page response to accusations that he faked his data. I think I’d recommend instead reading this shorter summary.
In the 23-page document, political science graduate student Michael LaCour of the University of California (UC), Los Angeles, attacks the methods and motives of researchers who raised questions about his research, but confirms that he lied about some funding sources and the incentives used to attract participants. And he admits that he destroyed the data used to produce the study, claiming that action was required by a UC Los Angeles institutional review board (IRB) in order to protect the privacy of participants.
LaCour’s response does not, however, directly answer a number of other questions surrounding the study—and it raises new issues. LaCour does not address, for example, why the company that he claimed had conducted his surveys says it has no knowledge of the researcher or his project and does not have the capability to conduct some of the claimed work. (After the statement’s release, LaCour told The New York Times that he did not use that company, but another unidentified company.)
If he lied about basic features of his protocol, why should we believe the rest of it? I also have to point out that claiming he had to destroy the data because his Institutional Review Board protocol required it is nonsense, given that he did not have IRB approval to do the study at all. Destroying data because he claims to be a stickler for the rules while being so lax he didn’t even get IRB approval is incoherent.
Ironically, he also complains that his critics did not come directly to him first, instead informing his co-author and advisor, Green. It strikes me as strangely similar to the complaints about those people who write blog entries about public figures and actions without first calling them up. If you screwed up so blatantly, if your actions reveal you are an untrustworthy source, it is entirely appropriate to pursue the truth without first asking a liar for permission.
Having been through the IRB process, I can understand the requirement to destroy data that contains personal information after extracting the needed metadata. But that was after going through the process.
In some ways, this research appears to be what the “chocolate as a diet aid” authors were trying to expose.
Mykroft (@1), can one retain the data if the personal info has been scrubbed/anonymized?
What the fuck is wrong with him O_o?
Somebody forgot the first rule of holes….
@usernames
I included destroying the data in my IRB proposal. Hard to anonymize network packet data, with emails, browsing data, etc.
“Forgot” may not be the right word. As far as I can tell, his career in science is toast, no matter if he stops digging or not. So, he figures, he might as well go on the offensive like a politician who says “the evil media misquoted me out of context”.
Before or after peer review?
And does the fact that he lied about basic features of his protocol not invalidate the study in and of itself?
From the best post in the history of the internet:
Basically, trusting anything LaCour has to say would be an order of magnitude past merely being foolish. Even considering what he says as a basis for concluding anything other than that he’s a lying liar who lies would be a colossal waste of time. There’s perhaps some value to noting that his lies are particularly ridiculous, but more important is to note that everything he says about his project, his career, and his reputation should be presumed to be a lie and acted upon as though a lie whether or not it appears ridiculous on the surface. You can’t extract truth beyond the fact that he’s a liar from what he has to say, no matter how you massage the data. What he has to say is not actually useful data for establishing anything beyond whether or not he’s said it.
Yeah, in the social sciences there are legitimate privacy concerns that can warrant destroying data. My point is that it’s peculiar to care about IRB requirements when you’ve completely ignored IRB up to that point.
How do you have IRB requirements WITHOUT GETTING IRB APPROVAL TO CONDUCT THE STUDY IN THE FIRST PLACE? This guy is so full of shit, I can’t even.
Per Green’s retraction letter, the university investigated with Qualtrics LaCour’s claims that he destroyed his data and found no evidence to support that. LaCour counter-claims that Qualtrics wouldn’t be able to know either way, because he never gave them his username and ID for the survey.
(From Retraction Watch) I guess it depend on Qualtrics’ privacy policies, but if they weren’t actually able to tell, wouldn’t they have said as much to the University?
Neuroskeptic has a nice analysis of some statistical issues with LaCour’s recent statements.
Donald Green, the second author, didn’t have IRB approval from his institution which is why he didn’t do an independent data analysis which would probably have raised questions in his mind.
Also LaCour discussed an email from Jason Peterson, a uSamp VP. uSamp says they have no employee by that name. LaCour has not tried to explain this discrepancy.
More issues:
I have written, submitted, rewritten and resubmitted (and subsequently received approval for) a butt-load of social science IRB protocols in my career, at three different institutions, UC Berkeley, University of Nebraska Lincoln, and UC San Diego. I have never been required to destroy data. Indeed, I have been required to keep data in a locked cabinet in an office with limited access. But never to destroy it. (And most of it was videotapes so the informants were easily identifiable). LaCour is an untrustworthy reporter about nearly everything, I assume.
Any data that can be normalized can be anonymized. In fact, it should probably be done before the researcher does his analysis, as an extra safeguard against bias.
E.g. you have millions of packets with ipaddrs. 10000 subjects = 10000 ipaddrs. You create a lookup table with the 10000 ipaddrs, numbered 1-10000. Each table entry also contains the set of non-identifying characteristics of the individual who uses that ipaddr. Then each time you run into an ipaddr in a packet, you look it up in the table and substitute its table entry number into the packet (thereby anonymizing that information in the packet, which can then be permanently retained). Once you are done, you the delete the identifying-information (the ipaddr) from the lookup table, and now it can be retained too.
After all, who cares about identifying-information when analyzing the data? It is just a way to connect the packet information payload to the set of characteristics of the individual, not to the individual herself. The individual’s identity is irrelevant, and can and should be anonymized out.
I forgot to add, saying the IRB required him to destroy the data is just the scientific way of saying the dog ate his homework. If the IRB would not accept anonymizing the data in lieu of destroying it, they are not an organization that promotes open and accountable science. If I were a researcher, and valued my reputation, I would not have anything to do with such an organization. Results based on a deleted data set are inherently irreproducible results, and belong here: http://www.jir.com/
23 pages! He knows the value of volume if nothing else. It’ll probably turn out he spent a month writing some special software to generate counter criticisms.