How not to do science


Via Mano, we get to learn what it was like working in the stem cell lab of Piero Anversa.

The day to day operation of the lab was conducted under a severe information embargo. The lab had Piero Anversa at the head with group leaders Annarosa Leri, Jan Kajstura and Marcello Rota immediately supervising experimentation. Below that was a group of around 25 instructors, research fellows, graduate students and technicians. Information flowed one way, which was up, and conversation between working groups was generally discouraged and often forbidden.

Raw data left one’s hands, went to the immediate superior (one of the three named above) and the next time it was seen would be in a manuscript or grant. What happened to that data in the intervening period is unclear.

A side effect of this information embargo was the limitation of the average worker to determine what was really going on in a research project. It would also effectively limit the ability of an average worker to make allegations regarding specific data/experiments, a requirement for a formal investigation.

The general game plan of the lab was to use two methods to control the workforce: Reward those who would play along and create a general environment of fear for everyone else. The incentive was upward mobility within the lab should you stick to message. As ridiculous as it sounds to the average academic scientist, I was personally promised money and fame should I continue to perform the type of work they desired there. There was also the draw of financial security/job stability that comes with working in a very well-funded lab.

On the other hand, I am not overstating when I say that there was a pervasive feeling of fear in the laboratory. Although individually-tailored stated and unstated threats were present for lab members, the plight of many of us who were international fellows was especially harrowing. Many were technically and educationally underqualified compared to what might be considered average research fellows in the United States. Many also originated in Italy where Dr. Anversa continues to wield considerable influence over biomedical research.

Wow. I was sure lucky. When I was in grad school, I was in a research group of about the same size, with primarily 3 people running 3 labs: Chuck Kimmel, Monte Westerfield, and Judith Eisen. They were independent and co-equal. Their post-docs and grad students and undergrads pretty much had free run of all of the labs, to the point where it was often difficult to tell who was officially associated with which lab. We had weekly lab meetings in which we’d freely share data with each other; if someone had a good idea or a set of relevant skills, we’d have collaborations. When it was time to publish a paper, the people who had done the work would get together to write it, and authorship reflected the research team, not some hierarchy…so I emerged from grad school with papers published with both Monte and Judith, but not a one with my titular lab head’s name on it, because he had these scruples about not putting his name on work to which he had not personally and directly contributed.

I thought that was how everyone did science, as an open and egalitarian process. I guess I was wrong.

If you want more examples of science being done badly, Harvard has released the full report on the Marc Hauser scientific misconduct case. It’s clear that he faked data, and demanded that his students confirm his hypotheses. Hauser’s defense: “people in his laboratory conspired against him, due to academic rivalry and disgruntlement”, a claim that did not hold up in the investigation.

I seriously cannot imagine Chuck Kimmel, or any of my academic mentors, ever tolerating any of the shenanigans Hauser was pulling. Just the idea that you’d gather data and then pass it up a hierarchy for analysis, and be told by your advisor what it meant…bizarre.

Man, was I lucky. I hope all of you who are in grad school are following these scandals, and seeing modeled how science should not operate — and are standing up for scientific integrity in your own labs. Be involved in every step of an experiment, collaborate and share ideas, question authority. It’s the only way to do good science.

Comments

  1. Antiochus Epiphanes says

    This was precisely my experience teking in a med school research lab in the early 90s. Shenanigans soured me to high profile research and largely motivated my study of plants.

  2. says

    It no doubt changes who works in those areas. Can anyone imagine the next Richard Feynman thriving in that kind of lab?

  3. says

    I’m at Plymouth State, in NH. I’m having an experience much more like yours, PZ. We’re an underfunded lab, but we’re a happy “family” here. I’m glad I didn’t hold out for a better stipend.

  4. David Marjanović says

    Information flowed one way, which was up, and conversation between working groups was generally discouraged and often forbidden.

    lolwut

    what is this I can’t even

    I don’t even

    What?

    How does one even go about enforcing a ridiculous ban on conversations!?! Monitoring everyone 24/7?

  5. David Marjanović says

    Forgot:

    Hauser’s defense: “people in his laboratory conspired against him, due to academic rivalry and disgruntlement”

    How would that excuse or explain anything? “Everybody hates me, so I had to make shit up”? That doesn’t even compute.

    1. Be conspired against
    2. Make shit up
    3. ???
    4. Profit!

  6. bortedwards says

    Is it my confirmation bias or the disproportionate number and size of medical labs, but are these stories of disfunction and allegations of blatant fraud overrepresented in the medical fields? If so my feeling would be that the cutthroat scrimmage by so many for relatively little funding has gone beyond healthy competition and the model is now broken.
    I see this too in the slightly less back-stabby field of evolutionary biology, where although the pressures to publish high impact research early may be slightly relaxed, there is a percolating feeling of depression and impending doom about any future in the field. That is slightly OT I know, but it all adds up to heart shredding questioning of why we are doing what we are doing anymore…

  7. David Marjanović says

    Right after the part PZ has quoted:

    This combination of being undesirable to many other labs should they leave their position due to lack of experience/training, dependent upon employment for U.S. visa status, and under constant threat of career suicide in your home country should you leave, was enough to make many people play along.

    Even so, I witnessed several people question the findings during their time in the lab. These people and working groups were subsequently fired or resigned. I would like to note that this lab is not unique in this type of exploitative practice, but that does not make it ethically sound and certainly does not create an environment for creative, collaborative, or honest science.

    Lessons Learned

    So what, if anything, did I learn from spending a period of my life in my scientific nightmare? The conditions I have written about are not unique, although the particulars of how the misconduct happened may be. The simplest explanation is that, in spite of the efforts of ethical watchdogs, these are behaviors that science is selecting for with its current funding and publication mechanisms. I was glad to learn of the investigation regarding this lab but without vigilance and alterations to current structures, newer, more careful versions of Piero Anversa will undoubtedly move in to take his place.

  8. David Marjanović says

    are these stories of disfunction and allegations of blatant fraud overrepresented in the medical fields?

    That’s how it looks to me. I’ve long thought that fraud occurs where it can pay off – in fields where actual money is involved.

    Outside of those, you get Aëtogate.

  9. Moggie says

    If Piero Anversa were conducting an experiment to devise the worst way of doing science, he could hardly have improved on this.

  10. gussnarp says

    I had one professor who I always respected and thought was quite good. He was rigorous in the classroom, and he could politely savage any research presentation with all the right questions in a way that led to better science at the end – but there were students who did research directly under him who felt very differently about him and felt that he was very controlling and that he attempted to shut down research that was leading to conclusions that contradicted his work. That sort of thing should not go on, and I certainly hope that the students in my department were wrong and overstating the case. Certainly it was nothing on this level, but nonetheless, if it was going on it’s disappointing and needs to be stopped. But how do a couple of grad students take action on this when their careers are on the line and the professor has all the power?

  11. says

    I’ve heard all sorts of stories about who gets their name on a paper and in which order. Things like the lab’s head co-signing all papers, in addition to the actual persons who did the work. (Or even because their lab owns a key piece of equipment used in the work done and published by another team…) But this is a whole other order of bad behavior!

  12. says

    Overuse and misuse of authoritarian instincts is a problem all over the place. It’s a general human problem that transcends mere social group as a category. Though the scale of the problem is different in intensity and kind between groups.

  13. numerobis says

    be told by your advisor what it meant

    That’s so bizarre. My grad school experience was that *I* had to tell *my advisor* what the findings meant. And as a postdoc, same thing: I spent many hours trying to figure out what my student was trying to explain about his findings. First off, I didn’t have time to make up explanations for his stuff, I was busy with my own; secondly, students don’t stay students for long, so they need to learn how to explain right quick.

  14. ChasCPeterson says

    I was lucky too. The organismal biology crowd at UCLA in the 80s was supportive and collaborative and data were for sharing and talking about. In paticular, this:

    he had these scruples about not putting his name on work to which he had not personally and directly contributed.

    was an explicit principle.
    I learned pretty quickly that the rules arew different elsewhere. Soooo many senior authorships are unearned; makes me queasy.

  15. says

    @5, David Marjanović

    How does one even go about enforcing a ridiculous ban on conversations!?! Monitoring everyone 24/7?

    Not necessary, if you go about it the right way, the way that (for example) labs manage to somehow not move towards gender parity without actually doing anything legally actionable. If you do things right, and set the “right” tone, people will monitor themselves. For example:

    1. Have obscure rules which are either essentially impossible to actually obey or which are extremely awkward and therefore unlikely to be obeyed. If you catch someone talking to someone outside their group, hit them with a punishment for breaking those rules. Don’t enforce those rules otherwise.

    2. If anyone mentions to you that they were talking to someone in another group and got a good idea/started wondering about something/whatever, make your displeasure clear through body language and tone of voice, but not words, and talk to other people in the group to the exclusion of that person for the next several weeks.

    3. If anyone asks you about communications with other groups, tell them it’s how the lab has always operated, and cite previous important work as showing that the approach “works”. Be condescending. If they persist, claim that collaboration will effect the funding structure and you would have to take money away from their group and share it with the other group.

    4. Keep a list of tasks which are not actually lab work and which would generally be considered tedious by the non-lab staff who would ordinarily do them — such as filling out forms in duplicate or triplicate (bonus points if you refuse to permit photocopying of some, but not others). When you have detected someone talking, call them in and tell them you really need this task done ASAP, and since you heard they have some spare time…

    Come up with a few more ideas like these, and apply them, and it won’t be long before your group actually polices itself.

    @7, bortedwards

    Is it my confirmation bias or the disproportionate number and size of medical labs, but are these stories of disfunction and allegations of blatant fraud overrepresented in the medical fields? If so my feeling would be that the cutthroat scrimmage by so many for relatively little funding has gone beyond healthy competition and the model is now broken.

    I think you have that backwards — the “relatively little funding” part, at least. After all, the drug company executives assure us that the high cost of drugs in the U.S. is because they have to fund research. Why, you don’t think they would lie, do you? 😇

    I would put forth an extra fillip: the medical industry has discovered that it takes a substantial amount of time for research fraud to be reliably uncovered. Fish oil? Toxic anti-cholesterol medications? Whatever. It will take 5 or 10 years for the studies to come out which disprove the effects or demonstrate the damage. Meanwhile, they can make hundreds of millions of dollars peddling the dubious medication or treatment. By the time the penalties are paid — and those penalties are typically a fraction of the profits made — the folks involved will all have retired or gotten new jobs elsewhere. I doubt that when, say, Pfizer gives a grant to Harvard Medical School they come out and say “we need you to find us an effect we can patent, we don’t care if it’s real or not” — but I bet after all these years of people switching jobs between industry and medical school administration they need to do more than just wink, and I doubt that lab admins need actual written instructions that say “the money keeping your lab equipped came from Pfizer, so go prove that their latest patent is good for people and falsify the data if you have to”. I’m only surprised fraud doesn’t happen more often.

  16. David Marjanović says

    Come up with a few more ideas like these, and apply them, and it won’t be long before your group actually polices itself.

    …I see, thanks.

  17. bortedwards says

    @16, The Vicar, lol, I appreciate the quip, and while I agree that compared to other areas of science (such as mine) medicine can still seem like the steroidal golden goose, drug companies still sprinkle their patronage sparingly and selectively (further exacerbating the backstabbly nature of the impoverished scrabbling at their feet).
    I think your point about the time-to-exposure being a potential motivation for dishonesty is interesting, although I’m not sure how it applies to coal-face scientific research. Surely those people would not *generally* be banking great personal fortunes nor moving on to other pastures within the window of grace. But then I have never understood the motivation for so many of these blatant frauds, especially high profile ones where exposure is pretty much inevitable.
    To my mind I see that under enormous immediate pressure to further a career/succeed in funding/gain fame people succumb to:

    – self delusion (*knowing* that you have the right theory and the data just needs to be nudged until it can be conclusively proven)
    – an innate human failing at assessing long-term consequences

    or am I being too generous at people motivations?

    Maybe the risks aren’t that great.
    I’d be fascinated to see a “where are they now” article on some of the people central to these frauds to see if it really DOES affect their career. Do they even lose their positions? Does it affect their grant prospects? Does the collective community forget quickly, fail to vet job applicants, or not even care when these people apply for new positions?

  18. bortedwards says

    * Hmmm… I should have done a simple search before suggesting there should be an article on the consequences of fraud. Naturally it has been looked into. Eg from http://arstechnica.com/science/2008/08/what-are-the-consequences-for-scientific-misconduct/

    “…37 of the scientists published an average of 2.1 papers per year before they were found guilty. After, they averaged 1.0 paper per year as of late 2003, with 12 publishing nothing at all. Thus, there was a significant decline in productivity, but a large portion of the scientists were still able to get their work accepted for publication. In terms of official punishments, over half of them received 3-year debarments from obtaining grants and contracts, and all 43 of them were banned from Public Health Service advisory boards for an average of 3.5 years. Despite this, 16 out of the 37 traceable scientists were still employed in academia.”

    not sure that’s a terrible deterrent…

  19. mikeyb says

    I’ve witnessed indirectly not fraud necessarily, but cases in which profs with labs have large contingents of foreign students on scholarship, and work them to death, weekends, holidays, sometimes 12+ hours a day for weeks on end, to raise publication #s, i.e. put their names on publications on the backs of the students. What leverage do the students have, complain and get sent back to their home country with no funding and no degree. I’m afraid this isn’t as uncommon as one might think. Unfortunately it works, high publication # correlate to respect in a given field, irregardless of how it is obtained.

  20. David Marjanović says

    high publication # correlate to respect in a given field

    Not just to abstract respect, but to success in grant applications and job applications.