It isn’t at all unusual for the authors of scientific papers to leave a comment at a blog discussing their work — it’s happened here quite a few times, and it’s a good thing. It’s a plus when they confirm what you’ve said or add more information to the discussion, and it’s also wonderful when they correct you on errors. I think most scientists are getting the idea that blogs are tools to help disseminate scientific ideas to a wider audience than the science journals can. They certainly don’t replace the journals, but add a way to inject the results into the public sphere, where they can be part of a popular conversation.
Sometimes you do find scientists who don’t quite get it. Dr Isis wrote a critique of a paper in the NEJM that reported a correlation between the change to daylight savings time and heart attacks; she thought the data was interesting, but the interpretations were sloppy. She pointed out some observations that were glossed over, suggested that some specific interpretations were a bit off, and listed some other articles on similar topics.
The authors took exception to the criticisms and left a comment. Unfortunately, it wasn’t the kind of informative comment, pro or con, that advances an argument — it was more of a condescending dismissal that ignored her comments and suggested that she needed to learn some basic epidemiology. And then there was this bit:
There can be many other explanations and pathways not written here (again we had a strict limit). The reviewers and editors agreed to our interpretation as probably the most likely one. So were experts in this field all over the world who commented our study so far. We would actually encourage you to write a comment to NEJM. NEJM is well known for its devotion for scientific debates on recently published papers. That would be a normal way to debate and discuss scientific findings. We would also have a possibility to answer on an “equal ground”.
The first part is a particularly annoying misconception. Peer review is only a first, preliminary hurdle for a paper to cross; passing peer review and getting published does not mean that your work is right. Some incredibly awful papers get through the review process, somehow. Getting published only means that now your paper is going to be opened up to wider criticism. Don’t take the attitude that publication means vindication; I know reviewers, and I’ve reviewed papers, and I know that reviewers are sometimes lazy, sometimes susceptible to croneyism, and always overworked, and that publication doesn’t mean you are right.
The last part shows that the authors have the wrong idea about blogs. A blog post is different from a letter to NEJM; it can reach a much wider audience, for one thing, and uses a little more stylistic variety than dry academic writing to appeal to a larger group (Dr Isis is definitely guilty of that — she uses humor, which is often sadly lacking in medical journals). I get the impression that the authors would prefer criticisms be made in the journal, not because it would directly target the best people able to understand the argument, but because it would limit the number of people who would see the disagreement.
The attitude that this is the “normal” way to discuss science is also aggravating. It is a restrictive view that contributes to the popular conception of scientists as aloof and unengaged with the culture, and it’s not true. We need to change the idea of normal so that talking about science over breakfast is normal, that having a conversation about science around the watercooler at work is normal, that guys at the bar get into arguments over science instead of football (sometimes) is normal. Change normal!
And finally, a blog is equal ground. The authors could have easily thrown in a few observations and explanations that supported their position, rather than treating Dr Isis like some ignorant nobody they could swat away by telling her to take a course in epidemiology. Explain your answers as you would to an undergrad or bright high school student. If you can’t, it implies that you aren’t looking for an equal opportunity, you are looking for a way to avoid probing questions.
(via DrugMonkey)
Also check out the Aerosteon saga series at the marvellous Sauropod Vertebra Picture of the Week for another incident of scientist versus blogger.
There are several more posts in this saga, and the comment threads on all are quite enlightening.
Oh yes
“If you can’t explain something to a first year student, then you haven’t really understood it.”
I use that as a self-test when I think I know something. Most often I fail for few months when I’m learning something new. I can only recommend.
Reminds me of the time I got a Distinction for a paper (or it may have been a HD) that I expected to get barely a pass for. I found that I couldn’t write it without humour. I seem unable to write anything without humour. Anyway, I ended up with what was basically an academic paper that took the piss out of itself and the subject matter, and out of the reader.
The person grading it commented that they relished the opportunity to read an academic paper that made them laugh out loud, without it being the spelling that made them do so.
I say we take off, take the piss, and nuke the site from orbit. It’s the only way to be sane.
It was for things like this that I made PZ Myers the subject of my most recent Scientist Appreciation installment. I think you do very good things for the advancement of science education and perception.
In case anyone is curious, you can find my appreciation post here.
Blogs are not equal ground in this case. Dr. Isis was making fun of the authors in her post and it takes an exceptional person to be able to come into an unfamiliar group, stand up for yourself, and not come off as either humourless (if you ignore the jokes) or an ass if you try to give it back (as the authors did in this case – I think the comment about taking a course in epidemiology was a failed attempt at meeting a snarky post with snark of their own).
Isis also has the ability to delete or write new posts that change the course of the discussion, as she did. Instead of discussing science she started a new post to complain about the author’s suggestion of writing a letter to the editor. In other words, because it’s her blog, she can pick up on a minor part of their comment and derail any scientific discussion. Again, not equal ground.
Not sure I agree with you. I do not know what paper or discussion this refers to, but I would point out that in your quote the authors do not state that the review process makes their interpretation the correct one, but the most likely one. That may or may not be true, but there is a difference between those two statements. I do, however, agree that reviewing a paper does not ensure that its interpretations are correct (nor, as we have found out so often in recent months, that the results aren’t all bogus). I also do not agree that a letter to the editor and a response by the author in a journal should or could be replaced by a blog discussion. Firstly a reader of a journal may never come across the particular blog in which the paper is being discussed, but would more than likely see a comment or letter appear in the next issue of the journal. In some cases such exchanges, particularly if they result in a retraction or an erratum will be indexed by data bases such as medline. If I need to find a paper I will use pubmed and not google and, consequently, I am more likely to come across whatever discussion may have occurred after the paper’s publication which may or may not be relevant to my own interpretation.
Journal articles usually focus on discussing and/or buttressing one or a few ideas in depth.
Blogs comments have virtually no focus, but this allows discussion of a multitude of ideas, to the whatever depth the commenters are motivated to dig to.
I’m not at all surprised at the trend to skim over scientific findings in blogs first, then dig into primary research only for notable points of contention, or to satisfy deep curiosity first stoked by a blog critique.
I’d be curious to know how the authors found out about the blog post – I seriously doubt they were reading the blog beforehand. Were they just trolling around the InterWebs, Googling their own names? If so, they deserve what they got! Surprise surprise, there are some opinions about you out there that are not in line with what you think about yourself – welcome to the world and the 21st century, now go back to your hole.
“Jane, you ignorant slut.”
It is fair game for anybody to criticize any paper in any format/venue that they want. Journal clubs and informal discussions with colleagues are also not “equal ground,” but papers are demolished in these ways every minute. I’d argue that a blog was a preferable format – at least you can respond point by point and you have longer than a walk down the hallway or an elevator ride to do so.
In the end, people deal with bad papers by ignoring them.
Personally I find Dr. Isis’s style supremely annoying; the whole high-snark self-aggrandizing pseudonymous-third-person thing just pushes my buttons. Were she to blog about the “crappy analysis” in one of my papers, I’d probably go off in a humorless fashion too.
Isis was more than a bit condescending in her “review”. I don’t think defending her is the right thing to do here.
I wonder to what extent I agree with that post.
The disadvantage of peer reviewed journals is that they are not funny, but on the other hand they guarantee the debate to be only at the level arguments, without help of sarcasm/irony/… that some can skillfully use to fills gaps in reasoning. It is a fact, that sometimes good style replaces good reasoning and that it is easy for the readers (including me sometime) to fall for it.
Also, the answer of the criticized authors is very likely to be lost in the middle of those from other “not so qualified to judge the matter” readers (who might get the feeling they are arbiter of the debate).
So, speaking about science on the net is nice, as long as it consists of qualified scientists teaching others. I might sound elitist here, but we can see how it looks like when the voice of scientists get lost in the middle of blogs : you have the so-called debate evolution vs creationism.
Sure it’s not all the high heels? :)
PZ writes (paraphrasing) that peer review is only the start of the process. Perhaps a semantic thing, but I always thought of the whole process as peer review, after all, the point of publishing is to get your ideas in front of the scientific community, who will consider, test, check and try to poke holes, in other words, review them. While I was never a full fledged member of the community (I only got my masters before I pursued other interests) this seemed to me to be a common view. Somehow in the intervening years, the term peer review seems to have been reduced to the portion of the editorial process that essentially checks that the paper isnt wasting everyones time. I guess Im wondering if the proper response isnt “peer review is only the start of the process,” but “the important part of peer review happens after you publish and all your peers, not just those two or three selected by the editors, get to review your work.”
So, speaking about science on the net is nice, as long as it consists of qualified scientists teaching others.
I haven’t read the original posts so won’t comment on the main topic, but Vincent – I think you are hugely wrong about this. Science is a set of mental skills that that everyone can and should use, and arguments ought to matter more than qualifications. I’d far rather that lay people err on the side of disrespect for scientists than that they consider themselves unworthy of questioning the higher-ups.
Oooh, are we allowed to ask questions?
Are we allowed to read other papers (as Dr Isis did) and bring them up?
What about questioning the relevancy of the work, are we allowed to do that?
If anything sounds (and is) condescending, it is your comment. I write a fair bit of science on the web, and, regardless of the reader statistics, I classify an article without a reasonable number reader questions/comments as a failure. And, no, mostly the discussion doesn’t generate into stupid evolution/creationism “debates.” In fact, one of the things I find interesting about this kind of writing is learning exactly which topics, why, and how they generate analogs of the “evolution response.”
“the authors” – “Don’t criticise our methodology in front of the acolytes!”
I commented on the letter in question on my blog (in Swedish, so don’t bother clicking around :)), but stressed the fact that 1:o, it’s only a pilot study screening of patient records and 2:o, it’s only published as a correspondance , not as a full blown article.
When reading the letter in NEJM those two aspects are also pointed out by the authors, as well as their ideas for further research based on the loose hypothesis of the effect being stress related. Isis seems to have taken the letter to NEJM as a bigger study than it really is, and I can understand the frustration of the two scientists when the critique goes beyond their small pilot study and into possible correlations and conclusions that can’t possibly be drawn. Pathways and biology aside, the main critique should instead be pointed on the screening method itself. To narrowing the screening to just two parameters, when the base data includes so many unknowns is to step into a marshland of possible outcomes.
That said, I must say that both the two scientists and Isis show a huge lack of humour and ability to discuss things in a civilised manner. Bloggers can be a big help to us scientists, pointing out problems in our reasoning or coming with fresh views on an issue. However, the blogosphere sometimes takes on itself to use studies to make statements rather than discussing the articles it cites. If Isis is sincere in her/his wish to help the scientists develop and expand their hypothesis, she/he should use a different tone. If, however, the goal is to discuss current science in a satirical manner, she’s/he’s spot on. But in that case it should be clearer. Bad jokes aren’t jokes, they’re just bad.
Peer review does not mean everything is perfectly correct, or that the conclusions will pass the test of time. The reviewer must essentially take what is written, check that citations say what the author says they say, maybe find some the author missed, and see that the methods used were appropriate, and that everything makes sense at the end. And be able to conclude that the paper advances the science the journal covers. The reviewer rarely sees the raw data, just the condensed version used for publication. So peer review is a very coarse sieve that catches gross mistakes.
Mikael Hiort af Ornäs,
Dr. Isis is a female scientist.
Pedantic moment of the day:
“…data was…” should be “…data were…”.
I agree that science over the breakfast table is a grand idea, and that blog discussion is ditto. BUT…
The gradation of blogs, from celestial to execrable, guarantees that some seriously stupid things get said and presented as proven fact. Most dangerous are the blogs that do this in a plausible way. Currently I find the ADHD deniers most infuriating (or their spiritual kin, the “cure it with diet and nature walks” cadre). Treatment delayed is treatment denied, and untreated ADHD leads to lifelong economic and social problems which usually can’t be fully compensated for in adulthood.
So, a blog discussion of serious subject must rest upon published, vetted and critiqued sources.
I personally find it frustrating to pursue a likely link or online journal source, only to find myself up against a pricey subscription wall. Grrr. And the price for one paper, or one day of searching that journal, equals the cost of a paper-and-ink good quality hardcover book. More grrr.
Noni
fuq gawddamn dst. the only way i’d vote for a repooplickan is if he or she promised to end dst.
“Explain your answers as you would to an undergrad or bright high school student. If you can’t, it implies that you aren’t looking for an equal opportunity, you are looking for a way to avoid probing questions.”
Good on you PZ! There are thousands of us newly emerged from the matrix of creationism. Come down from your high horses friends and tell us what you have seen.
SC: Thanks, I wondered about that. Scientist or not, my critique on her critique still stands, though :)
Matt: I know my English isn’t perfect. My own, strictly uneducated guess is that it has to do with the fact that English isn’t my native tounge. However, I appreciate the corrections. It’s the only way to learn :)
One important advantage that publication in a journal may present is its ability to focus the thread of the discussion in a place that people can readily find and follow. Posting discussion to blogs dilutes the thread to the extent that the public (lay or technical) can find and follow the discussion: not always an easy task. To that extent, the point of “an even playing field” is well made. An author can’t respond to criticism he/she is not aware of. Perhaps in the future, journals will consider opening a less-formal mechanism of discussion (e.g. a focused blog) that will allow/encourage informal debate yet still provide a known source for this debate to occur.
During my undergraduate honours year we were all required to spend time working as demonstrators in 2nd year lab sessions. We were paid but the point of it was brought home to me when a student approached me and asked me to explain how the action potential worked (the electrical spike when nerves/muscles fire). I trotted out the sort of answer I would in an exam, she replied, ‘yes, but how does it work?’. The standard explanation had not worked for her. So I had to find another way to explain it. I agree, if you cannot teach what you know how do you know you really know it?
Also if peer review is the last word why do so many people with newly published work speak about it at conferences and go on the road speaking about it? If you stand up in front of your fellow specialists your work is going to get a going over. Such a process should of course be welcomed and I fail to see why it should not work in a wider context. The idea that science should be kept from the masses would have astonished our Victorian forebears, those great showmen who loved nothing more than the public lecture. Remember Darwin did no publish evolution in Phil.Trans.Roy.Soc.London. He sold his book to whoever wanted to buy it.
An author can’t respond to criticism he/she is not aware of.
But why should they? You’re suggesting the equivalent of the authors running about, butting in on every hallway and class seminar conversation in which their paper comes up. Just because it’s on a blog doesn’t mean it’s a formal request for the authors to drop by. They did in this case, but instead of saying “Interesting points, mind if we address them?” they said “Stop talking about it. You can only discuss our paper in a letter to the editor.” That’s absurd. Every comment about a paper does not have to be funneled through the original authors or publication.
“We need to change the idea of normal so that talking about science over breakfast is normal, that having a conversation about science around the watercooler at work is normal, that guys at the bar get into arguments over science instead of football (sometimes) is normal.”
QFT! I want to see two jocks at a bar arguing heatedly over how many times our ancestors went in & out of Africa, a la “taste great/less filling” commercials. That would be so awesome!
I also think that “pop science,” whether by book or by blog, serves as a great way to bring science out of the ivory tower and down into the realm of public understanding. Brian Greene, Stephen Hawking, Richard Dawkins, and PZ here are all doing a great job of bringing these “hifalutin” sciency ideas down to the rest of us. (That bit on the platypus genome was fantastic, by the way!)
Mikael Hiort af Ornäs, I was actually referring to PZ’s incorrect verb selection with regard to the word ‘data’.
2nd paragraph, 3rd line of his blog entry. I should have made that clear. Sorry.
Matt: No problem. I’m used to be corrected on my spelling and grammar, that’s why I thought you meant me :)
“I get the impression that the authors would prefer criticisms be made in the journal, not because it would directly target the best people able to understand the argument, but because it would limit the number of people who would see the disagreement.”
I don’t share this impression at all
Isn’t it just as likely the authors wanted criticisms to get made in the journal so that the thread of argument (and, presumably, their resposnes) are part of the ‘official’ journal record?
Or that they think the NEJM’s restrictions (on, e.g., # of citations allowed in a letter) are annoying, and if Dr. Isis had written in with her critisim it would have given the authors a chance to let the editors know how their restrictions interfere with scientific communication?
Or that they are just really straight laced about certain taboo subjects and they could think about the critism more impartially if they had not be acompanied by the Roosevelt on the commode?
You are making a lot of assumptions about their motives.
I’ve followed the whole debate from the start. The authors have said several things I don’t completely agree with, but by and large they have been quite reasonable and even eager to engage in scientific communcation.
They stumbled into a larger favorite borg argument about the elements of scientific style. I think their statements came off as much more “stogy conservative” than they intended because of the pre-existing associations many bloggers have about how “the outside world” views blogging.
It behooves us all to remember that there is not one correct way to communicate about science. Even if you, PZ, do set one particularly elegant example… that does not mean you’ve gotten it “right”. Or, more accurately, it does not mean others have gotten it “wrong”.
Oh dear, what a b**** this Dr. Isis turns out to be! Her blog reads like that of someone with a way overinflated opinion of themselves. I hope I never have to work alongside someone like that, or I would end up slapping her!
Oh, and she steals images for her blog without giving credit…
Exhibit A – http://www.homebakedmemories.com/pages/HBKitchenLatteArt.htm
Exhibit B – http://isisthescientist.blogspot.com/2008/11/weiner-would-help-dr-isis-succeed-in.html
Speaking ex-cathedra:
In American English, data is a mass known:
The bread was yummy.
The data was execrable.
We ain’t speaking Latin here, otherwise you’d need to properly decline datum — and besides, we’re not using datum as a given, in the Latinate sense, but the derived sense of a body of organized observations.
I can out-pedant any of you little bastards! If you’re going to do that, bother to pick up a local dictionary first.
Actually, I feel fortunate enough to actually have true open discourse and dialogue on various fields of science with a good core group of friends. But it feels like such a tiny clique, and when others with less comfort in the sciences attempt to join the discussion, it seems to them that we’re excluding their voices. The fact that it would be great for a larger amount of people to join an interesting discussion, yet, finding out that their level of comprehension on the subject leaves them somewhat excluded. Makes one feel like an elitist bastard.
Shockingly, I am a first year student in Epidemiology at U of M-Twin Cities.
It’s pretty obvious that this ecologcial study may be subject to what is called the ecological fallacy – that group data does not always reflect on individual data. I haven’t read the paper itself, but I would be shocked if the authors suggested specifically the persons with MI were definately more sleep deprived, epidemiologists know much better after 6 weeks of intro classes. We know confounding is likely in ecological studies and for that reason the purpose of this study was to generate hypotheses about links, not attempt to prove causality.
Dr. Isis really would have understood that much better had she taken an introductory class in epidemiology, and perhaps she would not have so disgracefully criticized the authors for problems with their study that they were well aware of. The paper is a notable ecologic study, not an in-depth cohort with libraries of supporting evidence.
The only thing I ever argue with my friends about is the latest scientific discovery. :D
The paper in question was sensationalized by the media.
I question the authors true intentions if they had no problem with the ‘good’ (scientifically inaccurate) PR, and cry foul when someone completely outside mainstream media says ‘Wait a minute, not so fast…’. Who cares how it was phrased, the authors could have added something constructive in the comments, but they made a conscious decision to be butthurt.
Ive gotten emails from individuals after Ive reviewed their papers, + or -. No one has behaved like a snob.
Except Behe.
“Except Behe.” LOLZ
ERV,
I’m not sure it’s neccesary to criticize the authors intentions int his case. It is the fault of the media for not being able to accurately write about the science behind the article. They sensationalize it on purpose in order to sell more copies of their newspaper.
Studies like this, like I have said above, are useful for generating hypotheses about links between exposures and disease. If studies like this never happened, we might have never investigated further many exposures we now know are harmful to human health. Studies like this do not attempt to establish nor do they claim to establish proof of causality…it is merely hypothesized based on general group-level data, not individual data.
Pedant, heal thyself. I assume you meant “mass noun.”
As to your real point, you are staking out the descriptionist extreme. Sure, most Americans use the Latin plural “data” as a mass noun like “water” or “equipment.” That horse (like “agenda” and “media”) is long out of the barn, and nothing to be done about it but shrug and accept it as OK.
But in science, we use a lot of Latin and Latinized terms that do retain the singular/plural -um/-a forms. Cilium/cilia, stratum/strata, bacterium/bacteria, flagellum/flagella, phylum/phyla, et al. In a scientific context, “the data are…” is never wrong, whereas the more pedantic of my colleagues will judge you harshly for “the data is…” Here is an editorial (pdf) by tjhe editors of an international scientific journal on the subject.
But this argument pops up all over the ‘nets all the time and is never really concluded to anyone’s satisfaction. The last time I tried–somewhat in a Devil’s Advocate spirit–taking a hard-line prescriptivist position on the subject I got my ass handed to me, but I still think I was more right the wrong, in the context of science. Biology, at least.
First he published an “abstract” in the Proceedings of the Linnean Society (London) Vol 3 1858. pp 45-62. Then he rushed out the book for sale to everybody (1859).
Virgil:
*putting aside personal opinions about Dr I’s style*
BITE ME. (And by the way, you’re a suckass writer.)
OK, speaking of all this, I just got into an argument with a guy in the break room over whether octopi and crabs are fish or not. This started because he claimed that only three things lived in the sea: fish, amphibians, and mammals (he later conceded plants, but refused to classify bacteria as life because they could not assist in tying down Gulliver). I knew that arthropods, molluscs, and chordates are distinct, but he didn’t believe me and I basically had to Wikipedia it all just to prove my point.
So yeah, arguing taxonomy for the win.
…besides which, there are no “amphibians” in the sea.
No amphibians in the sea?
I followed the “controversy,” and it seems they were offended by Dr. Isis’ humorous and flippant style. After all, she captioned a picture of a teddy bear on a toilet claiming it represented their analysis. I’d be offended by that, probably.
But questions of humor aside, her blog really isn’t a level playing field. It’s HER blog. I’d suggest the authors start their own blog to have a discussion about their research. There they can cite as many sources as they want, and wax quite poetic about their hypotheses and analyses. Then they can meet Dr. Isis’ (and everyone else’s) criticism head-on.
I didn’t think they were dour, humorless researchers. Just a little nonplussed and perhaps somewhat offended. Blogs are normal for some of us. They just might be a tad behind the curve.
Rana cancrivora has remarkable salinity tolerance, but is not marine in habitat. Brackish mangrove swamps != “the sea,” to me.
I don’t think that her blog is a level playing field.
1. How are the authors even suppossed to know about her criticisms?
2. Everyone reads the posts, not everyone reads the comments.
I agree they should have adressed her points instead of just telling her to take a course. After that they should have told her to take a course.
Oh, I know! And what about people who discuss the paper in the lunchroom? How are they supposed to know about that, either? Or those meanies who talk back and forth about it over email? Really, this kind of discourse should be allowed only in the official designated forums. Otherwise, it’s chaos (and so unfair!).
It is the fault of the media…
You mean mediums, of course.
I don’t think that her blog is a level playing field.
It’s not a playing field at all. It’s her blog. That’s the point. Her blog, her space, her rules. It’s not a public domain. It’s her blog.
My thoughts exactly! The authors got a bit huffy at the criticism so it’s easy to ridicule them, but it’s a good suggestion. Why is everyone so quick to jump on the authors?
I’m not sure what Dr Isis’s criticism has to do with the result, really. Pointing to an endogenous circadian rhythm does not refute the existence of other variables. And saying that people can be sleep-deprived for other reasons besides work, like taking care of a baby, does not negate a weekday-related effect.
If the woo merchants set up a journal, and appoint a number of their own as referees for the papers published in their woo-journal it won’t make their published papers anymore worthy or merit than their unpublished ideas were previously. As PZ says, the quality of the journal is dependant on the quality of editorship and referees, both of which are not always good in the conventional scientific world. When I was an active writer of papers in the physical sciences it was common knowledge that some journals set less severe hurdles on paper review than others.
My conclusion? A theory has reached a state of maturity in the eyes of the author and scientific community if it gets published – then its fair game for its potential opponents to tear apart. And the more forums used for this the better. But clearly forums with scientists with a vested interest in that research are the most appropriate if the criticisms are to do anything constructive.
Jesus fucking christ! What kind of violent misogynistic sick-fuck bullshit is this!? Get yourself some motherfucking counseling before you hurt someone, you deranged fucking scuzbucket!
35=pot
57=kettle
#58=dutch oven. HAHAHAHAHAHAH!!!
You speak lies, CPP. 58=Lemon Zester. Everyone knows that.
People who hide themselves in the Authority Principle are generally despicable.
Becca, #58
I’ve been surprised by the number of closet conservatives showing up in this thread. Surely a liberal would not talk as Comrade PP has done.
And aint it amazing how Dubya has shrunk government over the past eight years like a good Republican should.
Alan Kellogg- this lemon zester pals around with socialists, but is still confused by generalized efforts to determine voting behavior via kitchen equipment.