Or: wherein Stephanie Zvan shows us little folks exactly how we can step in and bloody the nose of a bloody bigot with a PhD.
In case you haven’t heard of this ongoing debacle, Dr. Satoshi Kanazawa recently published a rather controversial article claiming that black women are objectively less attractive. This study was published in Intelligence, a journal well-known for its persistent use of IQ as a valid measure of intelligence despite the academic dissent that IQ does not measure any one single thing and therefore can’t be used as a metric to study anything but the weak causal relationship IQ scores have with actual intelligence. When Dr. Kanazawa was presented with a good deal of dissent about the methods by which he produced the study, he went on to blog on Psychology Today about the study’s validity, claiming some interesting just-so hypotheses to explain why his results were correct, rather than engaging with the criticisms. The blog post was almost immediately retracted and Psychology Today apologized for the distress it caused.
This touched off a firestorm, mostly in that Kanazawa evidently has a history of not engaging with critiques of his papers contemporaneously. A number of scientists rallied to his defense, claiming that Kanazawa was “Sinned Against, Not Sinning”. There’s just one problem with the defense rallied: the defenders claimed that any critiques must needs be made in the journals themselves, and once past peer review, the paper is beyond reproach.
Oh, sorry. There’s just TWO problems with the defense. Stephanie Zvan points out the other with much relish (and you people had better bloody click through to that link!):
There are legitimate discussions to be had on the role of peer-review feedback in shaping the final published product. However, having that discussion and recasting a complaint about Kanazawa’s resistance to incorporating feedback are two very different things. Also, given what the criticism of Kanazawa actually was (that he doesn’t interact with feedback prior to publication) it seems a little odd to note that he incorporates feedback into later work. If the criticism is important enough to be dealt with, wouldn’t he produce stronger papers by dealing with it up front?
But back to the letter. There are a few short paragraphs providing information about two times Kanazawa later responded to criticism, followed by this closing:
Finally, we believe that the proper place to make criticisms of academic papers is in the journals in which they were published, not in letters to the press where they cannot be adequately answered.
Sorry, Stephanie, I have to interject to say: are you fucking kidding?
Okay, go ahead.
This–this!–is what makes this letter so entertaining. Even forgetting that Kanazawa brought himself and his work into the general public eye by writing a blog post about his “findings,” this is the richest vein of irony I’ve mined in some time. You see, while the idea that scientific ideas and their validity should be hashed out in journals is relatively common among scientists, it’s pretty rare among the signatories to this letter.
Oh. Wait. Turns out she wasn’t kidding, they actually said that. Stephanie even got published in The Journal of Are You Fucking Kidding, as though to underscore my disbelief.
She goes on to list an easy pickings set of links that show times when each signatory to the defense letter actually blogged about science in public, in direct contrast with their professed beliefs. I personally see no harm in blogging about science, engaging with your audience (and in many cases, with audiences that aren’t actually normally “yours” to begin with). It gives you perspective you might not otherwise be exposed to, and can oftentimes provide a baffle against the temptation to insulate yourself into an echo chamber. What I DO see harm in, is in ignoring valid criticisms outright, especially when they’re coming from people with as good of credentials (or better). Simply ignoring criticisms and carrying on as though your work is totally valid and the points they’ve made so utterly incompetent as to not merit consideration is galling. It’s the type of thing you see when someone has an unfalsifiable belief and they move the goal posts right in front of you when you provide them with evidence that they’re wrong.
Engaging with your critics and surmounting their criticisms is a fundamental part of the scientific process, and I can’t help but think that your science would come out all the better for it if people point out the flaws and you amend your work to compensate. You know, amend your CURRENT work. Not simply “incorporating the dissent” into future works. Especially when those future works are also apologetic to a cause you’re evidently trying to advance, despite precious little valid data to back you up.
Stephanie’s list of links also has a bit of a secondary trend, which I’m sure is not accidental. Each of the blog posts she links to seems to have a fairly controversial bent, regarding all manner of things from eugenics to speeches in front of White Nationalist conventions to the “perils of diversity” to defense of sweatshops. The common theme to all of them appears to be a generalized defense of racism. Considering Kanazawa’s paper, considering Kanazawa’s already controversial history, and considering the vast criticism leveled against his academic practices, the defense paper’s purpose is all too transparent: protect one of your own.
One question that Stephanie raised piqued my interest: “Someone for whom impact factor is a big deal will have to do the research on whether the letter writers are correct [in asserting Kanazawa's been published by many high-impact journals], but I would love to see the results.” As she and I both point out, Intelligence is fairly high-impact, but also high-controversy — it caters almost exclusively to people who believe IQ is actually worth something. It will therefore be cited very heavily by scientists who believe likewise. This may or may not be a self-feeding subculture of scientists, who may or may not be engaging in an amount of cherry-picking, bias, or other scientific fallacies that depend on people desperately wanting to be right even at the cost of parsimony with reality. It is akin to scientists in the Creation Science field, wherein people presume Goddidit and the science must flow from that initial premise or it is out of orthodoxy with their subculture.
I’m working on finding impact studies for each of these journals in which Kanazawa was published. I found an Excel spreadsheet of journals from 2007 with their Thomson Reuters impact factors, but his papers span from 1992 through 2011, and it would be unfair to provide a snapshot view of the impact of these journals in only 2007.
If I can’t find anything more recent (e.g., if nobody provides me with a login for the current Thomson Reuters Journal Citation Reports tool), I’ll put together a follow-up blog post with the numbers from 2007, with a scale as to where they fall in the “impact factor” for that trade. I might also have to eliminate some of the top journals in the field, as a number of them appear to act as aggregators and get disproportionately high journal impact which would skew the point I intend to make: that the journals Kanazawa is published in, are not in fact “high-impact” by any reasonable standard as implied by his defenders.