Unfortunately. What that means is that an endeavor that ought to be impartial and based on reasoned evaluation of the evidence is tainted by bias and unavoidable cultural preconceptions. We’ve got religion turning some people into credulous twits, but just as poisonous, we have sexism skewing our analyses.
The first thing we did was look at more than 3,000 articles published between 1980 and 2006 in 12 leading peer-reviewed international relations journals. We then controlled for every possible factor that could contribute to one’s citation count including the quality of the publication, its venue, methodology, the subject matter, and the researcher’s home institution (to name a few). We suspected that an article written by a tenured professor from an elite university, published in a top journal and written on a popular topic would get more citations than an article written by an untenured professor at a liberal arts college on an esoteric topic in a second-tier journal. What we didn’t know was whether gender would matter once you held all of these factors constant. Did knowing the gender of the author make other scholars cite an article more or less?
The results were striking. Even when we controlled for an enormous range of factors, gender remained one of the best predictors of how often an article would be cited. If you were female, your article would get about 0.7 cites for every 1 cite that a male author would receive.
This paper has garnered a lot of press here, here, and here, not because it’s telling us something we hadn’t already suspected but because the data are incontrovertible. Crunch the numbers in different ways and the results are always the same: articles written by women in IR are cited less than men, all else equal.
The authors of that study have some productive suggestions. One is anonymous review: publishers should mask out the authorship and affiliations when sending papers out for review. You’re judging the work on its own quality, right, so who wrote it shouldn’t matter. I do something similar when I’m grading papers — I refuse to look at the students’ names until I’ve evaluated the whole thing.
This would also diminish that other unfortunate bias, judging papers by what institution they came out of, rather than their content.
Another suggestion is simply to have first and middle names always reduced to initials. That’s not a perfect solution, but it helps. (It doesn’t help if you’re already known by your initials, but that’s a different problem.)
I have another suggestion: maybe graduate students should all get some kind of education in equality as part of their training, so they don’t go on to be bigoted asshats when they go on to full science careers. I’ve heard it all: prejudice against women, against blacks, against Asians, against historically black colleges, against liberal arts institutions. Maybe scientists should learn not to pay only lip service to that scientific virtue of objectivity.
You could gather every scientist/CEO/college dean/politician/hiring manager, have them SCREAM in unison into the largest bullhorn known to humanity, “We don’t hire/publish/promote women/Mexicans/disabled people because we JUST DON’T LIKE THEM” and people will STILL say it has nothing to do with sexism/racism/whatever.
In other news, AT LEAST this guy (and the Alex IS a male Alex) seems to get it right. The comments are your typical dudebro, can’t-take-a-joke bullshit though.
Apart from non-English gendered surnames or for papers in languages that gender surnames. As I’m sure Mrs Myersová would agree.
I think addressing this one is going to take more than just a little equality training. I’d hazard the guess that this bias may be, at least partially, a subconscious reflection of our patriarchal culture in general. That is, more a sign of a broader syndrome than some endemic misogyny in academia.
I’m fully in favor of the fully unidentified source idea, but as with any ideal solution, there are always complications.
I’ve always dreamed of the impossible ideal of an identity-blind criminal justice system, where technology could make both victims and defendants to be perceived by each individual juror or judge to be the same ethnicity and gender (and maybe age) as their own. Of course this will pretty much always be impossible and fraught with all kinds of issues I haven’t even considered, but I still see this as the ultimate way to at least minimize the horrifying effect of those biases in the justice system.
We need to fight the cult of “objective rationality” that breeds overinflated assholes like Thunderfoot.
Because you don’t become magically rational or objective because you have a science degree.
Just look at all the bullshit the “fathers of science” believed in.
And there’s actual research that suggest that the more “rational and objective” people believe themselves to be, the more bias they’ll show.
#4 PDX_Greg: What about cases such as hate crimes,? In such cases color/sex/whatever-blind trials would be a hindrance to say the least.
“Witness, when Person A was smashing Person B into the pavement, was he saying…a specific slut that applies to one of the plaintiff’s biological or cultural characteristics?”
That should be “slur”. Damn autocorrect.
It depends on the size of the field. I know, for instance, that ‘planetary rings’ is small enough that unless the writer is an outsider or new or especially reclusive, many people will know him or her, and may even be able to guess from the sub-topic and the writing style for a blind review.
It would be easier to in a large field where everyone doesn’t know everyone else. (I don’t know how focused various areas of biology are.)
is it bad that at this point I’m considering a unisex name for any hypothetical spawn (that, plus sticking them with the last name I currently have, rather than my dad’s or my current partner’s last names, because theirs say “furriner” and mine sez “WASP”)
*sigh*
everything sucks
Maybe irrational brute force is part of the answer. Unless a paper written by a woman is cited, force authors to submit a list of papers authored or partially authored by women “that [the submitter] considered while constructing the paper’s citation list.]”
A while ago, the NFL helped the problem of no/few Adrican-American coaches by imposing a requirement that every time there was an opening for a head coach in the NFL, AT LEAST one African American be on the interview list. No other requirement, just interview at least one black coach. pH
So, maybe we should pass laws that require equal citations? I can just see it now, “Ladies and gentlemen of the jury, this author stands accused of citing more male-authored papers than female-authored papers….”
That works in large fields. But in small ones… I once submitted a paper to a journal that does double-blind peer review. The manuscript didn’t contain my name, my coauthor’s, our affiliation or any such stuff. Naturally the angry reviewer recognized us from our arguments and likely our style; and the same way we’re quite sure who he is.
In large fields it does. In small fields, see above. Plus, this would exacerbate the increasing problem of homonymy – there are two Andrew R. Milner in vertebrate paleontology –, and keep in mind that not everyone has a middle name. I don’t, for instance.
And that’s before we get to gendered surnames – kept, BTW, in former Soviet republics whose national languages don’t even distinguish “he” from “she”.
And then you move, and all the advantages become disadvantages, cats & dogs living together, madness. For instance, Germany has a law that requires everyone to have at least one gender-specific given name. Many Turkish names are unisex – all the people this concerns apparently had to adopt a middle name. *facepalm* Similarly, your current last name screams “foreign” to much of the world, though admittedly WASPs are not Acceptable Targets in any place you’re at all likely to end up in.
Wait… peer review isn’t already anonymous? This seems like something that’s just obvious. Keep the reviewers from knowing the authorship and source institution until after they’ve made their critique. Just… duh? Why the hell is that not already a rule?
Not how it works.
Frankly, I’m disappointed anonymous review isn’t already the norm. Not that I’ve been doing this academia thing for very long, but whenever I send out papers for conferences, most of them require you to remove your name and affiliation from your submission, as well as any obvious self-references in the body itself (such as, if you’re D. Smith citing your own paper from 2010, word it like “As shown by Smith, et al. (2010),” instead of “As we previously showed (Smith 2010)”). Seems fair to me, and like it should help reduce gender/racial bias, as well as bias toward established big names over talented new blood.
Reviewers are by default anonymous, though they can and often do opt out. Authors are usually not anonymous – depends on the journal.
I said above that I once submitted a manuscript to a journal with double-blind peer review. More recently, I submitted another manuscript to another such journal, the only other one in my field I know. My coauthor and I had to rescind the privilege of being anonymous, because that paper was so blatantly an update of an earlier paper of ours, which of course we had to cite heavily, that trying to anonymize the manuscript would have been both a lot of work and ridiculously futile. We’d basically have needed to try to pretend we’re other people updating those people’s paper, which is too close to lying for comfort, and then it’d still have looked really suspicious.
I’ve never seen a conference that told authors to remove their names. Granted, most of them accept all contributions, though one has become pretty ruthless (because it gets so many hundreds of submissions every year).
This one, however, doesn’t allow you to put any references into your abstract. Put one in, and your abstract is immediately rejected for failing to conform to the prescribed format.
This is only going to get you part of the way, though. There are all sorts of ways in which people who have no intrinsic bias themselves can subtly contribute to the problem. For instance, I suspect there is a positive feedback effect at work here: Papers that have already been cited many times are more likely to get future citations. So even a slight gender bias is going to self-perpetuate.
well, a German last name is not one of the available options at the moment, anyway. All I got for options is Polish, Scottish, and Metis (which in Europe would be read as French)
I have never seen a conference anonamyze submissions either.
The subdisciplines I work in are so small that most of the major players know me and recognize my writing style and work. And vice versa. Recently I got into a technical argument (via the editor) with a reviewer who clearly knew who I was because they were using a female pronoun to refer to me in their review (which was supposed to be anonymous for a book chapter) and was being patronizing. After thinking about it a little, I figured out who it was myself (don’t know him directly but by reputation and through others). It was all I could do not to call him up an argue with him over the phone – but I was professional – and I won because I was also correct. I have wondered if I would have to put up with the same shit if I was male. Keep getting told “but everyone is harsh in reviews.” Um…
what
@12, David Marjanović
“For instance, Germany has a law that requires everyone to have at least one gender-specific given name.”
That used to be the case, but was ruled unconstitutional a few years ago. If you can speak German, the relevant paragraph is No. 15.
This kind of progress tends to surprise me when it happens in Germany. I’m so used to Germany being socially very… traditionalist/conformist.
“This kind of progress tends to surprise me when it happens in Germany. I’m so used to Germany being socially very… traditionalist/conformist.”
I don’t know, I feel like we’re getting better in that department. For example, we’re the first European country to introduce a third gender option on our birth certificate, acknowledging the existence of intersex people. Then again, since we’re talking about names, there’s a measurable effect that leads to teachers here judging a child’s intellectual capacity by their name – ie, treat children with names typically associated with lower class people worse. Money quote: “Kevin is not a name, it’s a diagnosis.”
that’s probably true, but it still feels very inflexible. It feels more inflexible than U.S. culture (the conservatism here is more flavored with ignorance than rigid traditionalism, AFAICT)
I’m jack’s complete lack of surprise. The same thing has been shown to be true elsewhere, but IIRC usually with race as a more dominant marker than class.
Why are anglophone names considered low-class in Germany, btw? I don’t remember that from when I was living there…
I am always amazed at the mental calisthenics people do to avoid recognizing gender-based bigotry.
re: that court-ruling mentioned above, I’m so not surprised that the bavarian court tried the “won’t somebody think of the children” argument. where the hell did they get the silly idea that having a clearly gendered name is to a kid’s benefit, though? Because I suspect it’s from their rectum, not from any research on the topic.
Also: “natürliche Ordnung der Geschlechter” is giving me eyeroll-strain (translation for the non-German-speakers: “natural order of genders”)
If only dudebro programmers would remember studies like this one, and the one about symphony orchestras choosing more women after auditions were done behind screens, then next time they’re insisting that the tech world is a totally equal meritocracy, and “all that matters is how well you can code.” That may, in fact, be what managers have in mind consciously, but there’s just too much evidence now that other people’s perception of “how well you can [code/play violin/etc.]” is biased.
“Why are anglophone names considered low-class in Germany, btw? I don’t remember that from when I was living there…”
It’s a fairly recent development, ~15 years or so, so the children affected by it are just starting to enter young adulthood. It’s likely the prejudice will die once they’ve fully entered the workforce and are actually interacting with the population at large, becoming normalized. There was a surge of children being named after popculture references, ie Hollywood movies and television shows, and the attitude is that this practice is something more ‘dignified’ (insert eye roll here) people don’t engage in.
There’s generally a trend now of people naming their children more exotic (read: foreign) names, often double or triple names, and there’s a faction of the population that relentlessly makes fun of it.
Oh, *phew*
(“Im Namen des Volkes”? Still? In Austria, that was apparently deemed too close to the other thing, and we’ve used “Im Namen der Republik” ever since.)
I didn’t even know there were male Heikes out there. I thought Heike was female (even had one in my class in elementary school) and Eike male, funny as that is.
What! :-) You’ve had a red-green government! :-) We haven’t even managed that much, indeed we haven’t managed to make it mathematically possible! And right now, a red-red-green coalition would be mathematically possible in Germany, while Austria’s possibly comparable other red party got 0.2 % of the vote.
And no mention of Jennifer and Jessica and Jacqueline? ;-( Oh, and Dennis!
Really? It was definitely a thing around the time you left. The idea is that these are fashionable names, therefore given by stupid people who run along unthinkingly with the latest fad, and name their kids after Hollywood celebrities (Kevin Costner… more recently all the way down to Justin Bieber).
Oh God, Kiara on the 3rd page. That’s from The Lion King II.
8th page:
Mike Anakin
Flay-Emilia
Kleo Patra
I have to stop. It’s addictive.
*snickers*
My personal favorite is Shanaya Shirin Jolina Caprice. It’s like they’ve taken the old German aristocratic tradition of people having half a dozen names, mixed it with Hollywood and created an abomination. Oh yeah, and you need to say it in a thick german accent.
Disclaimer: No, I don’t think there’s anything wrong with naming kids any of these names. …individually.
that has nothing to do with social conformism though.
This wouldn’t alter citation bias as much as publication bias, which was accounted for in the study. Nonetheless, I think it’s a good idea, even if in practice anonymity can’t be guaranteed.
Same as above.
My lab group will be discussing this study in our next lab meeting (that doesn’t already have reading scheduled).
________________________________________________________________________________
DDMFM: I once had a reviewer blow his anonymity by calling me on the phone (transatlantically) to complain about how much he hated my manuscript. I had spoken once or twice to this individual before, but not on what you would call friendly terms. Were I to reveal to you the name of this reviewer, you would be unsurprised :)
People put the names of their children on their car windows?
?????
I have a hard time taking people seriously whose criticism of a name is “how do you pronounce that?” Such provincialism.
oh the noes, a Polish name.
yeah, Germans are as big a set of assholes about names as Americans are. Not surprised.
SOLUTION: deputize non-human animals to do science!
@17, 20:
Hmm, maybe it’s just my field then (computational linguistics/natural language processing), or only the conferences I’ve submitted to thus far (including ACL, the biggest worldwide computational linguistics conference). The guidelines were very clear about anonymizing, and they didn’t look at the name attached to the online submission until they sent their decision.
I would be interested to see a similar study done for scientific journals rather than IR journals, and to see whether the position of the author’s name in the list of author makes a difference. Different disciplines have different traditions/conventions regarding the position of the senior authors on the paper. An interesting control would be authors from countries such as China and Korea where it is much harder – at least for westerners – to determine gender from names.
I agree with others above than anonymizing manuscripts is a good idea in theory, but harder in practice, and very hard to apply in a fair way across fields. As we move forward with open access publishing and “open science publishing”, in which the community collectively reviews papers after appearing online, some of these issues will change, although they may be replaced with new challenges.
http://blog.f1000research.com/2013/04/25/storify-of-challenging-the-science-publishing-status-quo/
http://blog.f1000research.com/2013/05/10/video-from-open-access-to-open-science/
Tangentially related: last year, my end-of-term project for stats class was the following:
(1) Take 3 sequential issues of Science (chosen because it is high-impact and broad-spectrum) of the current year.
(2) Pull the research articles in those issues.
(3) Determine if the first and corresponding authors were male or female.
Given that women are ~51% of the species, a rough parity should have been expected. Perhaps the corresponding authors would have been disproportionately male (due to institutional sexism in days past).
HA!
The corresponding authors were 68% male, 15% female, 18% gender-neutral-and-unfindable-by-google.
The first authors were 61% male, 17% female, and 22% gender-neutral-and-unfindable-by-google.
Even if those 18% and 22% were all women (unlikely), the male:female ratio is at best 3:2.
*sigh*
johnradke @39
This is obviously the way forward!
I’m shocked, just shocked, that scientist are just as bigoted as the rest of the population… Or not since I’m not aware of special gender/non-bigotry training they get in school and we see enough bad science coming from scientist that even non-scientist like me can tell the logic falls apart. It’s like the medical field where most of the testing of drugs is done on males in a certain age range & they can’t figure out why the drugs don’t work as well on women or older people. Gender bias is not going anywhere in any field until we address it as a society. Holding any group to higher standards because they “should know better” if they aren’t getting lots of specialized education in anti-bigotry during their education and using specialized tools to debunk their biases is being unrealistic IMHO.
Esteleth @42
I’m not terribly surprised you saw that, in part because (in my experience) corresponding authors are most often the PI or group leaders which are still overwhelmingly men. This is true even in biology where > 50% of PhDs, especially in experimental disciplines, are women. I’m at a shiny new institute now, where all of the PIs are young and all newly recruited. Of 12 research groups, only 2 are headed by women and only one of those is experimental. You should expect much closer to 50% in this age group. My last institute was even worse, one 1 woman in 12 (and they’ve just pushed her out) and they never even tried to recruit any. They offered new positions to buddies, or previous post-docs, which apparently were never ever women.
It’s discouraging sometimes.
Sure, MadHatter, I expected the corresponding author (usually the PI) to be disproportionately male for that reason. But for the first author – the younger grad student or postdoc who did the day-to-day stuff – to also be disproportionately male? That, I will admit, threw me somewhat.
In many fields, anonymous review is almost impossible. Fields are very narrow and expertise highly localized. Back in the day when I reviewed physics papers in my specialty, I almost _always_ knew which researcher(s) the paper came from without ever needing to see the author list.
Personally I would put this paper in the using-statistics-to-promote-a-pre existing-hypothesis pile. In the first place I’d have liked to look at the raw data but it doesn’t seem to be included even in the supplemental material. Table 1 though indicates that the data is definitely not normally distributed as the means are much higher than the medians, especially for the all male authors, possibly less so for the all female or coed authors. The medians for all groups are pretty similar, maybe a bit higher for the coed group. So 50% of all papers regardless of the gender of their authors receive more than 5-10 citations and 50% receive less. Reported like this though the data would be considered not terribly exciting nor likely to generate much discussion.
Fun fact: Austria has a law that limits people to three given names. Maybe it’s part of abolishing nobility complete with its indicators like “von” & “zu”. The last emperor’s son was officially Herr Dr. Habsburg.
:-o Wow! That’s hardcore!
I can’t think of anyone, though. There’s one who might do this, but he’s closer to my field than to yours, AFAIK.
I’m sure that name simply wasn’t recognized as Polish (one of the comments points out it is), but was instead perceived as a very creative spelling of the German version.
Oh. That’s very far from my field. We’d probably have 3 culture shocks each if we met. :-)
Very true.
Good point!
I’d expect a huge bias towards men in last authors, in those disciplines where being last author is a thing. In those disciplines, like mine, the corresponding author tends to be the first author, because the one who did the actual work is often best qualified to explain it.
bwuh –
@#6 and 7, karley jojohnston
Yeah, I hear you. There are definitely issues with the idea. There are times when the race and/or the gender or the defendants or the victims are important enough to reveal, such as in cases where the race and/or gender (or any other “technologically masked factor” in my impossible scenario) is associated with a pattern of oppression, abuse, hatred, fear, or entitlement. Thus, it is not a perfect or workable solution, just the only way I could think of to eliminate as much adjudicator bias as possible.
Seems like the journals should pull their heads out and have authorship be anonymous during the review process. Go ahead and publish the paper (if it makes it through review) with all the authors’ names emblazoned across the page after review, but during it, have the authors and institutions be unknown to the reviewers.
Now in your example of a paper that couldn’t be anonymous because it was a blatant update to an earlier paper, then… that’s what happens. You withdraw your anonymity for that paper. Seems, again, obvious to me.
I still just can’t wrap my head around the idea that peer review for science journals (I’m an undergrad, so bear with me) isn’t double blind anonymous. That standard has existed in medical studies (that is, during the study, not the review) for decades, so why the hell wouldn’t we apply it in science journals? Eliminating bias of this kind is always a good thing.
This is a footnote from the non-paywall-bound Maliniak et al. MS.
I wonder if transgender scholars, maybe being more aware of gender issues in general, would not exhibit citation bias.
Just a polite request: can we stop with the presentation of names that are “inherently funny”? Because none of us actually know whether someone reading along carries one of those names. And I’m not sure we really want to become part of the name bullying, do we?
One reason, alluded to above, is that although you can anonymize the authors, you cannot anonymize their work. For example, I work in an extremely small field – the development of the inner ear – and have worked in this field for over 15 years. So if a reviewer reads an anonymous paper on work that builds on extremely similar work published by my lab over the last five years, it is highly likely that they will conclude the study under review was done by my lab – especially since preliminary accounts of the work will probably have already been presented by my lab at conferences prior to submission of the paper. So while anonymizing manuscripts is certainly a desirable ideal, it is extremely hard to implement fairly in practice.