For some reason, over the weekend, Jerry Coyne asked Steven Pinker to discuss some brief blog comments for publication and Pinker did. The blog comments in question were dropped by a busy and sleep-deprived PZ in response to someone jumping on an even more brief description of the evolutionary psychology panel that I moderated at CONvergence/SkepchickCon.
Rather than listen to the audio of the panel–which is difficult, yes–Coyne decided it was best to take PZ’s informal summary to Pinker. The results…well, the results make me very happy both that we structured the panel the way we did and that it was recorded for posterity.
There’s really not much point in discussing most of what Coyne and Pinker have to say before the video for the panel comes out–probably not until it’s been transcribed. What we had to say already addresses many of their objections. So those can wait rather than us doing additional work.
There is one statement of Pinker’s I wanted to touch on now, however.
M [continuation of previous sentence]:. . . and never actually look at genes and for that matter, ignore most human diversity to focus on a naive typological simplicity that allows them to use undergraduate psych majors at Western universities as proxies for all of humanity”
P: It’s psychologists, not evolutionary psychologists, who focus on Western undergrads –field research and citations of anthropology are vastly more common in ev psych than in non-ev-psych. PZ is engaging in prosecution here, not analysis – he’s clearly ignorant of the sociology of the fields.
As for diversity – is he arguing for genetic differences among human groups, a la Herrnstein & Murray?
No, he’s not, which is one of the reasons it might have been better to wait to comment until after viewing the panel.
As for Pinker’s argument here, I’m underwhelmed at best. First of all, even before critiques of cultural insularity were a big thing in the general population, I was taking classes in college on developmental psychology from conception to death in old age, as well as classes in non-normative psychology. I won’t tell you that the educated American white guy wasn’t the standard against which everything else was measured, but psychology was looking well beyond college students. Additionally, the critiques about how well results could generalize were already coming from within psychology, as my methods class demonstrated well. My senior research project involved replicating a study that had been done on a university population (not just students) in the general public.
Secondly, even if psychology in general were worse than evolutionary psychology in this measure (it probably is on citing anthropology; I wouldn’t know where to start for meaningful comparative data on field research, but there is quite a lot of it in psychology), that is not a scientific defense. It could be a tu quoque, except that none of us criticized evolutionary psychology by suggesting we like vanilla psychology better. In fact, PZ was comparing evolutionary psychology to evolutionary biology.
In any case, problems in psychology research do nothing to excuse problems in evolutionary psychology research. If you want to make a scientific defense of evolutionary psychology along those lines, you need to explain why these practices don’t prevent us from relying on research outcomes. A scientific defense of heavy use of Western college students would be that these studies are essentially pilot studies, that they inexpensively weed out negative results, and positive results prompt more expensive field work in other, very different populations. Unfortunately, that doesn’t seem to be true.
No, I’m not just relying on my personal impressions for this. After seeing Pinker’s response here. I decided to look at the evolutionary psychology literature as represented by its eponymous journal Evolutionary Psychology. I chose it because it is open access. I was able to read beyond the abstracts for every article to determine how samples were drawn. This is also the one journal I know of that limits itself to evolutionary psychology instead of including things like behavioral ecology or more general anthropology. That meant I didn’t have to code for type of research being presented in each study.
I started with the 2013 issue. Then, concerned that publication cycles might make me miss an issue in their favor, rich with studies done in non-WEIRD (Western, Educated, Industrialized, Rich, Democratic) populations, I kept going through all of 2012. I ended up with 105 articles. Of those, 61 were studies. The remainder were mostly book reviews and literature reviews. I excluded one study, as it involved computer modeling, leaving me with 60 studies.
How many of these studies were done only using college students? More than half. In 33 studies, the population whose preferences were used as a proxy for human universals was a population of college students. Another six studies used a combination of college students and other populations. One of these additional study populations was young, educated Israeli adults. Two were populations from around the university attended by the student populations.
Other university town populations were used on their own, without student populations, but many of the studies that did not use college students could not. Studies of blind dates, cyclists, criminals, pregnant women, sleep deprivation, parents of premature babies, younger children, soccer referees, musicians, and severely disabled people all drew from specialized populations.
More striking than the use of college students, however, was the geographic restriction on the populations used. Out of 60 studies, 51 drew their samples entirely from the U.S., Canada, and Europe. The exceptions were (mostly students) from Japan, Singapore, China, Israel, World Cup countries, St. Kitts, Mexico, historical records from around the world, and an international sample drawn from Amazon’s Mechanical Turk. While not all the samples were “Western” (in the odd, non-directional meaning that word has accrued), only those last four–7% of the total–were distinctly non-WEIRD.
Maybe Evolutionary Psychology has a particular bias in this respect and the evolutionary psychology research that’s published in journals with broader missions fills in the gaps. I don’t know. If someone wants to fund a study for me, I’ll be happy to look into it.
In the meantime, however, I’m still quite comfortable saying that the use of college students and WEIRD study populations is a major problem for evolutionary psychology. I’m even comfortable saying that–to the extent that evolutionary psychology concerns itself with human universals–it’s more of a problem for evolutionary psychology than it is for everyday psychology. Psychology of the standard variety is often quite happy to present its results as belonging to a particular milieu.
Evolutionary psychology is not. This means that evolutionary psychology has additional tests to do to support or falsify its fundamentals. As long as the vast majority of research in evolutionary psychology is done in these groups, evolutionary psychology is failing to put those fundamentals to the test.
And all the cries of “prosecution” in the world won’t change that fact.