Did you fall for this? Science published a paper which claimed that reading literary fiction, you know that stuff that gets taught as highbrow reading material in college literature classes, is objectively better than genre or popular fiction at improving your mind and making you better able to understand other people’s mental states. It was all over the popular news sites.
Language Log shreds the paper wonderfully. It’s a great example of stirring the muck and taking whatever odor wafts out of the mess as a great truth. You might be able to see some obvious flaws just from my brief description: what the heck is “literary fiction”? Isn’t that a contentious division already? How do you recognize it (mostly, I fear, it’s because they’re old books that you’d never pick up to read for pure pleasure)? How did the authors of the study choose it?
As it turns out, the authors hand-picked a few passages from books that they subjectively placed into their categories of literary vs. popular fiction, had subjects read them, gave them a couple of standard tests of theory of mind or empathy, and got a very weak statistical effect. You know, if you shovel garbage at a wall, you can probably find some seemingly non-random distribution of the pattern of banana peels, too.
But it got published in Science, which is dismaying. Unfortunately, here’s where Language Log fails — they infer nefarious commercial intent from it.
The real question here is why Science chose to publish a study with such obvious methodological flaws. And the answer, alas, is that Science is very good at guessing which papers are going to get lots of press; and that, along with concern for their advertising revenues from purveyors of biomedical research equipment and supplies, seems empirically to be the main motivation behind their editorial decisions.
Oh, nonsense. I’m sure that Science is quite careful to keep editorial/review and advertising decisions entirely separate, and if their main concern was peddling expensive biomedical gear, why would they waste space on a simple and flawed paper using cheap psychological techniques? There are trade journals that are much better sources for overpriced gadgetry and reagents.
I’ll also point out that they’ve reversed the situation: Science isn’t good at guessing what papers will appeal to the popular press, the popular press is accustomed to turning to a few journals, like Science and Nature, for finding what the scientific soup d’jour is.
This is not to say that Science or Nature are objective paragons at finding the most important science of the day. To the contrary, both are self-consciously elitist journals, jockeying for position as the premier sources of distilled scientific wisdom. Language Log completely missed the boat: paper decisions at Science are not made to satisfy either the popular press or the scientific supply houses; they are decisions to appeal to the tastes of other scientists, and the financial benefits flow secondarily from that.
There is a Sci Culture, just like there is Pop Culture and High Brow Culture and Redneck Culture. And all of these fragments of a greater whole have their various organs of communication and modes and expectations of behavior, which are all much more complicated than being simply driven by the invisible hand of the market.