Fla*ed protocol *eakens brain study


There’s an obvious design fla* in this experiment reported in the LA Times. The researchers ran a simple experiment *here the subjects *ere sho*n a series of letters, and they *ere supposed to tap a key *hen they sa* one, but not the another. The subjects *ere classified by their political vie*s, from left-*ing to right-*ing, and a correlation *as discovered: students *ith liberal vie*s had more brain activity and made fe*er mistakes than conservatives. This fits *ell *ith my biases, but I *ouldn’t *ant to s*ear to the trust*orthiness of the *ork. There *as a very poor choice in the design.

The letter that you *ere supposed to tap *as “M”. The one that you *ere supposed to avoid *as an upside do*n “M”, the letter also kno*n as “*”.

Unfortunately, it’s a *ell-kno*n fact that *e left-*ingers have acquired a strong aversion to the letter “*” over the last fe* years. Much as I’d like to claim that *e are s*ifter and *iser in cognitive tests (as *ell as being much more *ell-endo*ed), *e must admit that our fello* moonbats may have made fe*er mistakes *holly because “*”‘s give us the *illies, and *e *on’t use the *icked letter *ithout *arrant. *e t*itch and feel a t*inge *henever *e click the key bet*een “Q” and “E” on our keyboards, and *e actually *ish *e could *ipe out “*”.

Look at the Democratic contenders for President: Hillary Rodham Clinton, Barack Obama, Joe Biden, Chris Dodd and Bill Richardson. Not a “*” among them. (*e kno* you’ll mention Ed*ards, but he’s doomed: he’s got a “*”).

*hy, it *as a *retched, a*ful choice of letter pairs that ske*ed the *ork into *orthlessness.

*ell, if you *ant a heavy*eight revie* of the study, see Cognitive Daily. There are no “*”‘s in Dave or Greta Munger, you can trust them.

Comments

  1. DrFrank says

    Is that the liberal atheist equivalent of saying Jehovah, then?

    Hmm, I guess I should probably get stoned after that.

  2. says

    Imagine my own personal pain. Not only did I get saddled with a pretty goddamned goofy name, but I actually share an initial with that retard.

  3. Kseniya says

    Like *hoa, *hat a *eird *ay to *rite.

    LoL, yes, but thank The Designer for giving us the “Replace All” function, which was adapted from the “Find All” function. Please note that the transitional “Replace Some” feature has not been found in the software record, which clearly disproves Evolution.

    The bias could have been eliminated if they’d used “R” and “Я”, though it’s possible that one of these characters might trigger an aversion response in parents of young children.

  4. AlanWCan says

    Conservatives [made more mistakes] when they saw a W, researchers said.. Now that is insight.

    “Not all conservatives are stupid, but it is true that most stupid people are conservative.”David Hume

  5. cm says

    Not to mention that the W is way over to the left on the keyboard and the M is down toward the right. Couldn’t they have chosen a more balanced layout of selection keys? Did subjects respond with the same hand for both? Did they control for handedness? Etc. Gotta go look up the study.

  6. Barn Owl says

    If the study had found that conservatives have more brain activity than liberals, the “research” would be immediately dismissed or ignored as the total unscientific crap that it is. It’s crap regardless of whose bias it confirms.

    *hat kind of *ankers revie*ed it for Nature Neuroscience, any*ay?

  7. sphex says

    Ummm… I hate to break up the party, but I do this kind of research, and I read the original article, as well as the supplementary info. I was all set to find fault with it, but it was actually well done.

    Commenter cm, let me clarify: The subjects weren’t *pressing* the “M” or the “W”- rather those were the letters that flashed on a screen. For half of the subjects, “M” appeared 80% of the time, and “W” appeared 20% of the time. These subjects had to press a button (probably like a joystick button) whenever they saw an “M”, but NOT press the button to the “W”. For the other half of the subjects, “W” appeared 80% of the time, and “M” appeared 20%… these subjects had to press the button when they saw a “W”.

    The fact that the ‘meaning’ of the letters was counterbalanced across subjects suggests that the ‘design flaw’ that is referenced above did not, in fact, exist.

    The idea is to set up a ‘habitual’ response (in this case: see letter, press button- this is what is required 80% of the time), but then occasionally have a trial in which the knee-jerk response has to be overridden (those 20% of trials). This creates a response conflict that can be seen in the stimulus-locked EEG.

  8. GDwarf says

    Well, in fairness, the scientists never claimed that it showed anything about the personalities of conservatives or liberals, just that it would seem that there’s some connection between political ideology and your brain’s ability to inhibit movement.

  9. Owwmiwwow says

    Wight! Iw’we had it! Enowf is enowf! Now mowe of this uttew dawm nownsense! Hewe! Hawe sowme wubbas, dawm it aww to heww!

    Wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
    wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

    PS: Geowge Walkew Buwsh is nowt a wankew juwst becawse yuw say sow!

  10. sailor says

    Seems clear to me the study showed liberals had more pre-frontal control than conservatives. But maybe conservatives are hard-wired to tap quickly and often? Maybe Craig could use that as a defence?

  11. g says

    John McCain, Fred Thompson, Rudy Giuliani, Mitt Romney. No *’s there either; I guess both sides learned the lesson. (Obviously Ed*ards is doomed and Ne*t Gingrich is going to stay out.)

  12. Dan says

    *o* that’s quite an odd survey you got there, *orks out pretty *ell though

    VVTF is up *ith the people doing the *’s though?

  13. N.Wells says

    The conservatives may have been slowed down by mini-orgasms or the like every time a W appeared.

  14. David Marjanović says

    John McCain, Fred Thompson, Rudy Giuliani, Mitt Romney. No *’s there either; I guess both sides learned the lesson. (Obviously Ed*ards is doomed and Ne*t Gingrich is going to stay out.)

    Let me also note that *illiam Jefferson Clinton al*ays called himself “Bill” on, *ithout exception, every occasion, except *hen it *as brought up during his impeachment proceedings. *e no longer need to *onder *hy. Bubba clearly *as ahead of his time, as so often.

  15. David Marjanović says

    John McCain, Fred Thompson, Rudy Giuliani, Mitt Romney. No *’s there either; I guess both sides learned the lesson. (Obviously Ed*ards is doomed and Ne*t Gingrich is going to stay out.)

    Let me also note that *illiam Jefferson Clinton al*ays called himself “Bill” on, *ithout exception, every occasion, except *hen it *as brought up during his impeachment proceedings. *e no longer need to *onder *hy. Bubba clearly *as ahead of his time, as so often.

  16. says

    You sir have offended me with this Rogue language. What is this Swearing you are offering to your readers – we all know that inserting *’s in the place of letters implies you are covering up dirty language.

    For Shame! I shan’t be reading this blog again!

  17. melior says

    The bias could have been eliminated if they’d used “R” and “Я”…

    Except for the fact that estimates of US dyslexia rates range from 10-20%.

  18. SteveC says

    As one wag on slashdot.org put it:

    From the article: “Conservatives [made more mistakes] when they saw a W, researchers said.”

    Slashdot wag said:
    “Ain’t that the truth.”

  19. MTran says

    Greg: I have no idea what you’re talking about. W is between M and V on my keyboard. ;)

    Hooray, another Dvorak user! I’ve got so much nerve damage I can’t even use qwerty. And my weird keyboard acts as a nice security device when people try to use my computer, which is a nice bonus feature.

  20. cm says

    sphex, what’s your call on n = 43 subjects, r values in the .3 to .4 range, and the metric for political belief a range? Strikes me as little you can conclude from this.

  21. N* m*r*e v*w*ls says

    St*rt s*v*ng m*n*y! St*p b*y*ng v*w*ls! Th*y *nly m*k* *p * fr*ct**n *f th* *lph*b*et *nd *r* h*rdly n*c*ss*ry t* c*mm*n*c*t* *n *ngl*sh.

    *bv***sly, r*m*v*ng th* W’s *s *n *n-*m*r*c*n, *nt*-p*tr**t*c *ct**n. C*t*z*ns b*w*r*! H*m*l*nd S*c*r*ty w*ll b* kn*ck*ng *n y**r d**r s**n.

  22. says

    Hmm, I read this study and had the exact same idea:
    http://rhinocrisy.org/2007/09/liberals-is-smarterness/

    Re: #19, everything they did was correct EXCEPT for their choice of letter. Yes, they control for handedness, yes their statistics are sound. But there IS a difference in how liberals and conservatives treat those letters. If you don’t control for that possibility, you can’t be sure what you’re actually testing for – ability to avoid the “habitual response” reflex, or how much you hate “W”.

  23. Stan says

    What are y’all going on about? *ubya begins with a “*”, not a “w”. *uh. So what are we *emocrats supposed to do? Change the name of the party?

  24. outlier says

    I’ve done this type of research before, too, and I do think they didn’t use _only_ W and M, but other letter combinations. I think this is mentioned in the comments at Cog Daily. The M and W were only used as examples in the discussion.

  25. says

    Hm, this same test is used to diagnose hyperactivty/ADD/MBD/whatever it’s being called these days.

    So now I have to start looking for research on the political preferences of hyperacties. Oh, joy.

  26. Swamp Thong says

    Cute, but they did it both ways (M/W and W/M) ~ so the criticism is not really accurate.

    “Researchers got the same results when they repeated the experiment in reverse, asking another set of participants to tap when a W appeared.”

  27. Swamp Thong says

    Cute, but they did it both ways (M/W and W/M) ~ so the criticism is not really accurate.

    “Researchers got the same results when they repeated the experiment in reverse, asking another set of participants to tap when a W appeared.”

  28. says

    Reversing the letters doesn’t mitigate the issue – and that’s the only control they performed (they chose the letter pair because it was used by a previous study, and it’s the pair that’s optimally confusing, visually). The subject can assign whatever significance they want to the act of hitting or not hitting a ‘W’. Like “I am killing ‘W’ when I hit the button,” or, “I am not voting for ‘W’ when I hit the button.”

  29. cerebrocrat says

    The subject can assign whatever significance they want to the act of hitting or not hitting a ‘W’.

    Which is why claiming that the use of “W” as one of the stimulus letters is a significant flaw in the study is a stretch. The only way “W” as opposed to any other letter might affect this study is via salience, but that would be likely to vary not as a function of political affiliation, but of political awareness/participation. If “W” affected people in a systematic way predictable by political affiliation, then reversing the stim order certainly should control for this. This W business is a silly criticism.

    I have to say, the response to this study throughout the science blogosphere has been very disappointing. I would not expect science bloggers, who should/do know better, to be so quick to attack the study in some cases either without having read it, or without knowing much about the field from which it comes. At the very least, science bloggers should know which criticisms to direct at the study, and which to direct at the coverage of the story in the mainstream press.

  30. says

    WTF? No, you’re dead wrong. First, I went to NN and read the whole study, and the Supplementary Methods. The protocol assigns the subject a task: “Pick W”, or “Don’t pick W”, in either version. In order to achieve this task, the subject does whatever the hell they want in their own head. However, it’s probable, given the political bias already introduced to the process, that liberals will easily assign differential qualities to the letters based on their long experience with the letter “W”, while conservatives are less likely to perceive a strong difference between the two letters. That is, the two groups are armed with a different set of tools with respect to these letters, and are encouraged to USE those tools by the protocol. This is an obvious flaw in the protocol. Whether it explains the results, I don’t know. But it’s a factor that should be removed before assuming the (dubious) conclusion to be true.

  31. cerebrocrat says

    Look, for this “W” objection to work, the following must be true:

    1) In the context of a task for which political identification has been primed, the letter “W” will have a different salience than the letter “M.” (because of the association of “W” with President Bush)
    2) The salience of “W” will affect subjects’ performance on the task
    3) The salience of “W” will affect the performance of liberal and conservative subjects differently

    If you want to object that “W” is a bad stimulus because it has political salience that will affect the results, then it has to work one of two ways: either the effect is based purely on salience, with its value as a rewarded stimulus (CS+) or distractor (CS-) being irrelevant; or the effect is based on an affective association (liberals hate W, conservatives like W), in which case, the role of W as a CS+ or CS- would have to interact with this association. If the former is true, then performance should vary with the strength of political identification, but not with what the identification is. I.E., for really intense liberals or conservatives, W would seem more different from M, and would make the discrimination task easier regardless of which letter was the distractor. If the latter is true, then reversing the sign of each letter, as the experimenters did, is an adequate control, because the affectively-loaded “W” produces the same results within political affiliation groups whether it’s a CS+ or CS-, while performance still differs between groups.

    If there’s some other reason you think the letter “W” should matter, then you’ve got to flesh out the hypothetical basis for it, because “the two groups are armed with a different set of tools with respect to these letters,” is an assumption without empirical support I can see.

  32. says

    So, what about those historically-savvy participants who see one letter signifying George *. Bush, and the other signifying Richard *. Nixon? Do they get extra points or something?

  33. fallacy detector says

    If the study had found that conservatives have more brain activity than liberals, the “research” would be immediately dismissed or ignored as the total unscientific crap that it is.

    So you claim, but such claims are a classic form of fallacy.

  34. says

    Cerebrocrat, it’s not that difficult. Conservatives do not go around making bumper stickers saying “W LIED”, W with a line through it, etc. They don’t give as much of a shit about the letter W. They’re far more likely to refer to Bush in their heads as “President Bush”, not “Dubya”. So, salience differs for liberals and conservatives. That is, W is salient for liberals, but not salient for conservatives. That’s quite consistent with the results; in fact, it explains them perfectly.

  35. cerebrocrat says

    Okay. You’re offering an alternative hypothesis (“salience [of the letter W] differs for liberals and conservatives”) for this study’s results, which is fine. It doesn’t strike me as very likely, but that doesn’t matter; it’s testable. But your alternative hypothesis implies another: that salience affects performance on the task the same, whether W is the rewarded (Go) or distractor (No-Go) stimulus, and this is certainly not likely. Rather, if W has increased salience for liberals, then liberals should show improved performance on the task when W is the No-Go stimulus. Remember that the essence of the task is conditioning subjects to respond to the Go stim with a button press, and because the Go stim is presented at .80 probability, the response tends to generalize to any stimulus. That’s the easy part; inhibiting the generalized response to the No-Go stim is the hard part. If the No-Go stim has greater salience, that salience should assist response inhibiition – better performance. If the Go stim has greater salience, that salience should assist response acquisition (steeper learning curve, shorter response latency, etc). If there’s any effect at all on response inhibition, it would be to make it harder. So, you’d see an effect of stimulus order for liberals, but not for conservatives. These authors did that, and saw no such effect. Your alternative hypothesis would require a mechanism by which salience doesn’t work that way, in contrast to what the existing literature says about the effects of salience on learning. That obligates you to provide an alternative theoretical grounding for your hypothesis, if you want it to be taken seriously.

    As for me, I can’t get past the initial assumption, that the letter W is more salient for liberals than for conservatives. The nickname “Dubya” came to public attention via the Bush campaign itself, which also distributes “W” stickers with no cross-through. Some Bush supporters use “Dubya” as an approving nickname. On the other hand, no fire-breathing liberals I know, including myself, ever call Bush “Dubya.” We call him “Bush.” How representative we are, I don’t know; and my assertion is that you don’t know either, and are forwarding your alternative hypothesis with a confidence it doesn’t deserve without more empirical support.

  36. Anton Mates says

    Cerebrocrat, do be aware that PZ was joking about the “W” objection, as are most of us (with the possible exception of saurabh). Cognitive Daily seems to think the study was quite well done, and I agree.

  37. cerebrocrat says

    Yeah, I think I got stuck on saurabh. I did gather that PZ was kidding. But I ended up here after reading about this on a few other science or politics blogs, and it seemed like there was an awful lot of, “Ah-ha! I have found the critical flaw in this study! Foolish scientists!” kind of commenting going on, including by people who should know better. It annoyed, I complained. And then I perseverated.