Baseball Umpiring And Scientific Peer Review

This article describing a detailed statistical study of umpire reliability in calling balls and strikes is quite interesting. Some of the conclusions have implications by analogy for scientific peer review of grant applications and journal paper submissions:

Contrary to the expectation (or hope) that umpires would be more accurate in important situations, we found that they were, in fact, more likely to make mistakes when the game was on the line. For example, our analyses suggest that umpires were 13 percent more likely to miss an actual strike in the bottom of the ninth inning of a tie game than in the top of the first inning, on the first pitch.

The race of the pitcher, we found, also mattered, but not as much as other factors. Umpires were 10 percent less likely to expand the strike zone for African-American pitchers than for Caucasian pitchers, but race did not seem to influence whether an umpire called a pitch a ball when it was actually a strike.

One of the sources of bias we identified was that umpires tended to favor All-Star pitchers. An umpire was about 16 percent more likely to erroneously call a pitch outside the zone a strike for a five-time All-Star than for a pitcher who had never appeared in an All-Star Game. An umpire was about 9 percent less likely to mistakenly call a real strike a ball for a five-time All-Star. The strike zone did actually seem to get bigger for All-Star pitchers and it tended to shrink for non-All-Stars.


  1. colnago80 says

    Apparently, the study didn’t analyze the effect on balls and strikes calling by hitter. It was ofter intimated that Ted Williams got the benefit of the doubt on close pitches which he took because of his reputation as having a good eye at the plate.

  2. Trebuchet says

    So I’m even less familiar with peer review than with the actual mechanics of umpiring, but tend to understand that both are considered generally Good Things. Is the point of the thread that peer reviewers, like umpires, tend to favor publications by people already widely recognized in the field, even if they are wrong? That’s not surprising, I think. Human nature and all that.

  3. BBBShrewHarpy says

    Totally. I had exactly the same thought on reading this article this morning. Trebuchet, I’ve been on peer review panels when BSDs (of both genders) submitted very poor and carelessly written proposals and the panels essentially rewrote them during the deliberations, second-guessing the bad proposal to figure out what the BSD REALLY wanted to do and say, and somehow making the proposal fundable.

  4. Trebuchet says

    @3: I’m afraid I need a translation of “BSD”. Big Shot Dimwits? Ball Sucking Dorks? Bachelor of Science in Divinity? (Kind of doubt it’s that last one.)

Leave a Reply

Your email address will not be published. Required fields are marked *