This article describing a detailed statistical study of umpire reliability in calling balls and strikes is quite interesting. Some of the conclusions have implications by analogy for scientific peer review of grant applications and journal paper submissions:
Contrary to the expectation (or hope) that umpires would be more accurate in important situations, we found that they were, in fact, more likely to make mistakes when the game was on the line. For example, our analyses suggest that umpires were 13 percent more likely to miss an actual strike in the bottom of the ninth inning of a tie game than in the top of the first inning, on the first pitch.
The race of the pitcher, we found, also mattered, but not as much as other factors. Umpires were 10 percent less likely to expand the strike zone for African-American pitchers than for Caucasian pitchers, but race did not seem to influence whether an umpire called a pitch a ball when it was actually a strike.
One of the sources of bias we identified was that umpires tended to favor All-Star pitchers. An umpire was about 16 percent more likely to erroneously call a pitch outside the zone a strike for a five-time All-Star than for a pitcher who had never appeared in an All-Star Game. An umpire was about 9 percent less likely to mistakenly call a real strike a ball for a five-time All-Star. The strike zone did actually seem to get bigger for All-Star pitchers and it tended to shrink for non-All-Stars.