In his article in Sunday’s New York Times Magazine, Nate Silver makes the claim that better defenses trump better offenses in the Super Bowl. He bases this claim on the fact that the record of the twenty best defenses to play in the Super Bowl is 14-6, while the record of the twenty best offenses is 10-10. (The twenty best are based on an objective analysis of number of points scored and allowed relative to the league average and adjusted for strength of schedule.)
Silver needs to learn how to do contingency table statistics, because Fisher’s exact test tells us that there is a 1/3 chance that this difference in won-loss record doesn’t reflect any intrinsic difference in the likelihood of the best defense beating the best offense (and a 2/3 chance that it does). I know this isn’t science–where a nineteen-fold greater likelihood that a measured difference is real is considered the generally accepted threshold for giving any weight to that difference–but it is only twice as likely that this offense-defense effect is real than that it is just random chance.
That is pretty fucken thin gruel to publish in the NY Times.