In his article in Sunday’s New York Times Magazine, Nate Silver makes the claim that better defenses trump better offenses in the Super Bowl. He bases this claim on the fact that the record of the twenty best defenses to play in the Super Bowl is 14-6, while the record of the twenty best offenses is 10-10. (The twenty best are based on an objective analysis of number of points scored and allowed relative to the league average and adjusted for strength of schedule.)

Silver needs to learn how to do contingency table statistics, because Fisher’s exact test tells us that there is a 1/3 chance that this difference in won-loss record doesn’t reflect any intrinsic difference in the likelihood of the best defense beating the best offense (and a 2/3 chance that it does). I know this isn’t science–where a nineteen-fold greater likelihood that a measured difference is real is considered the generally accepted threshold for giving any weight to that difference–but it is only twice as likely that this offense-defense effect is real than that it is just random chance.

That is pretty fucken thin gruel to publish in the NY Times.

slc1 says

It is cliche in sports but it is widely known that defense wins games. In baseball, pitching and defense is 70% of the game. Good pitching stops good hitting every time. In Hockey, especially in the playoffs, by far the most important ingredient is the goalie. In basketball, again especially int he playoffs, defense is the most important ingredient. Most of the time, the team playing the best defense wins. In football, it is an axiom that a good defense will beat a good offense almost every time. Mr. Silver, who has a degree in statistics, is just telling us something we knew already.

lylebot says

“Fisher’s exact test tells us that there is a 1/3 chance that this difference in won-loss record doesn’t reflect any intrinsic difference in the likelihood of the best defense beating the best offense”

Holy jeezus fucke! The p-value is not the probability of the null hypothesis being true! It’s the probability that you’d get the value of the test statistic or greater *if* the null hypothesis actually is true.

MadMax says

Nate is a Bayesian, he doesn’t care about what Ronald Fisher says.

=8)-DX says

You do mean ice hockey right? In which case this seems complete nonsense to me. Yes, a goalie’s skill and decision-making is crucial, and a good goalie can make up for huge holes in defence as well as putting a stop to aggressive early play, but I’ve seen many games (of top world teams) where overall strategy, endurance and cooperation of the players is what decides the result. It’s up to the team to make as many of the offensives of the other side “catchable” for the goalie. In many situations such as power-play, the team’s decisions are crucial. The number of shots-to-goal and accuracy of shots has a huge impact on what the result is at the end.

But then I’m no expert (who is?)

slc1 says

Re #4

Yes, I’m referring to ice hockey. A former radio talk show host in the DC area, who previously broadcast in the Boston area, once said that the only thing that counts in the playoff is the goalie. And he was a self appointed expert in all things related to sports. As a total non-hockey fan, I wouldn’t go quite that far but clearly a hot goalie can make up for a lot of holes, particularly on offense, just as a hot pitcher in baseball can make up for a lot of holes on the offensive side (c.f. Sandy Koufax and Don Drysdale on the 1965 Dodgers, better know as the hitless wonders and Orel Hershiser on the 1988 Dodgers, another team of hitless wonders).

GregB says

These claims that “defense wins X” are silly gibberish. There is no intrinsic difference between an NFL playoff game and a regular season game, save the quality of the teams is higher.

As Bill James noted long ago, if “pitching is 70% of the game” were reasonable, why is it that pitchers (and ptiching) doen’t get 70% of the salary pool? Why don’t scouts speand 3/4 of their time watching pitchers? Why do coaches bother with hitting practice?

This is just an old, hoary comment made by people who haven’t really thought about, or looked at, the details.

slc1 says

Re GregB @ #6

Bill James was a schmuck who didn’t know his fuckken ass from a fuckken hole in the fuckken ground.The reason why pitchers don’t get 70% of the salary pool is because they only pitch every fifth day (at least starting pitchers, of which each team now boasts 5). And what makes Mr.GregB think that scouts don’t spend 70% of their time scouting pitchers. Citation needed. During a game, teams place one of their pitchers behind the screen behind home plate to chart pitches. The LA Dodgers not only chart pitches but have a coach with a radar gun stationed next to the pitch charterer to record pitch velocities.