The difference between state and national poll predictions


One of the puzzles of the current presidential election has been the divergence between the national polls (which indicate a close race with the lead fluctuating between Barack Obama and Mitt Romney) and the predictions of the poll aggregators based on state polling which have had Obama showing a fairly steady lead that would result in around 300 electoral votes.

Republican strategists have taken lately to using this divergence to argue that the poll aggregators, despite their past successes, are either stupid and don’t know what they are doing or dishonest and rigging the results to favor their candidate Obama,

This is implausible for two reasons. One is that these aggregators are putting their reputations and sometimes even their careers on the line and it makes no sense to risk those by giving a prediction that can be easily proven to be absurdly false. The second is that there is no secret magic to what they do. Anyone can do that kind of averaging since they use the same polling data that is publicly available to any of us. The small variations in their methodologies produce only slight variations in outcomes.

If the final results are completely off, this would not mean that the aggregators are wrong in their methods but that for some reason the entire polling methodology has gone completely off the rails for this election for weird reasons such as that people are suddenly lying more to the pollsters.

But this still leave open the question of why the national and state based results are not converging, at least not yet. Sam Wang, one of the poll aggregators, tries to explain what might be the reason here and here. In short:

In terms of predicting both state-by-state and overall electoral outcomes, state polls do extremely well. In 2008, I correctly identified the leader 49 out of 51 races. I called two races (Indiana and Missouri) tossups, and those races had margins within 1%. In addition, the 2004 EV median precisely matched the final outcome. In other words, state polls get it 98-100% correct. Answer: pretty darned good.

But if state polls use the same methods, why would they do better than national polls? Well, state polls have three advantages. (1) Most state races, even in swing states, are decided by margins of 2% or greater. So an error that makes a big difference in national polls doesn’t matter nearly as much for state polls. (2) State polls target more homogenous populations, which poses fewer technical problems to the pollster. For this reason, the systematic error might be smaller. (3) In critical swing states they are done more frequently. This focuses the data where information is most needed.

Sounds plausible to me.

Comments

  1. jesse says

    I don’t see why this is such a mystery.

    States will poll differently because no state is perfectly representative of the national polling environment. If I poll Idaho I will bot get much Democratic support — even if I polled only Democrats. If I poll Massachusetts I get a similar result in the opposite direction.

    Given the way our political system is set up (with the Electoral College) then surprise surprise, Obama is likely to win. Not a lock, but pretty likely. None of this is particularly advanced mathematics.

    To put it another way: With a minimum of about 66+ million votes, less than a majority of registered voters recorded (137 million) for the 2010 cycle, I could win the presidency (we are assuming everyone votes, here). That map would look weird, tho. More to the point, I came up with that number by just ordering the states by the number of electoral votes and assuming the candidate takes 50% +1. A candidate that did that would carry 42 states (a lot, to be sure) but they would mostly be states where hardly anyone lives, like Alaska.

    If polls showed the winning candidate behind in the national numbers but ahead in those states, then s/he wins. End of story.

    Losing the popular vote and carrying the electoral college has happened numerous times, as had winning popular votes by small margins but winning the electoral college in a landslide, as Obama did in 2008; the popular vote margin really wasn’t all that gigantic.

  2. Loqi says

    One is that these aggregators are putting their reputations and sometimes even their careers on the line and it make no sense to risk those by giving a prediction that can be easily proven to be absurdly false.

    It’s unfair to criticize Republican party officials for not making this connection. After all, they make absurdly false and easily disproven predictions and claims all the time, and their reputation has taken only a minor hit. In Right Wing Land, the correlation between actions and consequences is very weak.

Leave a Reply

Your email address will not be published. Required fields are marked *