In an earlier post yesterday, I mentioned the ‘house effect’ of polls. These are the size of the effects that a given polling outfit produces in favor of one or the other party. They are not necessarily biases in the sense of the polling firm deliberately distorting the results. It is often the result of methodologies that produce different effects such as sampling only likely voters vs registered voters, cell phones vs. landlines, robocalls vs. human calls, weighting by party affiliation, etc. Simon Jackman has an article explaining it in more detail.
These differences can produce systematic shifts in the results by as much as 2% that are not accounted for in the margin of error, which is usually just the random sampling error which can go either way. This raises some issues that poll aggregators need to consider.
Q: Which polls do the aggregators take into account?
A: Some take only the bigger, more reputable polls, while others take them all.
Q: Do the aggregators adjust for the house effects before inserting them into their poll averaging algorithms?
A: Some do and some don’t.
Q: How do the aggregators calculate the size of the house effect?
A: Some do by looking at their track record in past elections, comparing their predictions just before the elections with the final results. Jackman does it by simply averaging all the polls, assume that this average is close to the true mean, and then seeing by how much each poll differs from the mean to calculate the house effect.
These differences in methods leads the aggregators to different results, but since they use pretty much the same large database of polls, the differences are not that much.
Meanwhile, there are the fundamentals models of prediction favored by political scientists who mostly eschew opinion polls and use econometric data in their models. I have written extensively about Douglas Hibbs’s ‘Bread and Peace‘ Model. Based on the latest economic data Hibbs predicts that Barack Obama will get just 46.6% of the popular national two party vote and that the Democrats will win only 183 seats in the House of Representatives, down from the 193 they won in 2010 and the 190 they currently hold.
Hibbs is clearly an outlier but this is what I like about true academics and the aggregators. They will stick to their models until the end. If something goes seriously awry, they will investigate and create a new model based on the new data, but they don’t make ad hoc changes based on today’s newspaper headlines.