One of the benefits of being a modern pundit is that there is no price to be paid whatsoever for being totally wrong. Let me be clear that there is nothing bad about being wrong. After all, when you are predicting any outcome that is not certain, there is always the possibility, however carefully you do your analysis, of the actual outcome being one that is not the most likely, one that lies in the tail of the probability distribution.
What is problematic are those people who are wrong because they willfully ignore the data and science and math and simply go with what ‘feels’ right, which often corresponds to what they would like the outcome to be. They are like the people who want to go out on a boat on a lake in a lightning storm, arguing that the probability of lightning striking any given small area is small, and ignoring the fact that if you are the highest point in a given area, your probability of getting hit rises dramatically.
It seems to me that there are five kinds of pundits.
There are those who are simply innumerate and do not understand probability. They are the people like Joe Scarborough who savaged Nate Silver for saying that there was an 80% probability of president Obama winning, that anyone who claimed anything more than a 50% probability was crazy. Scarborough seems unable to wrap his head around the fact that it is perfectly possible for the margin between two candidates to be quite small (of the order of a couple of percentage points) while the probability of the person ahead turning out to be the eventual winner to be quite large.
The Colbert Report
Get More: Colbert Report Full Episodes,Political Humor & Satire Blog,Video Archive
Nate Silver explains to Stephen Colbert that there is nothing really fancy about what he and the other aggregators do. You take the polls and average them using various criteria for inclusion and weighting. You then take the state poll outcomes and run many, many simulations and see how frequently one candidate emerges the winner to get the probability. That’s pretty much it. It is not magic. Sam Wang even gives out his computer code.
The Colbert Report
Get More: Colbert Report Full Episodes,Political Humor & Satire Blog,Video Archive
Another group of pundits are those who give undue credence to impressionistic views. They get influenced by the size and enthusiasm of the crowds at rallies, the number of lawn signs they see as they drive around, the views of other pundits, and the chatter they encounter around them, usually from their friends, neighbors, co-workers, and acquaintances. If you have lived through enough election as I have, you will realize that these are notoriously unreliable guides. Towards the end of a campaign there are always enthusiastic crowds, falsely giving the impression of a last minute surge in popularity, especially for the candidate who is behind. I have been disappointed enough times in the past to not take these indicators seriously.
Then there are those pundits who think they are smarter than everyone else and base their predictions on indicators and data that they think are significant but which have not really been tested rigorously and whose past success may be due to nothing more than flukes. But as Adam Gopnik points out, their crude models work just often enough to make them think they mean something when they don’t. Karl Rove is an example of this. Once hailed as some kind of political genius because of his seeming ability to predict election outcomes and his glib use of numbers, his star began to fade when he famously derided NPR’s Robert Siegel in the 2006 elections saying that while others may use math, he alone had “the math” that showed the Republicans maintaining control on both houses of Congress. They actually lost both houses by huge margins. In the last election he predicted a Romney win with 285 electoral votes.
Then there are those pundits who don’t really care what the facts are, they know that their role is to espouse a certain point of view in the media. They are essentially actors in improvisational sketches who are given the broad outlines of their role and then they make up stuff as they go along to flesh out their role.
The last group of pundits are the flat-out liars and these people are also in the media or are campaign insiders. They know that what they are saying is not true but they say it out of a desire to boost morale among their supporters and to keep the money flowing in.
Will the results of this election result in less attention being paid to these people in the future? Sadly, no. In every election the media needs to promote the idea of a close, cliff-hanger election and so they will seek out those who will make such a claim and there are enough people who are willing to say anything to get on TV. We already see people like Jonah Goldberg, in his typically incoherent fashion, critiquing what he refers to as the “cult of the numbers” of the statisticians and saying, “But I like to think that people are different, more open to reason, and that the soul — particularly when multiplied into the complexity of a society — is not so easily number-crunched.”
Of course he likes to think that. It is only people who think that who will pay any attention to innumerate people like him. He does not seem to understand that while individuals can be idiosyncratic, their behavior in the aggregate may be predictable. In addition, the poll aggregators are not predicting outcomes based on tea leaves but on surveys of what people say they will actually do. Unless people are systematically lying, there is no reason to think that the predictions are wildly off.
As Jon Stewart points out, there was no shortage of people making absurd and unfounded predictions before the elections. But they will be back again and will not be asked about it, because being a pundit means living in a reckoning-free zone.
(These clips appeared on November 5, 2012. To get suggestions on how to view clips of The Daily Show and The Colbert Report outside the US, please see this earlier post.)
Relative to Nate Silver, he called 50 out of 51 states correctly (on another thread, I said that he called all 51 correctly; he called Florida for Rmoney, although he said his percentage edge was small and that the race there was close. Apparently, Obama carried Florida. As I pointed out, even non-conservatives like the WP’s Dana Milbank bad mouthed Silver prior to the election. None of his critics even came close to that record.
It goes beyond that… What Nate Silver and other poll aggregators do is not even to try to predict people’s aggregate behavior. Rather, they are measuring their aggregate behavior, and using certain methods to improve the accuracy of the measurements. The econometric models could be said to be trying to predict aggregate behavior, but the models based on polls are simply measuring.
It would be like if I reported on the number of words in a Shakespeare sonnet, and you whined that I can’t reduce such beauty to a mere number. Well, no, you can’t, but I can still count how many fucking words there are and accurately report that number.
Poll aggregation doesn’t seek to reduce voters’ reasoning to a mere number*, it simple tries to “reduce” to a number the count of how many will vote for each candidate. And, um, Jonah baby? That’s ALREADY A NUMBER.
* Again, econometric models could be said to do that, and their results ain’t too shabby either, so perhaps Goldberg should not be so resistant to this idea. But nevertheless, he is attacking the wrong target.
I had thought he had it 50.3% probability towards Obama — which in any case, I do not count as “calling it”. With the kind of model Silver uses, a probability in that range is essentially an “I don’t know”, i.e. a no-call.
Dear Dr. Signham,
You spent a few blog posts explaining the “bread and peace” model before the election. Are you going to do a post analyzing its accuracy?
Fair enough. That’s ain’t much of a miss.
It’s Prof. Singham.
What an exquisitely apt metaphor that is. Bravo.
Actually, I never use academic titles unless it is absolutely relevant and that is mainly when I write letters of recommendation. At all other times, I go by just my first name or Mr. Singham.
Yes, I will be writing a brief review of the performance of the fundamentals models shortly.
I must respectfully disagree with Prof. Singham here. It is a matter of respect to refer to someone by his/her earned title. Over at the Dispatches blog I always refer to commentors David Heddle and James Hanley as Prof. Heddle and Prof. Hanley. In case Prof. Singham is unfamiliar with these folks, Prof. Heddle is a professor of physics at Christopher Newport Un. and Prof. Hanley is a professor of political science at Adrian College.
And I in turn must respectfully disagree with slc1. The US convention since the 1920s (per H.L.Mencken’s American Language) has been to use academic titles only in an academic setting: elsewhere it’s Mr/Mrs/Miss/Ms. Even in that setting, communications like letters of appointment are usually addressed to Firstname Lastname, [initials of degree]. This particularly applies to Doctor, except for medical doctors. To indicate professional affiliations use, e.g., “Firstname Lastname, Professor of Creation Science at Siwash University, asserts that … “. Of course this convention is frequently honored in the breach.
Hmph.
I have a couple of degrees but don’t use the initials at work unless it’s relevant. It’s down right pompous depending on circumstance. It’s like not referring to bench scientists as Doctor when they have PhDs.