When Teachers Can’t Compartmentalize »« Polish Court Clears Way for Blasphemy Conviction

Probability and Political Predictions

Watching the right wing rage at Nate Silver is kind of amusing to watch. Andrew Sullivan points out that Silver’s predictions on the results of the election are quite in line with all the other major poll analysts and betting markets, while Jonathan Last tries to explain how probability works:

If Romney wins should that discredit Silver’s models? Only so far as anybody ever used them as oracular constructs instead of analytical tools.

One final word: People seem to think that it would reflect badly on Silver if Romney were to win while Silver’s model shows only a 25 percent chance of victory. But isn’t 25 percent kind of a lot? If I told you there was a 1-in-4 chance of you getting hit by a bus tomorrow, would you think that 25 percent seemed like a big number or a little number? Or, to put it another way, a .250 hitter gets on base once a game, so you’d never look at him in any given at bat and think there was no chance he’d get a hit.

This is something that any decent poker player knows, of course. Take the hand I wrote about the other day from the World Series of Poker, where one player had a pair of kings and the other had an ace and a king. They got all the money in before the flop and at that point the player with the kings was approximately a 70/30 favorite to win the hand. That means three times out of ten, the underdog is going to win. If you bat .300 in baseball, you’re probably in the hall of fame. And in that case, the underdog did win. And even prior to the very last card, he still had about a 15% chance of winning (he could hit any one of seven cards to win — the three remaining aces or the four remaining 4s; each “out” in poker is worth about 2.1% per card to come).

So no, if Romney wins that won’t prove Silver, or any of the other statistical analysts handicapping the election, wrong. But I’d be willing to bet that if you look at his national and state by state predictions, he’ll be pretty close. Why would I be willing to bet that? Because I understand probability.

Comments

  1. Reginald Selkirk says

    ‘Psychic’ fails to beat random probability, jumps immediately to the anecdote

    However, in a result they maybe should have seen coming, neither scored more than one hit in five from the readings.

    Subsequent to the this article’s initial publication Ms Whitton contacted MailOnline to complain that various outlets covering the story had failed to mention that one of her sitters was very surprised to receive an accurate reading during the experiment.
    ‘Why do the skeptics and non-believers omit the very significant details I am trying to prove it seems one sided to me if they are only going to comment on their own disbeliefs without telling the true facts,’ she said in an email.

  2. says

    We really need to find a way to drill it into people’s heads that probability is a real thing.

    One extreme example in my personal life: I was playing the new X-COM and targeted an alien just a couple squares away. The chance of hitting it was 99%, and yet my soldier missed. Sure, it was annoying, but I didn’t start whining that the game lied to me or that it miscalculated the chance. One big factor I kept in mind was that I was going to take a lot of shots over the course of the game, so such unlikely events were bound to happen sooner or later.

  3. Ben P says

    So no, if Romney wins that won’t prove Silver, or any of the other statistical analysts handicapping the election, wrong.

    Here’s the interesting thing. This is based on arguments I’ve had elsewhere.

    The problem isn’t necessarily that republicans misunderstand statistics (although they might), its that a lot of them simply seem to have no idea what Silver is actually saying, but just keep hearing from their own sources that he’s a liberal and thinks Obama will win.

    I was in an argument with some other people on another forum, and after a fairly lengthy argument one of the conservatives retorted with basically “well, you can believe Silver’s prediction that Obama has a 99% chance of winning the election if you want, we’ll see what the truth is on election day!”

    Someone then responds, “what are you even talking about, he’s said that Obama has about 70% chance of winning the election.”

    To which the response was just a “well I knew that, i was just exxagerating.”

  4. eric says

    at that point the player with the kings was approximately a 70/30 favorite to win the hand. That means three times out of ten, the underdog is going to win.

    This is why large tournaments are so brutal. Over the course of several hundred (or thousand) hands, a good player is going to be the 70/30 favorite in many hands. If that good player is getting all their money in on each of those hands, the overall probability is that he or she is going to lose all their money on one of them, and be knocked out.

    Its not P(lose) = 0.3 that kills you. Its P(win every time) = 0.7expN where N is large that kills you.

  5. says

    I wonder if there’s a misunderstanding (inadvertent or deliberate, who knows & who cares?) about how poll aggregate projections, which use probability estimates, differ from binary predictions of electoral outcomes?

    Hard to say Nate Silver’s “wrong” when all he’s saying is ‘Obama has a higher likelihood of winning the election than Romney, but Romney could still win it anyway’.

  6. Michael Heath says

    I get probability. However I struggle with understanding why odds are placed on pre-election polls relative to actual election results. That’s because I see a miss as a failure of methodology if a disproportionate number of polling result misses are outside of the margin of error. I really don’t understand how you can put odds on pre-election survey results beyond your result and margin of error.

    Here’s a non-realistic hypothetical to illustrate. If I have Obama/Romney at 49%/48% with a 3.5% margin of error as people walk into a particular voting precinct and I do this at 50 voting precincts, I get how the actual results would be something at the result +/- the margin of error about 95% of the time (assuming the last value is the pollster’s confidence). If 40 of the 50 polls are outside the margin of error, I’m skeptical of the methodology, even if people lied to pollsters*. I don’t understand where the probability of being right or wrong comes into play at all, if the poll results are outside the margin of error at a frequency greater than one’s confidence level, than I think we’ve got a methodology problem.

    Perhaps the odds are merely expressing the predicted outcomes within the margin of error that would swing the results from the expected result. E.g., my example above allows a Romney win while the result is still within the margin of error for my hypothetical poll. Is this what the odds are measuring?

    *Even if people lie, pollsters should and usually do take responsibility for the integrity of their method. A failure due to people lying merely points to the need for pollsters to become more sophisticated in massaging the data to find people’s actual positions after the poll but prior to publishing the poll, and/or improve their questioning methodology to make it easier to discern people’s actual positions.

    FWIW, I had Obama at 317 prior to the 1st debate. I now have him at 277 where Ohio is his path to victory. Every state where Romney is 2% or less behind in the latest state polls as aggregated at RealClear Politics I give to Romney, I give Romney all the states where he’s ahead.

    2% is my racist factor. That’s the number of non-partisans that pollsters think will vote for Obama but will instead vote for Romney. Keys in Ohio:
    a) Romney’s argument that would have effectively taken down the North American auto industry and
    b) early voting, which will make it far more difficult for Republicans to deny Democrats their right to vote in Democratic-heavy voting precincts as they successfully did in 2004.

  7. says

    Probability is one of my favorite subjects. Just yesterday I was telling my students in class about the Pennsylvania Lottery Rigging. After manipulating the machines the only possible numbers were 4′s and 6′s, and wonderfully enough the number drawn was 666. My favorite part if the story is that the head of the PA Lottery, who was not involved but was trying to quash the rumors, went on TV and said: “Look, since the lottery started every triple digit, 111, 222 etc has come up except 666, so people should not be surprised because” — and then those famous words in probability — “it was due!”

  8. unbound says

    The public at large (and actually most professionals I deal with) don’t understand probability well at all. Probabilities apply to groups (especially large groups) of people and events. Probabilities don’t mean much when you look at a single person or event (one of the biggest problems with “sciences” the rely almost entirely on statistics such as dietary studies).

    One approach I’ve used with some people is to use a deck of cards and pull out just black cards except for one red card. Give each person a card (face down) and ask them who has the red card. No one’s hand should go up (since the probability is 1 in 6 or 1 in 8 or however many people I’m dealing with). And when they do flip the cards, the realization starts to hit that one of them actually did have a red card. Just because there is a low probability doesn’t mean that you aren’t the one.

    My real life tales of low probabilities coming true include my own battle with Conn’s Syndrome (which was statistically likely to be the cause of my hypertension by far less than 1% especially due to the type of tumor they found) and my son getting a brain abscess (quite literally on the magnitude of 1 in a million). Doesn’t mean that the statistics are wrong, just means that you happened to be the one.

  9. jenny6833a says

    Johathon Last: “Or, to put it another way, a .250 hitter gets on base once a game ….”

    Ed Brayton: “This is something that any decent poker player knows, of course.”

    The statement by Jonathon Last is flatly false. And Ed Brayton appears to agree with this Last fellow.

    If Ed truly does agree, then neither of them understands what an average is or how an average is calculated.

    Both of them need to repeat 6th grade.

    Moreover, neither of them understands baseball.

    Good grief!

    Love and lollipops from,

    Jenny
    (who is blown away by such rampant innumeracy, and who will, if necessary, explain in 50 examples and no less than 500 words)

  10. cathyw says

    If you find yourself needing to explain this to the non-mathy: pull out two coins; flip them, explaining, “Two heads is a Romney win.”

    …at least since most people don’t carry tetrahedral dice in their pockets; tabletop RPG players are another group of people you won’t have to explain this concept to.

  11. Michael Heath says

    RandomFactor writes:

    The 99% figure comes out of Princeton.

    Thanks for the link RaondomFactor. My conclusion of Princeton’s prediction: No racism or voter suppression in the U.S., no siree. (I hope I’m wrong.)

  12. dave says

    BenP:

    The problem isn’t necessarily that republicans misunderstand statistics (although they might), its that a lot of them simply seem to have no idea what Silver is actually saying

    Thats not just a Republican problem: I have a staunchly democratic friend who recently stated that according to Nate Silver, Obama was all but a sure thing, up a field goal with 3 minutes to go. To which quite a few jumped on him saying being up a field goal with three minutes to go is pretty damn far from a sure thing.

    I never bothered to check if the “up a field goal . . .” bit was a quote of Silver’s or my friends own interpolation. Given Silver’s penchant for analogizing the election to sports, I assumed that part came from one of his columns.

  13. Michael Heath says

    unbound writes:

    Probabilities apply to groups (especially large groups) of people and events. Probabilities don’t mean much when you look at a single person or event . . .
    [...]
    Doesn’t mean that the statistics are wrong, just means that you happened to be the one.

    I’m the extended family missionary on this point (one side of the family). Of course the counter argument is the dopamine-inducing exclamation a miracle occurred. My success rate probably approaches zero except with the smarter kids whose parents aren’t totally off the deep-end.

  14. dingojack says

    unbound have you tried this one on your students?
    a. Take three cards, two black and one red, show the student that this is so.
    b. Taking note of where the red card is, lay the three cards face down in front of the student.
    c, Ask them to point to the card they think is the red card.
    d. Turn over one of the black cards.
    e. Ask if they want to swap, or stay with the card they think is red.

    Then ask them why they want to stay with their initial guess (by far the most likely outcome).

    f. Turn over the card they picked, most likely black. (Ask the students to note if it’s black or red).

    Rinse & Repeat until some inkling to how to calculate probabilities seeps in.

    :) Dingo

  15. uncephalized says

    @dave:

    I never bothered to check if the “up a field goal . . .” bit was a quote of Silver’s or my friends own interpolation. Given Silver’s penchant for analogizing the election to sports, I assumed that part came from one of his columns.

    Yep, it’s from this one from a few days ago. He was saying that being up a field goal with 3 minutes to go gives any given football team on average a 70% chance of winning the game. He was using it as an analogy to show that Obama was the favorite but not by any means the certain winner.

  16. yoav says

    As anyone who ever read a Diskworld book can tell you, million to one chances crop up nine times out of ten.

  17. says

    I found that nice, simple but illuminating probability question is this: “Suppose China modified its one-child policy so that every family could keep having children until they had a son, then they had to stop having children. What would it do to the distribution of boys and girls?”

  18. dingojack says

    Heddle – indeed, but still people fall for the ‘sucker bet’ (even in the real civilised world*).
    :( Dingo
    —-
    * you know, places that don’t have the death penalty… ;)

  19. Scott Simmons says

    It’s very confusing to most people to deal with probabilities in predictions of singular events. An example that I like to trot out comes from a gag bit that a Dallas sports radio program did a few years ago. One of their hosts was ‘interviewing’ another, who was doing an impersonation of Cowboys’ owner Jerry Jones. They ran down the list of games in the upcoming NFL season, and the interviewer asked ‘Jerry’ who he favored in each game. The response was always the Cowboys. At the end of the list, the interviewer then asked for clarification: “So, you’re saying the Cowboys will go 16-0?” “I didn’t say that,” responds fake Jerry, “you’re putting words in my mouth.” Cue audience laughter.

    Fact is, fake Jerry was actually right. Suppose, for simplicity, that he thinks the ‘boys have a 70% to win the game against any opponent, home or road. Then, if you ask him to pick any one game, he’ll say the favorite to win is Dallas. But if you ask him what he expects their record to be for the season, the answer (based on that same assumption) is 11-5. (Most likely outcome in a sample size of 16.) It sounds inconsistent, but it’s really not …

  20. jamessweet says

    So, there’s this thing that I’ve noticed…

    It seems like a lot of the most credible people who do (ideally) non-partisan fact-related work regarding politics — e.g. fact checkers, poll aggregators, etc. — happen to be liberals themselves. I have my own hypotheses why this might be the case (reality having a well-known liberal bias and all), but whatever the cause, the result is somewhat problematic: It’s very easy for conservatives to read bias when there is none. And I don’t even entirely blame them.

    I found myself in the uncomfortable position yesterday of linking to The Blaze, of all things, to debunk a Facebook meme that was going around. In this particular case, since it was an anti-Romney meme, obviously a conservative website had the most interest in doing the legwork to debunk it. And the information in this particular article in The Blaze was easily verifiable, and it checked out. In short: They were right, regardless of motivations. And they were the best debunking of the meme I could find in a quick google search.

    So I took a big gulp and linked to The Blaze. But it wasn’t easy. It made me feel… dirty.

    I imagine it must be difficult for conservatives, even those rare conservatives who actually care about the truth, when they have to defer to a source that they know damn well is run by a pinko leftie such as myself.

  21. says

    I found that nice, simple but illuminating probability question is this: “Suppose China modified its one-child policy so that every family could keep having children until they had a son, then they had to stop having children. What would it do to the distribution of boys and girls?”

    Assuming people keep having children until a son is born, the gender ratio would be 1:1. Do I win?

  22. says

    Area Man,

    Assuming people keep having children until a son is born, the gender ratio would be 1:1. Do I win?

    Sort of. Your answer is correct, but your assumption is unnecessary.

  23. says

    Sort of. Your answer is correct, but your assumption is unnecessary.

    On further reflection, I see that the assumption is indeed unnecessary. The simplest way to reason it is that every birth has a 50/50 chance of being a boy or girl, period. It doesn’t matter how many children people have.

  24. says

    dave–

    Nate Silver actually did use the analogy of Obama being up by 3 in a football game with only a few minutes to play, but he also explained that, historically, teams in that position win about 79% of the time.

  25. matty1 says

    @10 OK I’ll bite if no one else will, I think I have a passable understanding of probability and averages (or at least the commonly used measures) and I can’t see what is wrong with Ed’s post.

    It could be because I know nothing at all about baseball (or poker) and haven’t the fogiest what a .250 is, indeed I don’t even know why the initial 0 is left off let alone what it is a measure of. Please could you explain, what he got wrong and in particular how a Romney win would discredit the idea the Obama had a 70% probability of winning based on polls?

  26. imback says

    @dingojack #15, in your problem, the probability from the student’s point of view whether he had pointed to a red card depends on whether the student knows that (1) you knew where the red card is and (2) you intended to always turn over a black card.

    If he knows (1) and (2), then he knows that a black card would have been turned over no matter what, so no new information was gained when he saw the black card turned over, and so the odds that his original card is red remains 1/3 just as it was before seeing the black card.

    On the other hand, if the student does not know (1) and (2) and assumes you turned over a card at random and that there was actually a 1/3 chance that you would have turned over a red card, then he did gain information when he saw the black card and not a red card, and so the odds that his original card is red increases to 1/2.

    The game totally depends on what the student knows about your motivation.

  27. says

    imback #30,

    @dingojack #15, in your problem, the probability from the student’s point of view whether he had pointed to a red card depends on whether the student knows that (1) you knew where the red card is and (2) you intended to always turn over a black card.

    No. First of all there is no probability “from the student’s point of view,” there is just a probability (1/3) period.

    The easiest way to see this is to imagine that the same deal is presented on tv to two different viewers. One knows the dealer’s strategy and one doesn’t. The probability has to be the same for both viewers regardless of whether or not they know the dealer’s strategy.

  28. dingojack says

    The extra information from the turned over black card isn’t relevant (it’s part of the ‘suckering’). Because the player knows that one of the cards not picked is black they’ll think they have a 50-50 chance of picking the red card from the two face down cards. Psychologically they will be unwilling to swap to the card they didn’t choose.
    However –
    The card the player chooses has a 1/3 chance of being red and 2/3 chance of being black, both when first chosen and after the one of the black cards is turned over.
    Because the two hidden cards have a probability of 1 that one of them is red, the card the player didn’t choose has a probability of being red equal to (1-(1/3)) = 2/3.
    Always swap, you’ll double your odds.
    Dingo

  29. bfish says

    OK, I’m one of those people who gets easily tripped up with probability.

    I have a question about 538′s “chance of winning” number. I’m not sure this question even makes sense, but here goes:

    Right now 538 says that Obama has an 83.7% chance of winning the election. Is the assertion that if we had accurate enough polling data one could theoretically assign 100% probability to the outcome, but that existing polling data carries enough noise that the best we can do is assign 83.7% probability? Or is the assertion that if we held the election five different times, Obama would be the victor four times and Romney once?

    Does what I’m asking make sense?

  30. laurentweppe says

    Right now 538 says that Obama has an 83.7% chance of winning the election. Is the assertion that if we had accurate enough polling data one could theoretically assign 100% probability to the outcome, but that existing polling data carries enough noise that the best we can do is assign 83.7% probability?

    There’s the “noise”, as you call it
    There’s the possibility that every polling compagny made some big unforseen mistake in their models (say, underestimating the number of latino voters or overestimating the 18-30 years old turnout) which would “skew” every polls in the same direction
    And there’s the possibility of a last minute change of heart from the voters themselves (that’s how Truman won against Dewey).

  31. slc1 says

    Re Laurentweppe @ #34

    Actually, the problem with the polls in 1948 was that Gallup and the others stopped polling more then a month before the election, under the delusion that Truman couldn’t possibly catch up. It should be noted that they didn’t make the same mistake 20 years later when polls in September showed Nixon 15 points ahead and they kept polling until the day before the election, correctly predicting a close one.

  32. slc1 says

    Re matty1 @ #29

    A .250 hitter in American baseball gets a safe hit once every 4 times at bat (.250 equals 25%) Since, on average, such a hitter gets to bat 4 times in a nine inning game (not counting bases on balls or games where he ends up batting more then 4 times), that means that, on average, he hits safely once per game. It should be pointed out that a .250 hitter is considered a weak sister, unless a high percentage of his hits are home runs. Generally, a good hitter bats .300 or better.

  33. imback says

    @heddle #31 and @dingojack #32:

    Here are three possible motivation models for the dealer to follow:

    A. The dealer knows where the red card is, and will always turn over a black card that the student didn’t point to.

    B. The dealer always turns over a card at random from the two the student didn’t point to. Whether the dealer knows where the red card is irrelevant then.

    C. The dealer knows where the red card is, and only turns over a black card and offers a trade if the student pointed to the red card. Otherwise the dealer shows the student he lost by turning over the black card he pointed to. This is the Evil Monty Hall model.

    Under model A, there was no chance the turned over card would be red, so when it turned up black, the student had no new information, and so the odds the student pointed to the red card do not change from 1/3. The student will double his chances by trading.

    Under model B, there was a 1/3 chance the turned over card would be red, so when it turned out black, the student did have new information, and the odds the student pointed to the red card increase to 1/2. There is no loss or gain of chances by trading.

    Under model C, the dealer only turns over a card if the pointed to card is red, so when a black card is turned over, the odds the student pointed to the red card is now 1. He would lose a sure win by trading.

    So when the student sits for the very first deal, if the dealer gives no clue about his motivation, the student can only guess at what model to use. Of course, after a few deals, the dealer’s behavior would give some clues. This logic puzzle works best if it is clear that the dealer is up front with the student about his motivation from the beginning.

    Probabilities are not immutable. They change with different information. For instance, if the dealer knows where the red card is, when the student pointed to a card, the probability from the dealer’s point of view that the pointed to card is red is not 1/3 but either 0 or 1. Another example is when watching poker on TV, a viewer getting to see everyone’s hole cards computes a different set of win probabilities than each of the players do who only know their own hand.

  34. says

    imback,

    If knowing the dealer’s model affects the probability then you need to explain the scenario I described:

    Monty deals the cards and flips over a black. We are both watching. You know his model (that he only flips over a black) and I do not.

    We are each given the option to stay or switch. Are you saying that the probabilities of winning (by switching or not) is different for the two of us playing exactly the same hand?

  35. imback says

    @heddle #39, the answer is yes. For a starker contrast, let’s say the dealer privately told me he was playing under model A, always flipping a black card, and he privately told you he was playing under model C, flipping a black card only if a red card is first chosen. Let’s say we both believe him even though he must have been lying to at least one of us. He deals out the cards, as a team we choose a card, and then he flips a black card (not ours). As a team, we must choose whether to trade (without discussion). I would say trade and you would say keep, given what we think we know.

    It’s like poker on TV, where we see the guy fold who was a lock to win. He could very likely have made the best calculated decision given the information available to him.

  36. dingojack says

    Player behaviour ≠ probability.
    If I play aggressively at poker that doesn’t make me suddenly more likely to draw an inside straight or royal flush, does it?
    Dingo

  37. says

    I see.

    My point is:

    If we replay that scenario a thousand times and we both switch every time, then you, knowing the strategy, will win about 2/3 of the time and I, not knowing the strategy, will win the exact same ~2/3 of the time. The probability is the same (2/3 if we switch) regardless of what we know.

    That is, there is a 2/3 chancing of winning by switching regardless of whether I know the strategy.

    What you are saying is, knowing the strategy you will switch every time while I, not knowing, will switch about half the time (actually less than half for psychological reasons.) So you will win more. True.

    I am talking about a priori probabilities. You are talking about strategies to improve winning in which extra knowledge is not changing the a priori probabilities, but rather using them intelligently.

  38. imback says

    @heddle, well you’re not exactly talking about true prior probabilities in the Bayesian sense, since the probability you’re talking about is determined after learning the motivation of the dealer (to always flip a black card). That key piece of information is notably missing from the student’s point of view in the set up of the problem in #15.

    Again, probabilities are not immutable. Say in Texas hold-em, the flop is a A-K-Q. What are the odds Ed will get an A-high straight in the end? His opponent Fred can calculate the odds given the number of jacks and tens out and that there are 4 of Ed’s cards to learn in the final hand. However, Ed knows he has a Q-J and can recalculate the odds given there are 4 tens out and 2 cards to come. He calculates the odds are worthwhile to go all in. But on the TV broadcast, say they have an X-ray camera that can look through the already prepared deck and show viewers the last two community cards. Alas neither is a ten. The TV viewers know Ed’s a goner. All three parties at the same moment have a different view of the odds of Ed getting a straight. It’s all dependent on the amount of information known.

  39. imback says

    @dingo, you’re missing my point. Of course, betting or bluffing has no effect on what the next cards are. But behavior does matter when the dealer looks at the hidden cards and then reacts to that, and then that information is pertinent to the student’s choice.

Leave a Reply