There is a strong belief among athletes and sports fans that sometimes athletes enter ‘the zone’, or have a ‘hot streak’ where it seems they can do no wrong or at least perform much better than they usually do and thus have a much greater chance of success at hitting the ball or shooting a basket than at other times. There is a kind of plausibility story built around this idea. When you achieve success, it makes you feel good and confident and that sense of assurance may lead to a greater focus and thus better performance whereas failure may lead to greater nervousness and second-guessing of oneself that could prove harmful in fast-moving actions sports.
When people used to say that certain athletes were on such a streak, I would smugly correct them with science, saying that studies had shown that when analyzed statistically, such streaks were non-existent and were merely illusions, examples of confirmation bias in action. But the victims of my condescension may have the last laugh. Joshua Miller and Adam Sanjuro write about the history of analyses of such streaks and why they may have misled us.
In the landmark 1985 paper “The hot hand in basketball: On the misperception of random sequences,” psychologists Thomas Gilovich, Robert Vallone and Amos Tversky (GVT, for short) found that when studying basketball shooting data, the sequences of makes and misses are indistinguishable from the sequences of heads and tails one would expect to see from flipping a coin repeatedly.
In GVT’s critical test of hot hand shooting conducted on the Cornell University basketball team, they examined whether players shot better when on a streak of hits than when on a streak of misses. In this intuitive test, players’ field goal percentages were not markedly greater after streaks of makes than after streaks of misses.
GVT made the implicit assumption that the pattern they observed from the Cornell shooters is what you would expect to see if each player’s sequence of 100 shot outcomes were determined by coin flips. That is, the percentage of heads should be similar for the flips that follow streaks of heads, and the flips that follow streaks of misses.
The authors then explain why that plausible implicit assumption may be wrong and it is due to what they call the principle of restricted choice.
For example, imagine flipping a coin 100 times and then collecting all the flips in which the preceding three flips are heads. While one would intuitively expect that the percentage of heads on these flips would be 50 percent, instead, it’s less.
Suppose a researcher looks at the data from a sequence of 100 coin flips, collects all the flips for which the previous three flips are heads and inspects one of these flips. To visualize this, imagine the researcher taking these collected flips, putting them in a bucket and choosing one at random. The chance the chosen flip is a heads – equal to the percentage of heads in the bucket – we claim is less than 50 percent.
Since the expected value is less than 50%, when the earlier researchers found close to 50% they were not finding that there was no hot hand but that in fact there was, because the athletes were outperforming expectations. You can read the article for the reasoning behind this key point that underlies their claim. The authors then go on to apply this to the analysis of ‘hot hand’.
With this counterintuitive new finding in mind, let’s now go back to the GVT data. GVT divided shots into those that followed streaks of three (or more) makes, and streaks of three (or more) misses, and compared field goal percentages across these categories. Because of the surprising bias we discovered, their finding of only a negligibly higher field goal percentage for shots following a streak of makes (three percentage points), was, if you do the calculation, actually 11 percentage points higher than one would expect from a coin flip!
An 11 percentage point relative boost in shooting when on a hit-streak is not negligible. In fact, it is roughly equal to the difference in field goal percentage between the average and the very best 3-point shooter in the NBA. Thus, in contrast with what was originally found, GVT’s data reveal a substantial, and statistically significant, hot hand effect.
Thus, surprisingly, these recent discoveries show that the practitioners were actually right all along. It’s OK to believe in the hot hand. While perhaps you shouldn’t get too carried away, you can believe in the magic and mystery of momentum in basketball and life in general, while still maintaining your intellectual respectability.
The GVT paper’s conclusions challenged the conventional wisdom of its time and it took awhile to become generally accepted. As one can imagine, any work such as this that challenges the new conventional wisdom is also going to be challenged, with more data collected and analyses carried out to see if it holds up.