Probability can be deceptive. The concept sounds easy to understand and we glibly use the ideas in everyday life but there are many subtleties involved. Given how important probabilistic ideas are in life, some have suggested that it be introduced into school mathematics curricula much earlier than it currently is and discussed in more depth.
One subtlety can be highlighted by this example. Suppose that you read about a new study that finds that by taking some measures, you can reduce the risk of stroke by 50%. Should you adopt those measures?
To make a decision you need to weigh the benefit against the costs, which in this case may be that the measures involve taking medications that have rather serious side effects, or that the regimen involved is arduous, or that it is very expensive. But even taking those into account is not enough to decide because you need to know another key piece of information and that is the absolute risk numbers.
The relative risk is what is usually reported and quoted in the media. It is obtained by comparing the results of the test group with the control group. So if the test group has five positive cases and the control group has ten, then the relative risk reduction is 50%. But this ignores the importance of sample size that determines the absolute risk. If the sample size had been 10,000, the absolute risk would have dropped from 0.1% to 0.05%.
Paula Byrne, one of the authors of a recent study on the benefits of taking statins to lower cholesterol, says that focusing only on relative risk does not give enough information to make an informed decision.
Imagine your chance of dying from a certain condition prematurely is 0.2%, and there’s a drug that reduces your chance of dying to 0.1%. In relative terms (relative risk reduction), your chance of dying has been halved, or reduced by 50%. But in absolute terms (absolute risk reduction), your chance of dying has only gone down by 0.1%.
Although there is a 50% relative risk reduction, is it a meaningful difference? Would it be worthwhile changing to this drug, particularly if there are side-effects associated with it? Absolute risk reduction presents a clearer picture and makes it easier for people to make informed decisions.
In our study, published in Jama Internal Medicine, we found that the absolute risk reduction from taking statins was modest compared with the relative risk reduction. The relative risk reduction for those taking statins compared with those who did not was 9% for deaths, 29% for heart attacks and 14% for strokes. Yet the absolute risk reduction of dying, having a heart attack or stroke was 0.8%, 1.3% and 0.4% respectively.
Absolute risk reduction compared with relative risk reduction
News reports tend to focus on relative risk reduction because the larger numbers are more newsworthy. But Byrne argues, based on the work of her group with statins, that people deserve to be given the fuller picture.
The likelihood of a person developing cardiovascular disease in the next ten years is expressed as a percentage. For example, consider an overweight 65-year-old man who smokes, has high blood pressure and total cholesterol. He may be at high risk of cardiovascular disease, compared with a 45-year-old, non-smoking woman with slightly raised cholesterol and blood pressure and no other risk factors. If a doctor were to assess their risk of dying in the next ten years, the estimated risk for the man might be 38%, for example, whereas the woman’s risk might be only 1.4%.
Now consider the impact of taking statins for both. According to the study, statins would reduce the relative risk of dying by 9%. In absolute terms, the man would reduce his risk from 38% to 34.6%, and the woman from 1.4% to 1.3%.
Patients and their doctors need to consider whether they think these risk reductions are worthwhile in a trade-off between potential benefits and harms, including the inconvenience of taking a daily medicine, possibly for life. This is particularly salient for low-risk people for whom the benefits are marginal. However, people perceive risk differently based on their own experience and preferences, and what might look like a “good deal” to some may be seen as of little value to others.
Extremely health conscious people may feel the need to adopt many measures that they are told will reduce the likelihood of getting some serious illness. But if you are in a low risk category, then the cost-benefit calculation using absolute risk may well yield a different outcome than if you are in a high risk category. In the statins example, when presented with the relative risk results, you might expect both men and women as described to take statins in roughly equal numbers. But if they are told the absolute risk, one would not be surprised if the women did not take them at all, unless there are other reasons to do so. This result also shows how important is the somewhat recent realization that such studies should disaggregate results according to gender and other diversity metrics rather than what had been the standard practice earlier, of using just men to represent the entire population.
If the costs of adopting the measures are low (in the case of the pandemic, such measures might involve getting vaccinated, wearing masks, and practicing physical distancing), then one might be willing to take the measures anyway because there is little lost. The difficult cost-benefit calculation is when the costs are high.
Pierce R. Butler says
It ain’t just risk -- we’ll often see, e.g., businesspeople claiming their market share has risen 50% in a week (from 1% to 1.5%).
Here is a situation that doctors often encounter:
• 1% of women at age forty who participate in routine screening have breast cancer.
• 80% of women with breast cancer will get positive mammographies.
• 9.6% of women without breast cancer will also get positive mammographies.
A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
Let us start with 1000 women:
• 10 of these (1% of 1000) actually have breast cancer.
• 8 of these (80% of 10) will get positive mammographies.
• 990 (1000 – 10) do not have breast cancer.
• 95 of these (9.6% of 990) will get false positive mammographies.
• Thus there are a total of 103 (8 + 95) positive mammographies.
• Only 8 of these actually have cancer – about 7.8%.
Only around 15% of doctors get it right. Most doctors estimate the probability to be between 70% and 80%
Mano Singham says
You analysis is absolutely correct and is the way that these results should be interpreted.
This is why one should never agree to major treatment based on a single positive test. You should, if at all possible, get another one.
Not to mention that in this example, the risks are taken to be absolutely certain. But if 10 in 10000 in the control group get a stroke, then a 95% confidence interval for the risk is from 0.04% to 0.16%, so the reduction in risk is quite uncertain. (Part of the problem is that many people confuse a “statistically significant” reduction in risk with a “meaningfully large” reduction in risk.)
Risk definitely needs to be contextualized. And it’s not just risk of dying. There is also the risk of living years with a chronic condition that may reduce one’s quality of life. But even if we only consider risk of death -- much depends on how long one expects to live in the first place. That is why there is a recommendation to stop certain routine cancer screenings once there is little advantage in longevity if that type of cancer were to be discovered at that point in time. For example for a person with no risk factors beyond age it is not recommended to have a routine colonoscopy after age 75, and for people with such risk factors but no symptoms the limit is 85. (But if longevity continues to rise these recommendations may change.)
It’s lucky that statins are cheap and very well tolerated.
Daily vitamin pills likely have less benefit for most, and no one worries about taking them.
And if you manage not to get hit by a bus, then eventually one of CVD or cancer is very likely to get you in the end. Postponing that a bit, especially for high risk individuals, seems like a very reasonable tradeoff.
“the need to adopt many measures that they are told will reduce the likelihood of getting some serious illness”
There is even more to it. Reducing this likelihood means increasing life expectancy means increasing the likelihood of getting some other serious illnesses …..
This is the main reason the costs of health care rise. The fact that on average smokers die 7 -- 10 years before non-smokers means that the first miss all kinds of serious illnesses, which means that their health care overall costs less. Cf Atkinson and Townsend, Economic Aspects of Reduced Smoking, the Lancet 3 sept. 1977.
PS: I’m a non-smoker.
But even cheaper healthcare is that of a wellderly person (over 80, no chronic illness) who dies in their sleep. To get to this point without medication requires a robust genetic makeup -- basically none of the many risk variants. With preventive medication and supplementation we may be able to get even people with some genetic risk into this group.
file thirteen says
NEWSFLASH -- SCIENTISTS SHOW POSITIVE MAMMOGRAPHY INCREASES CHANCE OF BREAST CANCER BY 680%