Fact-checking in science


In science, collective judgments by credible experts that have passed through institutional filters are what make up reliable knowledge. These should not be confused with ‘facts’ which is the term given to the things that are directly measured. And yet, the media often conflate the two and in this article in Scientific American, Naomi Oreskes warns that doing so does a disservice when it comes to ‘fact-checking’ politicians concerning climate change warnings.

With the election cycle in full swing, it’s open season for journalists hell-bent on catching candidates out in lies and misrepresentations. In a world that has become relentlessly “truthy,” to borrow Stephen Colbert’s apt neologism, we need journalists, scientists and other experts to stand up for facts and keep the public debate honest. But when it comes to climate change, there is a tricky gray zone between facts and expert judgments.

One such zone has been on display since the release of a 2018 Intergovernmental Panel on Climate Change (IPCC) special report entitled Global Warming of 1.5 °C, whose authors concluded that we had 12 years left (now 11) to achieve radical reductions in greenhouse gas emissions to limit global warming. This alert has been widely cited, and politicians who have invoked it have been repeatedly fact-checked. But some of this checking makes the dialogue feel more like ice hockey—where “checking” is intended to disrupt play and establish dominance—than like an effort to help the public understand a complex but crucial issue.

But the IPCC wasn’t stating a fact in the first place. It was presenting a collective expert judgment—in this case, the consensus of 86 authors and review editors from 39 countries. Given this accounting, there will inevitably be a range of legitimate interpretations, and any translation will necessarily be a simplification subject to differences of individual opinion. With the finding understood in this way, the dynamic of fact-checking is misplaced. It’s as if, after 9/11, the media were fact-checking how politicians characterized the threat to America.

But let’s not fact-check things that aren’t facts. There is a world of interpretation—and therefore a range of justifiable readings—built into any expert judgment. We should discuss that reasonable range and flag claims that are obviously unreasonable. But we should not confuse judgments with facts. Doing so turns what should be a serious discussion into a score-driven hockey brawl.

In my book The Great Paradox of Science, I go even further than Oreskes and look closely at the term fact (and also words such as law, theory, and hypothesis) and how they are used in science. I conclude that, except for those things that we can directly experience via our senses, if we drill down deep enough, the measured ‘facts’ will also turn out to be collective judgments.

Comments

  1. Pierce R. Butler says

    … ‘facts’ which is the term given to the things that are directly measured.

    Wossa diff between “facts” and “data”, then?

  2. Roj Blake says

    Pierce, if you need to ask the question, I doubt you’ll understand the answer. 🙂

    Re the OP, it doesn’t always take a scientist to make sense of the data, although some people do refer to economics as “The Dismal Science”.

    In 2008-9, Australian Economist not Scientist, Ross Garnaut produced a report on climate change, the actions that were needed to mitigate, and how the future may possibly look. This is an extract from his report.

    And here we are.

  3. John Morales says

    But the IPCC wasn’t stating a fact in the first place. It was presenting a collective expert judgment—in this case, the consensus of 86 authors and review editors from 39 countries.

    It’s a fact that the special report reports the consensus of 86 authors and review editors from 39 countries, even if it is not a fact that the consensus opinion corresponds to reality.

Leave a Reply

Your email address will not be published. Required fields are marked *