Journals frequently send out pre-publication copies of what they think are important papers to science journalists under an embargo, where they are free to research the topic and gather material to write articles, but not publish them until the release date that the journal specifies. This enables journalists to write articles that put the research in context and provide alternative and critical views on the research in a timely manner. In the hands of good science journalists, this practice enables the general public to get a reasonable sense of what new research reveals.
But it has also become the practice for press releases to be released about scientific research. These releases are sometimes produced by the universities where the research is done, sometimes by the journals where the research is published, and sometimes by the companies that funded the research and think the results will be good for business. The idea here is to garner publicity for the research, the institution, or journal. There are also other organizations that take these releases and reproduce them verbatim on sites that look like magazines without identifying them as press releases, and the casual reader can be fooled into thinking that they are articles produced by science journalists and thus have some credibility. TV news channels, looking for cheap ways to fill airtime, have also been known to treat as ‘news’ what is in these press releases. While I try to see whether the science articles I read are by science journalists or are press releases. It is not always easy to spot the difference.
This can lead to highly misleading claims. One recent example was the report that said that a recent study had found a sharp drop in obesity rates among US preschoolers.
New federal data published Tuesday show a 43 percent drop in obesity rates among children ages 2 to 5 during the past decade, providing another encouraging sign in the fight against one of the country’s leading public health problems, officials said.
In a statement, first lady Michelle Obama praised the progress in lowering obesity rates among young children and said that participation in her Let’s Move! program was encouraging healthier habits.
While this was welcomed because it reversed a disturbing trend, it was also surprising because the decrease was not accompanied by corresponding drops in other age groups as one might have expected.
It now appears that the conclusion was misleading even though the study was done by the Centers for Disease Control and the results were published in the prestigious Journal of the American Medical Association. So what happened?
It turns out that the group of preschoolers, children ages two to five, included in the study were not a large enough group. In order for these results to be considered accurate, the study should have used a larger sample size. The study included 871 children, ages two to five, in its sample group. Because the obesity rate for this age group is fairly low, using low numbers makes the likelihood of errors from random chance higher. The CDC knew that their sample size was limited, and included that information with their study.
Knowing all of this, the conclusion from the CDC scientists was that there were no significant changes in youth or adults. They even included that the results needed to be interpreted with caution.
So given the highly tentative nature of the authors’ own conclusions, what happened to make this into a big story? The problem was the press release that was released by the CDC itself.
The latest CDC obesity data, published in the February 26 issue of the Journal of the American Medical Association, show a significant decline in obesity among children aged 2 to 5 years. Obesity prevalence for this age group went from nearly 14 percent in 2003-2004 to just over 8 percent in 2011-2012 – a decline of 43 percent.
One can hardly blame the writer of the press release because another part of the problem was that the authors of the paper had themselves said in the abstract of the paper that “There was a significant decrease in obesity among 2- to 5-year-old children (from 13.9% to 8.4%; P = .03)”, which is where the 43% reduction came from.
One had to dig deep into the paper and look at the actual results that are given in Table 6 to get the true picture. There you find that the drop was from 13.9 (10.8 to 17.6 with 95% CI) in 2003-2004 to 8.4 (5.9 to 11.6) in 2011-2012 for a drop of -5.5 (-9.6 to -1.4). When you see how large the uncertainties are in each number, you realize that the drop should be treated with healthy skepticism.
Press releases are often written by people who are generalists and have little detailed knowledge of the science they are writing about. Their job is present the research in an interesting way and this can lead to missteps like this. It does not help when the authors of papers use stronger language than justified in the abstracts and conclusions sections of papers, which are the only sections that many lay people read.