How journalists are searching for certainty during uncertain times

One of the most common questions during this pandemic has been: “When will it be over?” 

Second, lots of people have been asking: “How bad will it get?”

Reporters worldwide have been taking crash courses in statistics to help themselves and their audiences approach some sense of certainty in these uncertain times. And part of that search for certainty has been explanations of statistical uncertainty. 

If I had to grade journalists on this point during the crisis — not armchair critics and frothing pundits mind you, but journalists — I would give them a B+ on writing about uncertainty in epidemiology. Previously, I would say journalists generally averaged around C. I know I certainly did when reporting was my full-time job. But now that the numbers really matter, I have seen so many excellent examples of reporters trying to develop a comfort level with uncertainty for their audiences during these uncertain times.

From the outset, various statisticians have tried to help people see the way in which the pandemic might develop. Those forecasts, of course, all depend on the types of things that people do that may affect the spread and impact of the virus. Some of the earliest projections showed a massive percentage of the global population becoming infected and as many as 2.2 million people dying in the U.S. alone, not to mention the rest of the world. Those very scary projections had an impact. People — especially in places where there were deaths from the virus — started to stay at home more, work from home, and move around less, putting them in contact with fewer people and thereby lessening opportunities for the transmission of the disease. The waves of projections since those early forecasts have tended to present lower numbers of people who would become infected and die from the disease. 

The cases and deaths are still staggering. The coronavirus — which did not even exist as far as the world was concerned just a few months ago — now has killed more than 160,000 people worldwide, including more than 40,000 in the U.S. That means the virus has claimed more lives in a typical year than much more widely known viruses, including the viruses that cause Dengue fever, which by way of comparison causes between 17,000 and 50,000 deaths in a given year. It has killed more people than measles, more people than typhoid, and more people than all viral strains of hepatitis.

And here is where some reporters have done a nice job. I’ll give you three great examples.

Going wonky. Normally, I recommend staying away from overly technical language when trying to explain scientific concepts to your audience. But when trends are changing minute by minute and data science itself is wired directly into how we are responding to the pandemic, it is important for some of these concepts to be described in the same terms that scientists use. 

At the Sacramento Bee, Michael McGough and Rosalio Ahumada cited a chart from the University of Washington’s Institute for Health Metrics and Evaluation (where I work) that showed the trend in California for that period. In describing the chart, they cited the term statisticians would use to explain the upper and lower range for an an estimate. 

A shaded region representing a “95 percent uncertainty interval” shows that on the worst end of that interval, the state could instead peak at more than 350 deaths per day by early next week. The model, which assumes social distancing mandates remain in place through the end of May, now projects 1,783 deaths statewide, with the curve flattening significantly after early May. Six days ago, the same model predicted more than 5,000 deaths across California.

By using terms that may be unfamiliar to people, and then explaining those terms, reporters are starting to build up the lexicon in the public consciousness. This wouldn’t work during normal times, but these are anything but normal times. 

Showing the range. At the Associated Press, Seth Borenstein and Carla Johnson took what the Sacramento Bee did a step further. They actually explained the full range of estimates that are being produced at any given time. And they also walked people through the real-time complications of modeling a pandemic as it is happening. 

Their latest projection shows that anywhere from 49,431 to 136,401 Americans will die in the first wave, which will last into the summer. That’s a huge range of 87,000. But only a few days earlier the same team had a range of nearly 138,000, with 177,866 as the top number of deaths. Officials credit social distancing. The latest calculations are based on better data on how the virus acts, more information on how people act and more cities as examples. For example, new data from Italy and Spain suggest social distancing is working even better than expected to stop the spread of the virus.

Note what is happening here. People Italy and Spain changed their behavior and that had a dramatic impact on disease transmission. That, in turn, has in impact on the next wave of projections. It also changes those uncertainty bounds around the estimates — the range of 87,000 to 138,000 projected American deaths that they write about. Don’t be afraid to explain to your audience that the point of the estimate, the number that often appears in the headlines, is just one number in a wide range of estimates.  

Describing alternative scenarios. Remember that 2.2 million U.S. deaths estimate that I mentioned earlier? When you go back to the source material, a study published by researchers at Imperial College London, you find something quite surprising. That prediction of 2.2 million deaths in the United States was presented without any uncertainty bounds:

In total, in an unmitigated epidemic, we would predict approximately 510,000 deaths in GB and 2.2 million in the US, not accounting for the potential negative effects of health systems being overwhelmed on mortality.

Nowhere in this 20-page paper did the researchers present a range around that frightening number. To put it in perspective, there are about 2.9 million deaths in the United States every year. For one virus to suddenly kill nearly as many people as all other causes combined was an incredibly bold statement and one that should have come with a range of possibilities around it. 

Journalists, thankfully, stepped in to help provide the necessary context for their audience.  The Washington Post’s William Booth dug into the paper and tried to explain that the 2.2 million death number was a worst-case scenario. The “what if we did nothing” number.

If Britain and the United States pursued more-ambitious measures to mitigate the spread of the coronavirus, to slow but not necessarily stop the epidemic over the coming few months, they could reduce mortality by half, to 260,000 people in the United Kingdom and 1.1 million in the United States.

Note that this U.S. figure is still orders of magnitude greater than what IHME and other research institutes have forecast. But this was early in the pandemic. Social distancing started happening even before strict measures were put in place, and the number of deaths, as a result, has been lower than previously expected.

This is why the kind of reporting done by Booth and the other journalists is so important. Forecasts are what researchers are seeing right now. But as data come in, those forecasts must change. Put another way, we can change the future. If people didn’t stay at home and instead continued to gather and increase transmission of the virus, those death counts would have been higher. Instead, they did stay home and are staying home. Death counts are lower. 

By explaining that forecasts offer a range of possibilities, you help people see the range of possible routes a disease might take. In doing so, you are describing the uncertainty around the estimates, but you also are providing a higher level of certainty about the future than simply allowing people’s imaginations to run wild.