The Power of Small Data: Illuminate data absolutely and relatively, too
Reporters flocked last week to jungle gyms and bouncy houses around the country to interview parents who were brave – or foolish – enough to allow their children to play.
A new report from the U.S. Centers for Disease Control and Prevention indicated that playground injuries were on the rise. The report had plenty of caveats, which were mostly ignored in the reporting. But the most consistent flaw in the coverage was rightly called out by CHJ editor Ryan White in a post that said:
The playground brain injury rate rose from 0.03 percent of kids to 0.05 percent of kids over the 13-year span. In terms of relative risk, that’s a 67 percent increase! But the change in absolute risk is negligible: The injury rate went from about 1 in 3,333 kids affected to 1 in 2,000 kids.
White makes an excellent point about the difference between absolute and relative terms. It’s a reminder for reporters doing data stories to think about how they are conveying the messages that come out of their data. Sometimes a big percentage increase is an exciting story that you feel like your audience must hear, but it also might be a lot of smoke obscuring a rather flimsy story.
White pointed readers to H. Gilbert Welch, a professor of medicine at the Dartmouth Institute for Health Policy and Clinical Practice, for a primer on relative versus absolute risks. Welch provides another highly instructive example. A study in 2013 found that women who suffered from migraines were 40 percent more likely to develop multiple sclerosis. But wait, Welch cautions. Before you start to fret that your migraines mean that you are going to get MS, consider how rare MS is in the first place:
In fact, for women with migraines, the chance of developing multiple sclerosis over 15 years was considerably less than 1 in 100 — only 0.47 percent. To be sure, that is about 40% higher than the analogous risk for women without migraines — 0.32 percent — but it’s a lot less scary. More importantly, it’s a much more complete piece of information.
The most important thing for your audience is to do what White and Welch did. The first step is to write out your findings both ways. What are the percentage terms or relative terms that you would use to describe your findings, and what are the absolute terms or, as Welch puts it, “real numbers.” If you still think you have a story, show your audience both sets of numbers.
Chris Serres at the Minneapolis Star Tribune put together a great story about trends in caregiver staffing as more and more people hit older ages or suffer from debilitating injuries and illnesses. The piece, “Shortage of caregivers hits home as families scramble to find help,” showed how better employment prospects for low-wage workers was starting to drive people out of caregiver positions. Serres wrote:
As hiring accelerates in a tightening job market, thousands of openings for $10-an-hour caregiving jobs are going unfilled. The vacancy rate for personal care aides in rural Minnesota recently hit 14 percent — highest in at least 15 years, according to state workforce data.
If all you knew was that there was a 14 percent gap in the caregiver workforce, you might not be able to answer the question I posed in my last post: “Is that bad?” Serres starts to answer that question with the last sentence above, noting that the vacancy rate had reached its highest level in recent history. But he also provides some much-needed absolute numbers in an easy to read chart embedded in the story. The chart shows that in 2010, there were just 32 vacancies for caregiver jobs. In 2015, there were 3,285. And if you want to see the data for yourself, the Star Tribune provides a link to download it.
When you see such numbers, you have the adequate context for the rest of the story. It’s another reminder that when you have interesting trends that you want to show in relative terms, it’s always a service to your audience to provide the real numbers as well.
[Photo by Bridget Coila via Flickr.]
—
Related posts
The Power of Small Data: Why you probably don’t need ‘big data’ for your stories
The Power of Small Data: Lessons learned from a number-crunching career
The Power of Small Data: When Stymied, Make Your Own Database
The Power Of Small Data: Doubt Those Who Say They Have No Numbers
The Power of Small Data: Brace yourself for the data-doubters
The Power of Small Data: Ask Questions That Demand Answers
The Power Of Small Data: Defend Against Data-Doubting Critics By Exploring All Angles
The Power of Small Data: Weigh the evidence before you report it
The Power of Small Data: How Rwanda tried to save lives with better math