Complete Health Reporting: Keep NNT on the Tip of Your Tongue

This is part of my ongoing Complete Health Reporting series explaining the 10 essential elements to writing a complete story about a treatment, drug or device.

The most exciting exchange erupting on Twitter last week wasn't about Kony 2012. Or Nandito Ako. It was about NNT.

That's the number needed to treat. It is one of the best ways for you to follow this key rule for health writing: Always quantify the purported benefits of a new treatment in terms readers can understand.

The NNT discussion started with Scott Hensley at NPR's Shots saying he was intrigued by a new study showing the possible benefits of using the hallucinogenic drug LSD to treat alcoholism. He wrote: "Looking over the LSD for alcoholism meta-analysis. NNT for benefit: 6. Let me repeat: 6!"

That means that 1 in 6 people benefited from the treatments. That's remarkable because there are many drugs on the market and treatment protocols in use with much higher ratios.

Then Joe Rojas-Burke at The Oregonian wrote back "That's not so unusual. For naltrexone NNT=5, acamprosate NNT=7, in terms of preventing relapse." And he linked to an article in American Family Physician to prove his point.

This is good context and would be important for a reporter to note in a story about the study.

Hensley responded: "Good points. But 1 dose of LSD and NNT = 6 gets my attention. Lotsa caveats, of course. But I didn't expect that."

Hensley and Rojas-Burke are still in the minority. Many journalists, even in today's data-driven world, are math-averse. (The math department at the University of Montana – where I was fortunate enough to go to journalism school – has for a long time offered a class called "Numbers as News," open only to journalism majors.) This numerophobia is reinforced by well-meaning journalism tips that tell us to avoid cramming too many numbers into a story. One way to avoid that is by avoiding numbers altogether.

A journalist may write that heart attacks decreased because of a therapy or that a new drug showed a big improvement over rival drugs. What do those terms actually mean? How was the decrease in risk or increase in benefit measured? How much of an improvement over the old drug was found in the new drug? Remember that statistical significance may not equal clinical significance. What sort of difference did the treatment make in people's lives?

While Hensley and Rojas-Burke were tweeting about the LSD study's NNT, most writers provided very little backup for statements about the drug's benefits. ABC News wrote "Participants from three studies reported completely abstaining from alcohol, and this effect lasted between one and three months." That might lead readers to assume that all 536 study participants abstained from alcohol. Not true.

CBS News did what most reporters do with study stories. It quantified the benefits using percentages: "Researchers found nearly 60 percent of patients showed a clear improvement compared with 38 percent of the control group, and were less likely to relapse on alcohol." This leaves readers to wonder whether 60 percent means 6 out of 10, 60 out of 100, or 600 out of 1,000.

The actual numbers can make a huge difference in both the importance of the study, as Hensley noted, and in the way readers understand the study.

Kristina Fiore at MedPage Today was one of the few to give readers the crucial number needed to treat with her piece.

"They found that the number needed to treat to achieve any improvement in drinking was 6, to maintain abstinence was 7."

As you will see in Fiore's piece, she also gave readers a sense of the strength of the evidence by providing numbers for p values and confidence intervals. We'll revisit how you can judge the quality of the evidence in a future post.

Obviously the NNT is just one way to write about a study's benefits. Do you have a favorite way you like to use? Drop it in a comment below. You also can send it to askantidote@gmail.com or @wheisel on Twitter.

Next: Why does it hurt so much to write about potential harms in a story?

Read more of the Complete Health Reporting series.