Question the Findings: Robert Davis' five ways to query new studies
Robert Davis, Ph.D., M.P.H. is on a mission: Reporters need to put on their skeptics' hats when they report on the latest and greatest in medical research.
The author of The Healthy Skeptic: Cutting Through the Hype About Your Health, Davis is a professor at Emory University's Rollins School of Public Health and the president of the health video company Everwell. He also has experience as a journalist on deadline as a television producer at PBS and CNN, an editor at WebMD and columnist at The Wall Street Journal.
"It's not to say that the tempting stories about miraculous technologies and saving lives are bad and not helping people," Davis said, "but we need to be skeptical about it."
Robert Davis explains why it is so important to scrutinize press releases:
For the reporter on deadline, he offers five ways to scrutinize new studies and the press releases that hype them.
1. Research: "Clearly not all research is created equal, but often when we read about health, research is shown this way," said Davis. Reporters need to pay close attention to different types of studies and how relevant they actually are to consumers. It is important to understand and explain the relevance and credibility of different research strategies, he said. Davis' basic hierarchy runs from test-tube research as the least credible to randomized clinical trials as the most.
2. Numbers: Put numbers in perspective by using absolute numbers. He compares numbers in studies to sales in department stores; even if you know an item is 50% off, you need to know the actual price. Numbers and statistics about prevalence should be questioned. Where did the information come from?
3. Conflicts and Biases: "There is a tendency to put doctors and scientists on a pedestal," Davis said. While there is not deception and fraud in most research, reporters need to be aware of who is funding research and what researchers stakes are in the results of their own studies. Simple web searches and scrutiny of institutions' website can turn up a lot of information about funders, doctors' investments and potential conflicts of interest.
4. Costs, Risks, Unknowns: Press releases often tell you more about the benefits of new drugs without telling as much about drawbacks and side effects. "The truth is there are very few penicillins," he said. With new devices, it is important to pay attention to more than just exciting technology. Ask about costs, effectiveness compared to older devices, and whether or not it is appropriate for everyone. Do the benefits outweigh the downsides? There are, for example, potential harms in mammography and it is important for reporters to make this clear.
5. The Big Picture: Research is not discrete. Findings are part of a body of work that reporters should understand in order to put individual studies in context.