Skip to main content.

Reporting on Evidence Requires More Than Spin

Reporting on Evidence Requires More Than Spin

Picture of Kate  Benson

Reporting on evidence based medicine is tricky. Although randomized controlled trials are sometimes considered the final word on EBM, the reality is they may not be. A study published in PLoS in December 2012 found that randomized controlled trials are sometimes misrepresented in press releases and subsequent news coverage. 

The Science Media Centre UK recently promoted one of a continuing series of papers based on a large RCT, however the results remain controversial among researchers - as does the reporting.  

Although the SMC UK did offer a number of experts as sources, they apparently chose not to include any outside experts who questioned the results and failed to mention industry ties for one of the sources they listed - as well as the authors. Industry ties, whether they be Big Pharma or disability insurers, are not an automatic conflict of interest, but transparency is crucial as is balance in reporting.

Variables count

Whether the results of a trial can be extrapolated to everyone depends on how the many variables were handled. Very few trials cover all variables. For example if an RTC studies white, middle-aged, males does that mean the results can be extrapolated to women, children or different races? It is far too easy to oversell a treatment, particularly in the headline, when the journalist fails to ask hard questions or use independent sources.

In the case of this trial, how the cohort was defined could presumably impact how results were interpreted and who they applied to.  When the initial study first came out U.S. journalist David Tuller MPH, wrote the standard article for the New York Times. However, after an outside expert unaffiliated with the trial contacted him about some of the issues, Mr. Tuller wrote a second piece about a key problem with the trial conclusions. The Wall Street Journal coverage followed a similar path.

Why was this potentially important? A number of large patient surveys showed that the treatments touted may be harmful for some patients. Although the study researchers insist they are not, the potential failure to include the patients most likely to be harmed in the study means there is not yet an evidence base for the safety of the treatment in such populations. While journalists themselves don’t determine if a treatment is safe, safety concerns should always be taken seriously and once again outside experts with concerns should be included. And as another U.S. researcher said, the benefits were modest at best even in cohort studied in tertiary setting where the patient had to be well enough to repeatedly travel to recieve treatment. This also limits generalization as patients too ill to travel were not studied.

So was it efficacious or effective?

Efficacy is when treatments are compared in the same study. In this case, there were also some questions raised regarding whether appropriate comparisons were made and whether the application of treatments matched what other researchers use and study. Once again, interviewing outside researchers who study the treatment these authors compared theirs to would have clarified issues regarding suitability and efficacy. Whether these treatments are effective in a standard clinical setting is not known.

How were results measured?

Most clinical trials publish their protocols first followed by the trial at its conclusion. This allows journalists who read both to determine whether endpoints were changed and ask why, if they were. As well, not all endpoints are meaningful or may use a different standard than similar trials.

Regarding the original trial the authors wrote a letter to the editor protesting the second article Mr. Tuller wrote. Mr. Tuller's reply to the letter clarified why the trial as it was done did not meet the gold standard of evidence-based medicine. Mr. Tuller could not have done that had he not read the full paper and protocols as well as interviewing appropriate outside sources.

In this case, perhaps even more concerning than the dearth of appropriate coverage was that the SMC UK was involved in the spin.

Spin can come from many different sources and may be best countered by reading both the protocol and the full paper, using independent expert,s including researchers in the field whose research results are different than that of the authors and providing meaningful context, as Mr. Tuller did. 

Image by Jaqian via Flickr


Picture of

PACE was not a randomized controlled trial. It was an open label study. CBT and GET helped no one in the trial and the trial contained no ME patients as Peter White the lead author admitted.

Picture of

Thanks, Ms Benson, for taking the trouble to report on this important issue, and the way that the Science Media Centre UK promoted the findings but chose not to include any outside experts who might challenge the findings.

Picture of

Thank you, Kate Benson, for focusing attention on Fiona Fox's spin machine.

Picture of

Thank you Ms Benson for providing a balanced account concerning the events around the PACE trial. Too many journalists simply regurgitated the press release and missed the concerted PR campaign by a group of psychiatrists to enforce certain treatments on the largely unwilling patient community. Patient complaints of harm (including attempts to forcibly take children into psychiatric care) have been brushed aside for 20 years, as has medical evidence that such approaches harm (Nunez et al 2012 March edition of Clinical Rheumatology).

Leave A Comment


Join us for a conversation on the latest COVID thread with Dr. Céline Gounder, a leading infectious disease expert, epidemiologist, medical analyst and host of the COVID podcast “Epidemic.” We'll discuss the emerging research, clarify what we know and don’t, and help attendees think through where the pandemic takes us from here. Sign-up here!


Follow Us



CHJ Icon