Well Sourced: Skip PubMed at your own health-reporting peril

A new study lands on your desk. It’s about a topic you have never heard of (brussels sprouts to cure brain cancer) by an academic you’ve never heard of (Agnes Mouthwash).

One way to quickly figure out your next steps is to check with PubMed. One reason to do so is that you’re already helping to pay for it through your tax dollars. The other reason to do so is that a simple Google search – or even a Google Scholar search – is not specific enough to provide you with the information you need. Google casts a wider net that includes papers and commentaries that are not included in PubMed. The latter requires a higher level of vetting that, in essence, does some of your work for you.

SOURCE: The U.S. National Library of Medicine’s PubMed

WHAT IT DOES: PubMed is the online search tool for more than 24 million abstracts and citations for health and science-related topics. The site is maintained by the U.S. National Library of Medicine, which calls the database MEDLINE. It’s made available online with the help of the library’s National Center for Biotechnology Information's Entrez system. That’s why you’ll see references to “MEDLINE” and “Entrez” when you’re roaming around PubMed.

You can search by topic, by author name, by date, by words found in the abstract, and a range of other areas. If you search for “Author Last: Mouthwash,” you will find that Agnes Mouthwash has never published a scientific paper. (That’s because she is a made up character by songwriter Jim Copp.) And if you search for “Text Word: Brussels sprout,” you will get a list of 28 articles, none of which talk about brain cancer.

Hospitals and doctors may claim to be experts, but what do their colleagues think about their work? Anyone can speak at a conference. Getting published in a journal that is reviewed by that doctor’s peers is a higher hurdle.

WHAT IT DOES NOT DO: The available abstracts usually don’t mention who funded the research. And when an article is retracted or corrected, it may not be changed in the PubMed database. Also, you will have to take a couple of steps in order to find the full articles. In PubMed, sometimes only a summary of the work is available, but at other times you can find the entire article. You also can use your organization’s library or local library network to find the article. It may be faster to ask the authors themselves or the journals for a copy.

DRAWBACKS: Not all studies are created equal. Be wary of studies based on a few dozen people in one shopping mall in eastern Montana. Always try to find out who funded the study. If a drug or device company helped pay for a study that makes a product look good, it doesn’t mean the study is bad. It should make you push hard for sources outside that company’s sphere of influence for a balanced look at the results. Find out what other types of research the authors have done in the past. Don’t assume the first person listed as a study’s author did most of the work. Often the lead author’s name was used to get the paper published while a less seasoned researcher, whose research methods may not be as tested, is second or third in line.

SUGGESTION: Ask for the resume of the medical director at one of your largest hospitals. If that person lists publications, track them down, and talk to a few of their co-investigators. You might find that the work doesn’t exist or that the medical director was merely a college student making coffee for the other researchers.

Remember, research is often incremental, so be cautious about repeating in your story claims of “breakthroughs” or “cures” without a lot of further research. Check the references cited by the doctor. Contact those researchers to ask them about the work. You might find that there are gaps in the research that weren’t obvious.

EXAMPLE: In 2002, Dr. David Ho made a splash when he announced that he had found a group of proteins that might block the HIV virus. Ho had a considerable amount of credibility in the research world and had been named Time magazine’s “man of the year” in 1996 because of his groundbreaking AIDS research.

But there were also doubts about what he had found. Other researchers pointed to potential problems in his methodology. On deadline just as the news of Ho’s announcement was coming out, Thomas Maugh at the Los Angeles Times made great use of medical research and talked with a range of AIDS experts who questioned some of Ho’s claims.

Then, in January 2004, Ho admitted that his sample had been contaminated. Part of his study was retracted by the journal.

You can find the article – and the retraction – in PubMed.

Photo by LWYang via Flickr.