Q&A with Keith Hoffman: Crunching Drug Data for Reporters on Deadline
The company AdverseEvents scares some in the pharmaceutical industry. Luke Timmerman at Xconomy interviewed the company's CEO Brian Overstreet and wrote recently:
When I told a Bay Area venture capitalist about Overstreet's startup concept last week, it triggered a gut reaction. "I'd like to put that one out of business!" the VC said (I think half-jokingly. I think).
Timmerman described the FDA's Medwatch system in a way that perfectly painted the problem faced not only by the agency, but also by health reporters:
The current system - in which doctors voluntarily fax or e-mail forms about bad reactions they suspect are drug-related - has numerous well-documented flaws. Only about 500,000 reports are sent to the FDA each year, about one-tenth of the estimated number of actual bad reactions. And once reports get entered, they are littered with misspellings, misclassifications, incomplete entries, and incompatible file formats that make it extremely difficult to search the database. Those barriers have made it tough for the FDA, or anyone else, to spot the early warning signs of a dangerous drug until millions of people have been exposed, creating front-page scandals and highly litigious cases like the ones with the pain reliever rofecoxib (Vioxx) and the diabetes drug rosiglitazone (Avandia). If AdverseEvents.com can iron out the problems in the Medwatch database, and become an online repository for simpler side effect reporting, it's conceivable this little startup could someday become an essential online resource for tracking side effects of $1 trillion worth of prescription meds sold around the world. It's the kind of data source that could make or break multi-billion dollar products, and in some cases, entire companies.
I talked with AdverseEvents' Vice President for Scientific Affairs Keith Hoffman about how reporters can make use of the company's work. The first part of our interview ran on Monday. The second part is below. Our conversation has been edited for length and clarity.
Q: What should health reporters who are working with the FDA's adverse event data watch out for and are there any common mistakes or misinterpretations that you have seen in the way the media reports adverse events?
A: The only way for health reporters to gain access to the FDA's data now is to download the millions of records as we did originally or to file a FOIA request. Neither of these are timely or efficient. And in either case, they will still have to combat the issue of the data they receive being so disaggregated. We hope that with increasing awareness of what we're doing at AdverseEvents, health reporters will come to our site or get in contact with us to get the information they need. While we are a for-profit business, we view what we're doing as a significant public health benefit and we want to make sure that complete and accurate information is getting out to the public about drugs and their side effects.
Q: Does this mean that you would provide reporters with some of your data for free or that you just hope they use your website as a search engine to lookup specific drugs? It would be great for reporters to be able to take a big table of drugs and events and do some comparisons of types of events, etc. I'm not sure if there is any geographical coding, but that would help reporters doing local stories.
We currently make most of our data available for free to all users on the website, but we reserve our complex searching and analytic tools for subscription clients. So, we strongly urge reporters to call or email us with specific requests so that we can use those tools to ensure they are getting the most complete and accurate information they need for a story. For example, The Wall Street Journal published an article about the drug Doryx. They contacted us to find out about potential adverse events from pill splitting. While that isn't a stand-alone adverse event type within our database that can be searched, our analysts were able to use our premium analytic tools to locate a number of potential adverse events that matched the Journal's criteria. They used our data in the article.
Q: Explain to me what it means on your site when it says "primary suspect" versus "total patient reports," and which should reporters pay attention to if they are looking for information about a particular drug?
A: When a physician or manufacturer files an adverse event report with the FDA, they will label a drug as the primary suspect if they believe that that drug is the one specifically causing the serious side effect. Because many people take multiple drugs, it is important to be able to distinguish between the primary suspect cases and the total cases. That's why we break those two things out throughout the AdverseEvents site. We advise everyone using the site to focus on the primary suspect cases.
Q: I know that your Rx Filter system is proprietary, but tell me a bit more about what it does and give me a few examples of how it fixes problems with the FDA data, standardizes the data and to makes the information searchable.
A: RxFilter is the 17-step data refinement process that we had to develop to standardize and normalize the data from the FDA. It allows us to convert their messy data into a user-friendly, fully searchable, database of over 4,000 approved medications. Among the things we do in those 17 steps is auto-correct spelling and keystroke mistakes and correct for misclassifications of side effect types or conditions. I can't get into all the details since it is the "secret sauce," but I can tell you it took about 18 months to develop and perfect.
Q: You showed recently that two epilepsy drugs may be more harmful to a fetus during pregnancy than the FDA has previously reported. Have you heard from the FDA about whether it intends to change the ratings for those drugs?
A: After we released that report, a spokesperson for the FDA released a statement to a reporter that said, in part:
"I cannot comment directly on their findings for those specific drugs. The FDA continues to assess the safety of marketed drug products throughout a product's life cycle and continuously updates drug labeling with new information so that patients and health care professionals can have the best information possible to make treatment decisions."
We certainly don't expect the FDA to take immediate action, but we would hope that this alert – and the ones we have coming in the future – will trigger some measure of review. We think our analysis on the epilepsy drugs is something they should be examining, right now.
Q: Because you are cleaning the data and re-coding it to correct for errors in the FDA database, you're going to end up with more adverse events for some drugs and fewer for others. You probably are finding more because you are collecting adverse events for drugs that were previously under different names. What has been the reaction from drug companies about the fact that their products, in some cases, now appear to be more dangerous than they were a few months ago?
A: We don't think that the issue is whether their drugs appear more or less dangerous than they did a few months ago. The big issue is that, for the first time ever, there are now readily available data about drug side effects. Before we launched, accurately gauging post-marketing side effects and their incidence and outcome rates was extremely difficult.
On a comparison basis, there are going to be cases where some drugs are clear winners and some are clear losers. Knee-jerk reaction from drug companies may indeed be negative. But, we also understand that the people at drug companies are working hard to better people's lives. We think that they can use our data to better inform their own clinical development and business strategy decisions. Again, this is an entirely new type of data so it will take some time for pharma to fully understand and embrace it. But we think that they should, and will, do exactly that.
Q: It's great that so much information on your site is free, but it looks like your business model is really aimed at getting companies to buy your proprietary information. What can someone buy from you that they can't get for free?
A: We only make the basic searching functions available for free on the site. Professional users – drug companies, insurance companies, hospital groups, healthcare practitioners, etc., need deeper access to the data – and that's what we sell them. We've developed a number of analytic searching and reporting tools that enable them to streamline the searching process, zero in on the specific data that they need, and integrate it into their workflow.
Q: The bottom line for a lot of patients will be whether the drug makes them feel better, even if only for the short term. Long-term risks or seemingly inconsequential side effects – like fatigue or pain at the injection site – can seem like a worthwhile tradeoff for short-term relief. Right now, when I look at your list of drugs with the most side effects, I see that the top two are ranked high because of "injection site pain," but the third, Vioxx, is there because of a heart attack risk. Is there any way to balance the severity of the side effects to give people a sense of which risks they should not ignore?
A: Obviously injection site pain is a much less severe problem than a heart attack. But the reality is that many more people suffer injection site pain than heart attacks from most drugs. That said, the important information is contained in the detailed drug reports that anyone can access on the site. There they can see the full list of actual side effects for each drug and better understand the risk/benefit ratio for any medication they may be taking. Finally, in order for patients to properly weigh the risks and benefits of a given drug, we strongly recommend that they talk with their physician and pharmacists before acting, in any way, on the information we provide.
Related Posts:
Q&A with Keith Hoffman Part 1: Keeping the FDA Honest with Adverse Events Data
Adverse Events: Did a Scrappy Little Start-Up Just Embarrass the FDA?
Photo credit: Thirteen of Clubs via Flickr