Hyping A Supplement for HIV

Dr. Lipson writes his own blog called White Coat Underground, contributes and helps edit at Science-Based Medicine, and contributes to The Science Business Blog at Forbes.com where this piece originally appeared.

When writing about the pharmaceutical industry it's easy to become jaded. This is despite the remarkable medical advances made possible by the drug companies. It seems like each report of sleazy practices is worse than the last. If it isn't unfavorable data being swept under the rug, then it's positive data being over-hyped in remarkably creative ways. So it doesn't surprise me when I get emails asking me to hype the latest new thing.

Not all of these emails are ham-handed; they don't all read like newspaper cut-out ransom notes or Nigerian scams (although many do). Some are quite sophisticated, like a note from a friendly reader, a "just thought you would be interested". The latest missive in my inbox was neither deceptive nor sophisticated. It was from a PR firm offering an "opportunity to interview an esteemed HIV/AIDS and nutrition specialist."

I'm fairly certain my readership isn't so big that I'm going to get a chance at Montagnier, Gallo, or Ho. But the letter says he is "well-known in the global AIDS/HIV community" and "a pioneer". I'm an internist, and I read a lot, but I certainly don't know all of the big players in HIV medicine, so I figured it couldn't hurt to hop onto the internet and learn a little bit more about Dr. Jon Kaiser. According to his publicist:

Dr. Kaiser is very passionate about advocating nutritional supplementation and views it integral to successful disease and general health management. He would more than like to share and engage in a discussion about the supplements and the impact of the proposed reforms to supplement laws. I've included more information on Dr. Kaiser and K-PAX and I have pasted links to a 2001 double-blind, placebo-controlled study funded by Bristol-Myers Squibb that demonstrated an increase in the number of CD4 lymphocytes in HIV-infected patients who were taking Dr. Kaiser's natural immune support formula compared to patients taking a placebo (in addition to standard HIV medications). This formula has been refined over 15 years and is now in clinical trials with the goal of receiving FDA approval. If successful, Dr. Kaiser will be one of the first nutritional suppliers to gain FDA approval for the use of a nutritional supplement as an adjunctive therapy for disease management.

Now that's interesting. The only K-PAX I'd ever heard of spawned an alien named Prot who looks just like Kevin Spacey. So what kind of results is this guy seeing in the clinic?

Though long term stability in my patients has always been the rule, I can now definitely say that the progression of HIV disease in my practice is an extremely rare event. This experience, which has encompassed the care of over 500 HIV(+) patients during the past five years, allows me to make the following statements: Not one patient who has come to me during the past six years with a CD4 count of greater than 300 cells/mm3 has progressed to below that level. Not one patient who has come to me during the past six years with a CD4 count of greater than 50 cells/mm3 has become seriously ill or died from an HIV-related illness.

Wow. That is impressive. The treatment of HIV is very complicated, and it can be difficult for even the best patient remain adherent all of the time. I treat a lot of comparatively less complex treatment problems such as hypertension, and I can't claim that level of success (nor can most of my peers). So who is this guy?

According to the State of California, he is a licensed physician specializing in internal medicine and HIV medicine. This is despite the additional fact that he is not board certified in any specialty. In fact, he's not even board eligible since he completed only one year of post-graduate training. To specialize in internal medicine requires three years post-graduate training, and to specialize in infectious diseases generally requires an additional 1-2 years of post-graduate training. It's not unheard of for older HIV specialists to not have formal infectious disease training---many of them were internists who invented the field of HIV medicine out of necessity when the epidemic first appeared. But to practice internal medicine or infectious disease with only one year of post-graduate training is very, very unusual.

That doesn't mean he's not a good doctor. According to his CV, he is good enough at what he does that his peers have allowed him to serve on a number of boards of an HIV specialist organization called the American Academy of HIV Medicine. When I tried to verify this at the AAHIVM website, I couldn't find him listed, so I wrote his publicist. Dr. Kaiser responded:

As a founding member of the American Academy of HIV Medicine, I was intimately involved in setting policy objectives for the organization in its early development phase beginning in 2000. I formed the Academy's Reimbursement Committee in 2000, and was quoted in the attached AAHIVM newsletter (see middle column, paragraph #2). I have also been certified as an HIV Specialist by the organization on two separate occasions (see attached). It appears my membership to the national organization and California chapter inadvertently lapsed when I moved offices. This has been corrected and my membership is now currently active.

I'll have to take him at his word on that; the attached certificate covers a one year period and AAHIVM's website has yet to be updated. Anyway, given his claims about his successes in the clinic, I was quite curious to check out the data behind his claims. Actually, he hasn't published data on his clinical outcomes, but he has published a brief report in theJournal of Acquired Immune Deficiency Syndromes (JAIDS). In the letter I received from the publicist, Kaiser claims that his supplement K-PAX increases CD4 counts in HIV patients. CD4 cells are the type of cell whose numbers fall so precipitously in patient with AIDS. CD4 cell deficiency is one of the most dramatic laboratory findings in HIV patients, but it is not the only abnormality. Other types of immune cells may be present in normal numbers but function poorly. And CD4 counts, while roughly correlating with the appearance of some opportunistic infections, are not the most useful measure of disease activity in HIV.

In Kaiser's study, published as a "rapid communication", he measured many parameters in addition to CD4 count. This was a very small study (40 patients) with a very brief follow up period (12 weeks). In the results section, the author notes that there were differences in the characteristics between the test and control groups, and that these differences were not statistically significant:

(1) the micronutrient group had a lower CD4 count at baseline when compared with the placebo group (CD4: 357 ± 154 cells/?L vs. 467 ± 262 cells/?L, P = 0.13), (2) the participants in the micronutrient treatment group reported a greater number of months of neuropathy symptoms preceding enrollment than those in the control group (means: 21.4 months vs. 12.2 months, P = 0.14; medians: 14.2 months vs. 2.5 months), and (3) the micronutrient treatment group contained 3 patients with diabetes mellitus compared with zero in the placebo group (P = 0.09).

It is technically correct that most of these differences were not statistically significant. But look at the results:

The mean absolute CD4 count increased significantly by an average of 65 cells in the micronutrient group versus a 6-cell decline in the placebo group at 12 weeks (P = 0.029)

CD4 counts vary quite a bit, and are an imperfect measure of disease activity and immune function. As we can see from his groups, there was a very large range of CD4 counts in each group at the start. An average change in CD4 count of 65 seems anemic at best. Kaiser is more optimistic:

This study demonstrates that a micronutrient supplement administered to HIV-infected patients taking stable HAART significantly enhances CD4 lymphocyte reconstitution. Our findings support the potential for a broad-spectrum micronutrient supplement to be used as adjuvant therapy in combination with HAART to provide patients with a more robust CD4 cell rebound after initiating antiretroviral treatment.

I find the data from this pilot study entirely unconvincing. His conclusions are hyperbolic and premature. That's not unusual, though. Researchers sometimes get a bit excited about their work, and as an inexperienced author, perhaps he can be forgiven for a little unrestrained enthusiasm.

But it gets a bit more interesting that that. In a letter to the editor, a careful reader noted something unusual. A patient showed him a brochure claiming that K-PAX could raise CD4 counts by 26%. K-PAX, it seems, is the same product used in Kaiser's study. The writer was concerned:

Most disturbingly, the first author on the paper, Jon D. Kaiser, MD, seems to be the same person mentioned in the brochure as the developer of K-Pax Vitamins.

Given the recent controversies at other medical journals about the failure of authors to disclose potential conflicts of interest, I am sure that this article would not have been published without disclosure (or at all) if the Editors had been aware of the conflicts of interest in this case.

There is a long, sordid history of conflicts of interest in published research. Some drug companies have gone so far as to print their own faux-journals containing only favorable studies. That is why most journals have strict disclosure rules for conflicts of interest. A study being sponsored by a drug manufacturer does not invalidate it, but failure to disclose this connection is unethical and problematic on many levels.

Kaiser was (correctly) allowed to respond:

After reporting the improved immune reconstitution of patients taking the micronutrients plus highly active antiretroviral therapy (HAART) to the Bristol-Myers Squibb team, I anticipated that they would show interest in pursuing the development of a therapy that had the potential to act as a safe and beneficial immune modulator.

On learning that they had no interest in pursuing the development of this compound, I chose to form a company, K-PAX, Inc., to keep the micronutrient supplement in production and make it available for sale while I worked to get the study results published.

I neglected to inform the Editors of this journal of this conflict of interest and any potential bias that existed during the paper's submission and publication process. Nor did I inform the other authors on the paper of my financial interest in the company. They received no personal compensation for their efforts.

In other words, he is the guy who makes and sells K-PAX, and he owned up to failing to disclose this profound conflict of interest. I asked Kaiser through his publicist about this. His response was less conciliatory:

This research study was performed before there ever was a company or financial interest in a product (2001-2003). The data were analyzed by an independent Data Analysis Firm selected by Bristol-Myers Squibb in 2001. The JAIDS editorial board performed a thorough evaluation after Dr. Smith raised his concerns and found no evidence that either the data analysis or study conclusions were inaccurate.

I'm unimpressed by this response. The use of one small study with an undisclosed conflict of interest and to promote a product makes me uncomfortable. I am not comforted by the claims of "independent data analysis". Conflicts of interest can affect more than just data analysis: they can poison every step of a study. But even had this study been done without any conflicts of interest, it's a small, problematic study. What's worse is in the letter sent to me, he says

Building upon his success in HIV patients, Dr. Kaiser is applying the lessons learnt in HIV to patients with cancer, Chronic Fatigue Syndrome and other chronic diseases, general health and the elderly.

HIV is a serious disease, the study of which underlies many modern advances in biology and medicine. To try to piggyback one's own interests on this foundation is unseemly at best. To then claim to apply these ideas even more broadly is deeply unsettling.

(Dr Kaiser has responded to this piece, and I have responded to him at White Coat Underground. )