Dr. Fallon, director of the Columbia Lyme Center, and colleagues have recently published an open access article (I’ll post the link below): “A reappraisal of the U.S. Clinical Trials of Post-Treatment Lyme Disease Syndrome.” The article reviews the findings of the four clinical trials and accurately lays out the state of the science in chronic Lyme research. This is important because future research needs and policy decisions are determined by the state of the science.

So what does Dr. Fallon say? He makes three important points.  Sample size matters. Differences between patients (so-called heterogeneity) matters. And, finally, that it is important to separate the question of whether treatment is effective from the question of whether the benefits of treatment outweigh the risks. I’m going to tackle these issues one-by-one in a series of blog posts.  This blog will focus on the issue of sample size.

Dr. Fallon describes the four NIH trials, the Krupp Stop-LD study, the Klempner seronegative study, the Klempner seropositive study, and Fallon neurologic study. He then makes a couple of critically important observations.  First, he points out that the sample sizes in all of the studies are quite small and size matters.  Small treatment trials can only detect large treatment effects.  This is why the pharmaceutical companies use large treatment trials that are able to detect much smaller, but clinically important treatment effects. The table below shows the sample sizes of each of the NIH trials. (I’ve combined the two Klempner trials.)

As you can see, they are all quite small pilot studies that are really intended to generate future research rather than provide definitive answers. Dr. Gordon Guyatt, one of the leading figures in evidence-based medicine says this size of study cannot provide definitive answers.  “Providing definitive answers in the face of low event rates and small-to-moderate treatment effects necessitates sample sizes in the thousands or tens of thousands…. Funding for such mega-trials is very limited, and is often restricted to industry sources.”

What does this mean?  Well, we haven’t done the type of trials necessary to provide definitive answers, and it is important for people to recognize that we are dealing with emerging rather than settled science.  Emerging science requires more research and funding for that research.  And, given that the profit potential for treatment of this disease has not attracted pharmaceutical interest, definitive trials are unlikely to be done in the near future.

The other thing that happens when sample sizes are small is that researchers increase the size of the treatment effect that is regarded as treatment success to achieve statistical power. The Klempner trials did this.  They required a treatment effect that was much larger than what a physician would expect or regard as improvement in clinical practice.  But the sample size in Klempner was too small to detect smaller treatment effects so they authors concluded that treatment was not effective.  A more accurate conclusion would have been that because of the small sample size the study could not determine whether a clinically relevant treatment effect existed.

But here’s the remarkable thing—the Krupp study found a large treatment benefit from repeated therapy on fatigue despite having a small sample size.  Moreover, this finding was confirmed by the Fallon study.  This is important.

Fallon concludes by saying: “Treatment guidelines that dismiss the research findings as showing no efficacy do an injustice to the evidence and are not helpful to clinicians and patients.”

My next post will cover Dr. Fallon’s discussion how the NIH trials deal with variation between patients (heterogeneity) and why this matters.

References:

Klempner M, Hu L, Evans J, Schmid C, Johnson G, Trevino R, et al. Two controlled trials of antibiotic treatment in patients with persistent symptoms and a history of Lyme disease. The New England Journal of Medicine. 2001 Jul 12;345(2):85-92.

Krupp LB, Hyman LG, Grimson R, Coyle PK, Melville P, Ahnn S, et al. Study and treatment of post Lyme disease (STOP-LD): a randomized double masked clinical trial. Neurology. 2003 Jun 24;60(12):1923-30.

Fallon BA. Lyme borreliosis: Neuropsychiatric aspects and neuropathology. Psychiatric Annals. 2006;36(2):120-28.

The LYME POLICY WONK blog is written by Lorraine Johnson, JD, MBA, who is the Chief Executive Officer of LymeDisease.org, formerly CALDA. Contact her at lbjohnson@lymedisease.org