Medications. Comparing Antidepressants to Placebo

Listening Closely to Prozac…

In the mid-1990s, psychologist Irving Kirsch (and his colleague Guy Sapirstein) wanted to determine the size of the placebo effect in clinical depression. The result was an explosion of literature that has been calling some of our most widely-accepted practices into question.

Recently Kirsch published a book, The Emperor's New Drugs, that summarizes the work from his perspective (which is unabashedly skeptical about the antidepressants, as the title suggests).

In his original study, Kirsch collected published studies of depression treatment that included a pill placebo condition – studies originally intended to examine antidepressant medication effectiveness.

Once you get a lot of similar studies on a subject, you can combine them and perform a meta-analysis.  In effect, this is like treating the participants in all of these separate studies as though they were in one giant study. A meta-analysis has a tendency to smooth over irregularities between trials and can show the general trend of results. On the other hand, they can also obscure some of the nuances of individual trials.

Fortunately, some of the trials included a no-treatment condition. This allowed Kirsch to compare the response to placebo to the response associated with just the passage of time. Some people get better on their own, after all, and there are a few other factors (reviewed in a recent post, here) that can otherwise inflate an apparent placebo effect.

Sure enough, Kirsch found that no-treatment groups had a small tendency to improve (Kirsch & Sapirstein, 1998). The placebo groups improved much more, however, indicating that the belief that one was receiving an active treatment had a significant therapeutic effect.

The surprise for Kirsch was that patients receiving antidepressant medications did not improve much more than those receiving placebo. In fact, the placebo group improved about 75% as much as the medication group. Put another way, about ¾ of the medication effect could be accounted for by expectancy or by spontaneous recovery with the passage of time. The active ingredient in the medication didn’t seem to add much potency.

In addition, Kirsch knew about the problem discussed a few weeks back: antidepressants have side effects and placebos do not, so patients receiving the antidepressants in such studies often figure out which study condition they are in. Suspecting they are getting the active drug, their expectancy effect may be greater. Kirsch raised the possibility that even the small difference between placebo conditions and active antidepressant conditions may be partly an expectancy effect rather than being attributable to the medication’s active ingredient.

This study created quite a stir when it was published, as it called the effectiveness of antidepressants into question. These are some of the most widely prescribed medications in North America, and it has been generally accepted that they work.

But what about the unpublished studies?

Kirsch had easy access to all of the published placebo-controlled antidepressant studies. But he knew that some of these studies never see the light of day. It’s easier to get a study published if it shows positive results than if it shows no difference between conditions. And perhaps the companies funding the studies would be less than enthusiastic about publishing data suggesting that their medications weren’t very effective.

The US Food and Drug Administration requires that all of the existing studies, published or not, be submitted to them before a drug can be approved. So Kirsch knew the FDA had them. He used a Freedom of Information request to obtain the complete data set and conducted another meta-analysis.

The results were even more discouraging than in his previous study. Response to the placebo condition was 82% as great as to the active medications. Only 18% of the response in medication conditions could be attributed to the chemical effect of the drug. And there was still the problem of unblinding of the study, because subjects could guess whether they were getting the placebo or the antidepressant.

The difference between placebo and antidepressant was still statistically significant. But Kirsch asked whether it was clinically significant. It amounted to an average difference of 1.8 points on the 51-point Hamilton Rating Scale for Depression.

I’ve treated depression for many years, and I can usually hazard a guess about where a client will score on the depression scales I use. But I wouldn’t be able to reliably guess within 1.8 points of their score – the difference is simply too slight. So even if there was a genuine chemical effect, it's questionable whether it was important from a patient's perspective.

Is there a publication bias favouring positive studies?

A subsequent study (Turner, Matthews, Linardatos, Tell, & Rosenthal, 2008) looked at whether there was a systematic difference between the published and unpublished drug trials. Of the 69% of studies that were published, 94% showed a positive result favouring the antidepressant being tested.

Of all the studies judged to exhibit a superiority of antidepressant over placebo, 37 were published and just one was not. Of those judged to have negative or equivocal results, 22 were not published, 3 were published with the negative result made explicit, and 11 were judged by Turner et al to have been interpreted in a misleadingly positive light.

A scrupulous reader evaluating the published literature would conclude that 94% of studies supported the effectiveness of antidepressants. One able to read all of the studies would find that only 51% were positive.

Whether or not a given trial supports a difference between drug and placebo is one thing; the size of that difference is another. Turner et al examined the difference in effect size between antidepressant and placebo for the total data set, and compared this to the difference found in the published studies. On average, the effect size was 32% higher in the published data than in the complete set of FDA-submitted studies.

This points to a serious problem. Clinicians are expected to follow the published literature and guide their practices accordingly. But a publication bias in favour of positive studies skews the picture.

Imagine a dutiful clinician asking each researcher whether antidepressant medications outperformed placebo in a given trial.  “Yes, no, no, yes, yes, no, yes, no, no…”

Then imagine the same clinician looking at the published literature.  “Yes, yes, yes, yes, yes, no, yes, yes, yes…” The resulting treatment decisions would likely differ markedly.

So do antidepressants work?

Let’s be clear. People who received antidepressants improved a great deal, on average. So yes, the medications work.

The question is why. Patients receiving placebos improved a great deal as well, though not quite as much as the patients receiving antidepressants. It is certain that much of the improvement we see in patients prescribed antidepressants is the result of expectancy rather than chemistry.

But is there a chemical effect too?

That remains unclear. It seems likely that there is, but we know that it is somewhat unreliable. We know that people who suddenly stop taking antidepressants will often experience a rebound depression, which suggests the medications must be doing something, even if we aren’t sure just what.

At this point I’m not willing to declare that antidepressants are placebos. I’ve seen too many remarkable effects in too many clients to dismiss the effectiveness of these medications out of hand.

My suspicion is that depression, like fever, is a symptom of many possible conditions, and that antidepressants may well help with some of these ailments but not others. I would guess that in 20 years we will look back and see that these weak effects were obtained by including a very diverse group of problems in the medication trials. (I've advanced this idea before on this blog, and I suspect that some of the more aggressive medication critics would snort at me for my timidity on this issue.)

Kirsch, who argues forcefully that there is no convincing evidence in favour of the antidepressants, would disagree with me. And his grasp of the literature is unquestionably better than mine. Others with expertise in this area have significant reservations about Kirsch’s work. At this point, a consensus has yet to emerge.

So: Is there a chemical antidepressant effect? At this point the answer remains unclear and unsettled in the research community.

References

Kirsch, I (2009) The Emperor's New Drugs: Exploding the antidepressant myth. London: Bodley Head.
Kirsch, I, Moore, TJ, Scoboria, A, & Nicholls, SS (2002).  The emperor’s new drugs: An analysis of antidepressant medication data submitted to the US Food and Drug Administration.  Prevention and Treatment, Vol 5, Article 23.
Kirsch, I, & Sapirstein, G (1998) Listening to Prozac but hearing placebo: A meta-analysis of antidepressant medication. Prevention and Treatment, Vol. 1, Article 2a.
Turner, EH, Matthews, AM, Linardatos, E., Tell, RA, & Rosenthal R. (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine, 358, 252-260.