While the necessity of proper control conditions in evaluating the efficacy or effectiveness of psychotherapies, or indeed of any health intervention, is undeniable, what exactly constitutes an appropriate control for an active intervention is far from clear.

In a new meta-analysis published in Acta Psychiatrica Scandinavica, Furukawa and collaborators comparatively examined three types of control conditions (no treatment, waiting list and psychological placebo) in trials of cognitive behavioral therapies (CBT) for adult depression.

Methods

The authors used a meta-analytic methodology called network meta-analysis (NMA). While traditional meta-analysis only allows for the possibility of comparing two interventions at a time, network meta-analysis permits the evaluation of the relative efficacy of all the included interventions, by integrating data both from direct head-to-head comparisons, as well as indirect evidence from studies with common comparators. In this way, NMA borrows strength from the entire network of randomised trials.

The authors included randomised clinical trials in adults, comparing CBT with a control condition. Three control conditions were considered:

1. Psychological Placebo (PP): the condition was considered as inactive by the researchers conducting the study, but was presented as potentially active to the participants. Moreover, the number of sessions and their duration, as well as the qualifications of the therapists providing the intervention, had to be equivalent to the active treatment in the study
2. No Treatment (NT): participants received no active treatment during the study and they did not expect to receive it after the study was over
3. Wait-list (WL): participants received no active treatment during the study, but were told they could receive one such treatment after the study was over

The studies were included if the primary clinical problem treated was depression, evaluated either by a diagnostic interview of by self-report scales. The authors excluded studies focusing on chronic or treatment-resistant depression, and studies where participants had another concurrent primary diagnosis, whether of a mental or physical disorder.

The primary outcome of the meta-analysis consisted of the number of patients who responded to treatment, based on changes on the Hamilton Rating Scale for Depression (HAMD), the Beck Depression Inventory (BDI) or any other validated depression scale, at the end of the acute phase of treatment. Analyses were, in as much as possible, intent-to-treat, meaning that they took into account the total number of randomly assigned participants, irrespective of how the investigators in the original studies analyzed their data.

Two independent raters evaluated the included studies for methodological quality, using the Risk of Bias tool developed by Cochrane Collaboration and some additional criteria such as researcher allegiance, therapist allegiance, therapist qualification and treatment fidelity.

The authors first examined individual trials grouped in pairwise, head-to-head meta-analysis and subsequently constituted the evidence network for the NMA and looked at its robustness. They evaluated comparative response rates of CBT and the three control conditions, by reporting odds ratios (OR) and corresponding 95% credible intervals of the OR. Briefly put, the odds of an event represents the number of those who experience it divided by those who don’t and it is expressed by a number from zero (the event will never happen) and infinity (the event will definitely happen).

Results

The meta-analysis included 49 randomised studies, representing 117 treatment arms.

• The results of the NMA indicated a well-connected, homogeneous and consistent network of evidence connecting CBT and the three types of control groups (PP, NT and WL) in the acute phase of treatment of adult depression
• Effect sizes for CBT were substantively different, depending on which control condition they were compared to:
  • Psychological placebo (PP): OR= 1.65 (0.76 to 3.13)
  • No Treatment (NT): OR= 2.36 (1.31 to 4.26)
  • Wait-list (WT): OR= 6.26 (3.90 to 10.1)
• In other words, WL control may generate bigger effect sizes estimates for CBT than NT or PP control
• Moreover, the indirect comparison between NT and WL revealed the former was significantly superior to the latter in eliciting response: OR= 2.9 (1.3-5.7)
• However, the authors note that the statistically significant differences between conditions were lost when they applied exploratory sensitivity analyses to correct for small study effects (i.e. small studies having systematically different effects to the large studies)

Conclusions

The authors concluded that:

The currently available best evidence, analysed by use of NMA, suggested that different control conditions lead to substantively different treatment effect estimates and that WL control may generate bigger effect size estimates for CBT than NT or PP.

They also added that:

In other words, WL could be regarded a nocebo condition if it indeed is inferior to NT, that is, doing nothing.

The authors speculate that participants allocated to WL might be more motivated to remain depressed so they can receive their originally desired treatment (CBT) after the completion of the study period, while those allocated to NT may actively seek alternative treatments.

Limitations

• The methodological quality of the included studies was sub-optimal, as already remarked by other meta-analyses on CBT and discussed in previous Mental Elf blogs. In fact, only a quarter of the included studies were rated as having a low risk of bias
• Importantly, small study effects (i.e. small studies having systematically different effects to the large studies) were notable, particularly in the comparison between CBT and WL

Link

Furukawa TA, Noma H, Caldwell DM, Honyashiki M, Shinohara K, Imai H, Chen P, Hunot V, Churchill R. Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta Psychiatr Scand. 2014 Apr 4, doi:10.1111/acps.12275 [PubMed abstract]

Sedgwick P. The nocebo effect. BMJ 2013;347:f6130 [Abstract]