Thank the Great Elf Mother – it’s Friday! Time to sit down with a few elfin chums to take on the task of accurately reviewing what a Meat Feast pizza really tastes like – we’re fun like that!

You slice it up and start work, but you notice already that everyone’s slice is a little bit different:

• The cheery elf with loads of meat on her slice says it does the job as advertised
• The poor elf in the corner looks close to weeping big fat tears into his dangly cheese slop of a slither, but at least his watery eyes blind him to his neighbour’s good fortune
• And one grumpy little elf only chews the crust and declares it not even a pizza

You wonder why you decided to assign the task to several people, and later re-order the pizza to consume by yourself; you are nauseous, confused and have no means of describing all that you have taken in.

Reviewing Clarke et al’s paper hot on the heels of the review of Biesheuvel-Leilefeld et al’s paper (see Raluca Lucacel’s blog) offers an opportunity to consider how two groups of researchers effectively asking the same question of the existing published data set (what is the effectiveness of psychological interventions at preventing relapses of depression?) come to similar but different conclusions; it’s a meat pizza, but Biesheuvel-Leilefeld et al seem to have got more pepperoni.

Both sets of authors are tackling a clinically pertinent question: can psychological interventions prevent people in remission from depression from relapsing at least as effectively as medication?

Many people do not want to continue taking antidepressant medication for prolonged periods, either because of side effects or because of feelings of dependency, or simply because they feel uncomfortable having to take any kind of medication everyday. The literature available on the topic is seen by both research teams as lacking in quality and being at considerable risk of publication bias.

There are also so many mediating and moderating variables in the shape of demographics depression history and past treatments; things that we already know influence relapse rates in depression, that conducting a meta-analysis may seem flawed from the start. However, evidently the two independent sets of researchers saw it as the best option for moving the field forward.

Methods

Faced with such a mixed bag of studies, what inclusion criteria would you impose?

• Would you only include English studies (Biesheuvel-Leilefeld et al)?
• Would you pool all psychological interventions and look separately at comparisons to medication and to any other treatment-as-usual (Biesheuvel-Leilefeld et al)?
• Or would you separate out cognitive behavioural therapy (CBT), mindfulness-based cognitive therapy (MBCT) and interpersonal psychotherapy (IPT), but compare them to all kinds of other non-psychological interventions (Clarke et al)?
• Do you look separately at relapse rates at 12 months and 24 months (Clarke et al) or just at 24 months (Biesheuvel-Leilefeld et al)?
• How long is long enough to look at relapse risk?
• And does any of this really matter when the quality of the included studies is questionable?

I put these out as general consideration points before turning to the task in hand of reviewing Clarke et al’s paper. There is, however, another rather specific point I wish to raise that seems to have a considerable influence on the results.

Both reviews include a paper that looks at 6 years of follow-up for 40 patients in 2 conditions (Fava et al, 1998). This paper is actually a brief report article, but somehow both review teams have extracted relative risk (RR) values at their time-point of interest:

• Clarke et al say that the RR is 0.11 at 12 months;  an amazingly large value that has a massive pull on their overall result for CBT (and a value with a very wide confidence interval).
• Biesheuvel-Leilefeld et al, from the same paper, derived an RR of 0.667 at 24 months.

Neither paper explains how they reached their result (although one can presume from contacting the authors) but this shows how small trials can have a really disproportionate effect on meta-analysis outcomes, and how a decision to look at 12 or 24 months as a period of detecting relapse in depression could completely alter your conclusions.

In fairness to both groups of researchers, particularly Clarke et al, the potential for one or two studies to influence the results is flagged up and raised as a point for future researchers.

Results

Let’s conclude with a visual summary of the “whose slice is it anyway?” problem (now might be a good time to call up the local pizza parlour before reading my critical appraisal of this study).

Clarke et al’s paper has a clearly presented literature search and quality appraisal section, complete with a flow diagram explaining exclusions at each step:

• 22 studies were included in the meta-analysis, which were the studies involving either CBT, IPT or MBCT.
• 7 further studies using ‘service-level programmes’ or other psychological therapies were still included in the review but were only evaluated qualitatively.
• As noted above, the authors combined all types of control conditions for comparison to psychological intervention.

Tabulation of the included studies makes explicit just how varied the individual research projects are, in scale, timeframe and in baseline levels of depression. Given the relatively low numbers of studies being analysed, the quality issues of the individual studies and the heterogeneity of study design, the results as presented here feel to me to be an indicator of potential evidence of efficacy rather than results in their own right.

Given that caveat, the study presents the finding that at:

• 12 months the relative risk of relapse compared to ‘control’ was lowered by:
  • 21% for MBCT
  • 22% for IPT
  • 25% for CBT

Where data was available for:

• 24 months (for 7 CBT trials and all 4 IPT trials):
  • CBT appeared to provide continued risk reduction at 28%, but
  • IPT was no longer significantly more helpful than ‘control’ conditions

Limitations

In interpreting their findings, the authors consider the similarity of the relative risk levels across the three styles of psychological intervention and suggest a non-specific therapeutic effect may be the active ingredient in risk reduction.

It is a shame, given this, that the authors did not then look at an overall comparison of all psychological treatments to medication and non-medication control conditions (though the authors contend that they could not be certain if non-medication controls were definitely not taking medication).

It is also unfortunate that they did not try to marry this hypothesis with the lack of an effect found for service-level interventions, as this stands against a simplistic input-effect theory.

Conclusions

In conclusion, it feels as though Clarke et al have had a nibble at an under-baked pizza and found they don’t really like the taste. It might be a good pizza with a few more minutes in the oven, but equally it might never taste satisfying, and actually the problem is that right now there are not enough ingredients to really make a pizza at all.

Links

Primary paper

Clarke K, Mayo-Wilson E, Kenny J, Pilling S. (2015) Can non-pharmacological interventions prevent relapse in adults who have recovered from depression? A systematic review and meta-analysis of randomised controlled trials. Clinical Psychology Review, 39, 58-70. [PubMed abstract]

Other references

Biesheuvel-Leliefeld KEM, Kok GD, Bockting CLH, Cuijpers P, Hollon SD, van Marwijk HWJ, Smit F. (2015) Effectiveness of psychological interventions in preventing recurrence of depressive disorder: Meta-analysis and meta-regression. Journal of Affective Disorders, 174C, 400-410. 10.1016/j.jad.2014.12.016

Lucacel, R. Psychotherapy can reduce the risk of relapse or recurrence of depression. The Mental Elf, 15 Jun 2015.

Fava GA, Rafanelli C, Grandi S, Conti S, Belluardo P. (1998) Prevention of recurrent depression with cognitive behavioral therapy: Preliminary findings. Archives of General Psychiatry 1998, 9, 816–820.