Disclosures: I am currently completing the third revision of the British Association for Psychopharmacology (BAP) Bipolar treatment guideline. I have advised many pharmaceutical companies doing trials in the bipolar therapeutic area. I was Principle Investigator (PI) for a trial of individual psychoeducation for bipolar disorder (OXTEXT6 – in submission).

The revised NICE Guideline for bipolar disorder was published in 2014 (NICE, 2014). Psychological treatments were recommended:

• As the primary modality of treatment in primary care for bipolar depression
• As equivalent to medication in the management of bipolar depression in secondary care
• Sharing equal importance with medication in the long term

To what extent are these ‘Key priorities for implementation’ supported by evidence from clinical trials?

Methods

Sameer Jauhar, Peter McKenna and Keith Laws have re-examined the analyses of trial data from the NICE website and added a small meta-analysis to re-allocate data omitted by the NICE group. Their ‘personal view’ article was published in the Lancet Psychiatry on 4th February 2016 (Jauhar et al., 2016).

Results

NICE documented around 170 meta-analyses of individual psychological interventions, and several ‘composites’ of different therapies together. No strategy was described to handle the problem of multiple analyses (each declared significant at p<0.05). Each meta-analysis contained only a small number of trials. The largest contained six, over half consisted of only one!

Cognitive Behaviour Therapy (CBT) figured largely in the recommendations. However:

• Analysis of six trials of individual versus treatment as usual gave a standardised mean difference of −0.31 at post-treatment, which was not maintained at follow-up
• Two meta-analyses of group CBT reported no benefit at post-treatment or at follow-up
• When CBT was compared with an active control (supportive therapy), the control intervention was superior
• Other interventions considered showed a similarly inconclusive patterns.

NICE undertook more than 30 additional meta-analyses in which data from different types of psychological intervention were pooled. Thus, for relapse-prevention:

• The effect of psychoeducation, therapy for treatment adherence, and CBT, compared with treatment as usual (TAU) (five studies), was significant (RR 0.74, 0.63 to 0.87)
• Yet another five studies of three group interventions (psychoeducation, CBT, and mindfulness-based cognitive therapy) were not (RR 0.86, 0.61 to 1.20)
• The largest single study of CBT in relapse prevention was intended to be definitive following the publication of smaller, positive, exploratory studies. It was completely negative, even though it was CBT versus TAU, and favoured some kind of positive outcome. However, it was omitted from the ‘meta-analysis’ that was used to support the conclusion that psychotherapy is useful in relapse prevention.

Conclusions

The authors state bleakly that many clinicians reviewing the available evidence would not come to the same strong conclusions that NICE did. I agree, and I know that expert opinion in this country agrees too.

Strengths and weaknesses

It is a massive achievement that methodology for psychotherapy trials exists at all. The systematic development of CBT for anxiety and eating disorders has transformed psychiatry in my working lifetime. Many therapists have resisted the idea that their work could be evaluated in any meaningful way by so vulgar a process as a clinical trial. So, a therapy based on coherent cognitive theory, developed by experimental observation and tested in as fair a way as possible in a clinical trial is something to be acknowledged and implemented widely for patient benefit.

The fundamental problem is that in bipolar disorder there is no adequate theoretical model yet, with the exception of its most pragmatic incarnation, psychoeducation. If we could trial psychoeducation against psychoignorance, it is hard not to guess which is preferable. However, most other approaches, especially CBT, are borrowed from the unipolar world. The testing of therapies without adequate theory or understanding of mediating mechanisms lays investigators open to a variety of problems which may devalue the primary evidence, let alone a phoney secondary analysis.

What is placebo in a psychotherapy trial?

The choice of a fair comparison treatment is much more difficult than for a medicine where a placebo tablet of identical appearance can be used. It is often simply a poorly specified ‘treatment as usual’ (TAU) condition. When, as is commonly the case, the active treatment is superior to TAU, no specificity can be claimed for the content. Any equivalent contact time with interested staff might generate the same advantage.

An alternative to TAU is a ‘waiting list’ control group. This is also problematic because any subsequent benefit for an active treatment may be due to (or amplified by) a waiting list’s potential nocebo effect. So we often do not know what elements of a psychotherapy are actually effective when a trial is positive. A specific claim for efficacy requires comparison with a plausible control procedure that engages patients and therapist in a relationship which omits only the putative active ingredients of the therapy.

The problem of scale and allegiance bias

Psychotherapy trials are often small scale and particularly at risk of allegiance bias; i.e. investigators are heavily invested personally and professionally in showing that their treatment works. This may consciously or unconsciously influence how treatments are delivered and results described.

At a most basic level, patients, therapists and, most critically, raters are rarely blind to the treatment. The lack of blinding matters a lot, but it introduces bias that is hard to quantify. Publication bias is objectively measurable and has been shown to be as important a problem for psychotherapy trials as for drug trials (Flint et al., 2015).

Lack of disclosure

If one’s work is published in a high profile journal, it could increase the interest in workshops and professional training that the PI provides. And for such additional work, there may be substantial remuneration. In addition, a particular psychotherapy may be endorsed in a guideline, which may lead to additional rewards via HEFCE’s research assessment process.

Systematic searching for disclosure of interest in descriptions of psychosocial interventions suggests most PIs declare none (Dragioti et al., 2015). And, clinical psychologists who were part of the bipolar Guideline Development Group for NICE made no disclosures either, except of funding for other trials (see Appendix 2 of NICE Bipolar Guideline). There may be no concealment intended, but unquestioning positive self-regard may be a problem.

Harms

Finally, the collection of ‘adverse reactions’ to psychological treatment also appears to be unsystematic and hence under-appreciated (Nutt and Sharpe, 2008). So benefit-risk estimates tend to be optimistic.

Given that patients with bipolar disorder can be potentially disinhibited, the risks that a therapist (who may often be unregulated by medical standards) be tempted to take advantage of the intimacy that comes from therapeutic contact are not negligible, but they are almost never considered. Indeed, NICE itself seemed to regard psychosocial interventions as virtually harmless.

Summary

Generic problems provide a counterpoint to the ‘positive regard’ (Nutt and Sharpe, 2008), which may characterise NICE’s approach to psychosocial interventions. Jauhar and colleagues have done some hard yards to identify bias in the NICE methodology. What they conclude is damning. But, make up your own minds whether NICE was biased or not. It matters.

Links

Primary paper

Jauhar S, McKenna PJ and Laws KR. (2016) NICE guidance on psychological treatments for bipolar disorder: searching for the evidence. The Lancet Psychiatry. 4 Feb 2016 DOI: 10.1016/S2215-0366(15)00545-3 [Paper open access until 19/2/16]

Other references

NICE (2014) Bipolar disorder: assessment and management. NICE guideline CG185, Sep 2014.

Dragioti E, Dimoliatis I and Evangelou E. (2015) Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal. BMJ Open 5: e007206.

Flint J, Cuijpers P, Horder J, Koole SL, Munafò MR. (2015) Is there an excess of significant findings in published studies of psychotherapy for depression? Psychol Med 45: 439-446.

Nutt DJ and Sharpe M. (2008) Uncritical positive regard? Issues in the efficacy and safety of psychotherapy. Journal of psychopharmacology (Oxford, England) 22: 3-6. [Abstract]

Photo credits

• Matt Biddulph CC BY 2.0
• Kenny Louie CC BY 2.0
• Kit MacAllister CC BY 2.0
• LaurPhil CC BY 2.0
• joeymanley CC BY 2.0
• Roland O’Daniel CC BY 2.0