Disclosures: I am currently completing the third revision of the British Association for Psychopharmacology (BAP) Bipolar treatment guideline. I have advised many pharmaceutical companies doing trials in the bipolar therapeutic area. I was Principle Investigator (PI) for a trial of individual psychoeducation for bipolar disorder (OXTEXT6 – in submission).
The revised NICE Guideline for bipolar disorder was published in 2014 (NICE, 2014). Psychological treatments were recommended:
- As the primary modality of treatment in primary care for bipolar depression
- As equivalent to medication in the management of bipolar depression in secondary care
- Sharing equal importance with medication in the long term
To what extent are these ‘Key priorities for implementation’ supported by evidence from clinical trials?
Sameer Jauhar, Peter McKenna and Keith Laws have re-examined the analyses of trial data from the NICE website and added a small meta-analysis to re-allocate data omitted by the NICE group. Their ‘personal view’ article was published in the Lancet Psychiatry on 4th February 2016 (Jauhar et al., 2016).
NICE documented around 170 meta-analyses of individual psychological interventions, and several ‘composites’ of different therapies together. No strategy was described to handle the problem of multiple analyses (each declared significant at p<0.05). Each meta-analysis contained only a small number of trials. The largest contained six, over half consisted of only one!
Cognitive Behaviour Therapy (CBT) figured largely in the recommendations. However:
- Analysis of six trials of individual versus treatment as usual gave a standardised mean difference of −0.31 at post-treatment, which was not maintained at follow-up
- Two meta-analyses of group CBT reported no benefit at post-treatment or at follow-up
- When CBT was compared with an active control (supportive therapy), the control intervention was superior
- Other interventions considered showed a similarly inconclusive patterns.
NICE undertook more than 30 additional meta-analyses in which data from different types of psychological intervention were pooled. Thus, for relapse-prevention:
- The effect of psychoeducation, therapy for treatment adherence, and CBT, compared with treatment as usual (TAU) (five studies), was significant (RR 0.74, 0.63 to 0.87)
- Yet another five studies of three group interventions (psychoeducation, CBT, and mindfulness-based cognitive therapy) were not (RR 0.86, 0.61 to 1.20)
- The largest single study of CBT in relapse prevention was intended to be definitive following the publication of smaller, positive, exploratory studies. It was completely negative, even though it was CBT versus TAU, and favoured some kind of positive outcome. However, it was omitted from the ‘meta-analysis’ that was used to support the conclusion that psychotherapy is useful in relapse prevention.
The authors state bleakly that many clinicians reviewing the available evidence would not come to the same strong conclusions that NICE did. I agree, and I know that expert opinion in this country agrees too.
Strengths and weaknesses
It is a massive achievement that methodology for psychotherapy trials exists at all. The systematic development of CBT for anxiety and eating disorders has transformed psychiatry in my working lifetime. Many therapists have resisted the idea that their work could be evaluated in any meaningful way by so vulgar a process as a clinical trial. So, a therapy based on coherent cognitive theory, developed by experimental observation and tested in as fair a way as possible in a clinical trial is something to be acknowledged and implemented widely for patient benefit.
The fundamental problem is that in bipolar disorder there is no adequate theoretical model yet, with the exception of its most pragmatic incarnation, psychoeducation. If we could trial psychoeducation against psychoignorance, it is hard not to guess which is preferable. However, most other approaches, especially CBT, are borrowed from the unipolar world. The testing of therapies without adequate theory or understanding of mediating mechanisms lays investigators open to a variety of problems which may devalue the primary evidence, let alone a phoney secondary analysis.
What is placebo in a psychotherapy trial?
The choice of a fair comparison treatment is much more difficult than for a medicine where a placebo tablet of identical appearance can be used. It is often simply a poorly specified ‘treatment as usual’ (TAU) condition. When, as is commonly the case, the active treatment is superior to TAU, no specificity can be claimed for the content. Any equivalent contact time with interested staff might generate the same advantage.
An alternative to TAU is a ‘waiting list’ control group. This is also problematic because any subsequent benefit for an active treatment may be due to (or amplified by) a waiting list’s potential nocebo effect. So we often do not know what elements of a psychotherapy are actually effective when a trial is positive. A specific claim for efficacy requires comparison with a plausible control procedure that engages patients and therapist in a relationship which omits only the putative active ingredients of the therapy.
The problem of scale and allegiance bias
Psychotherapy trials are often small scale and particularly at risk of allegiance bias; i.e. investigators are heavily invested personally and professionally in showing that their treatment works. This may consciously or unconsciously influence how treatments are delivered and results described.
At a most basic level, patients, therapists and, most critically, raters are rarely blind to the treatment. The lack of blinding matters a lot, but it introduces bias that is hard to quantify. Publication bias is objectively measurable and has been shown to be as important a problem for psychotherapy trials as for drug trials (Flint et al., 2015).
Lack of disclosure
If one’s work is published in a high profile journal, it could increase the interest in workshops and professional training that the PI provides. And for such additional work, there may be substantial remuneration. In addition, a particular psychotherapy may be endorsed in a guideline, which may lead to additional rewards via HEFCE’s research assessment process.
Systematic searching for disclosure of interest in descriptions of psychosocial interventions suggests most PIs declare none (Dragioti et al., 2015). And, clinical psychologists who were part of the bipolar Guideline Development Group for NICE made no disclosures either, except of funding for other trials (see Appendix 2 of NICE Bipolar Guideline). There may be no concealment intended, but unquestioning positive self-regard may be a problem.
Finally, the collection of ‘adverse reactions’ to psychological treatment also appears to be unsystematic and hence under-appreciated (Nutt and Sharpe, 2008). So benefit-risk estimates tend to be optimistic.
Given that patients with bipolar disorder can be potentially disinhibited, the risks that a therapist (who may often be unregulated by medical standards) be tempted to take advantage of the intimacy that comes from therapeutic contact are not negligible, but they are almost never considered. Indeed, NICE itself seemed to regard psychosocial interventions as virtually harmless.
Generic problems provide a counterpoint to the ‘positive regard’ (Nutt and Sharpe, 2008), which may characterise NICE’s approach to psychosocial interventions. Jauhar and colleagues have done some hard yards to identify bias in the NICE methodology. What they conclude is damning. But, make up your own minds whether NICE was biased or not. It matters.
Jauhar S, McKenna PJ and Laws KR. (2016) NICE guidance on psychological treatments for bipolar disorder: searching for the evidence. The Lancet Psychiatry. 4 Feb 2016 DOI: 10.1016/S2215-0366(15)00545-3 [Paper open access until 19/2/16]
NICE (2014) Bipolar disorder: assessment and management. NICE guideline CG185,
Dragioti E, Dimoliatis I and Evangelou E. (2015) Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal. BMJ Open 5: e007206.
Flint J, Cuijpers P, Horder J, Koole SL, Munafò MR. (2015) Is there an excess of significant findings in published studies of psychotherapy for depression? Psychol Med 45: 439-446.
Nutt DJ and Sharpe M. (2008) Uncritical positive regard? Issues in the efficacy and safety of psychotherapy. Journal of psychopharmacology (Oxford, England) 22: 3-6. [Abstract]
RT @Mental_Elf: Today Prof Guy Goodwin on @sameerjauhar @keith_laws paper about psychological treatments for bipolar disorder https://t.co/…
Join the debate on the new Laws/Jauhar paper at the @Mental_Elf #ElfCampfire next week: https://t.co/YxpB5HuE9B
Morning @bengoldacre What do you think of our blog today? https://t.co/R8aFf6OfIX #bipolar #psychotherapy #bias #disclosure #NICE
@Mental_Elf @bengoldacre Worrying anti-science helps neither patients nor practitioners. Does this reflect a trend in mental healthcare?
@Mental_Elf @Si1verSurfer @bengoldacre serial problem with NICE evaluations of effectiveness of psychotherapy for psychotic conditions
Is the NICE guideline for bipolar disorder biased in favour of psychosocial… https://t.co/byLco7OTsR #MentalHealth https://t.co/lGdKMCvtFo
.@Mental_Elf Bias works for and against- meta-analysis is best when team includes treatment expert https://t.co/oWBpYVtAeq
@TilWykes I am somewhat puzzled about what evidence shows that “meta-analysis is best when team includes treatment expert”?
Indeed, evidence suggests that the opposite is true.
For example, Dragioti et al (2015) recently assessed researcher allegiance bias in meta-analyses and RCTs of psychotherapy. http://bmjopen.bmj.com/content/5/6/e007206.full They found that researcher allegiance was present in 40% of psychotherapy meta-analyses they sampled. And of those allegiant meta-analyses around 90% had an author who had also been a co-author of an allegiant RCT included in the Meta-analysis
…and lots of evidence shows that researcher allegiance inflates effect sizes in psychotherapy trials
It seems then that much better arguments can be made for excluding so-called ‘experts’ from meta-analyses
I can understand the advantage of people with expertise to give advise on aspects of analysis. There is a difference between advise and advocating for a treatment Both NICE guidelines for SCZ & bipolar do seem to cross line to advocacy not based on best analysis of evidence. The decisions made in analysing the evidence were not the best. Perhaps on balance, less “expertise” on the treatment and more “expertise” on analysing evidence
I personally think therapy research is perhaps not measuring the right outcomes It would be better measuring “soft” but vital outcomes such as greater self understanding, placing experiences in context etc as a complement to medical treatment
@TilWykes Indeed, but isn’t meta-analysis also best when it includes the appropriate studies? @sameerjauhar @Keith_Laws @TheLancetPsych
@Mental_Elf @TilWykes @Keith_Laws @TheLancetPsych all of the above.
Superb summary of very useful paper https://t.co/txUaNRfgPG
Interesting reading for @CNWLNHS staff on bipolar disorder https://t.co/00mXCVrD1c
NICE recommend psychological treatments for bipolar depression, but is there enough evidence to back this up? https://t.co/R8aFf6OfIX
NICE recommend psychosocial interventions for bipolar depression – why? https://t.co/4OGEVM05Xo via @Mental_Elf
Powerful case for bias in NICE Bipolar guidelines @OxPsychiatry https://t.co/TSP368PJIQ
If you have a spare minute today, read this! What NICE does and ‘doesn’t’ tell you…yikes! https://t.co/QuyeBzqV2N
@Mental_Elf @sameerjauhar @Keith_Laws excellent blog – very interesting comments on lack of disclosures
Interesting comments on the problems of porting psychological treatments from unipolar depression to BPD https://t.co/sfI3i0zMiQ
.@Mental_Elf Correct but takes an expert to advise on details so the right studies ARE included https://t.co/Z3sfu6SIpi
@TilWykes And do you agree that those NICE Guideline Development Group experts should disclose their interests? @sameerjauhar @Keith_Laws
Excellent blog highlighting problems with lack of disclosure & bias https://t.co/QjiPWc5fuw
New viewpoint in @TheLancetPsych NICE guidance on psychological treatments for bipolar #OpenAccess for 2 weeks https://t.co/R8aFf6OfIX
.@Mental_Elf Interest is not right word – need to report their expertise and so should meta-analysts https://t.co/lFFR4beEFI
@TilWykes Most Principle Investigators studying psychosocial treatment disclose no interests https://t.co/DVisAspayo https://t.co/svPOzZoObO
.@Mental_Elf Again not distinguishing expertise from “interest”. If financial gain then that’s an interest e.g. DBT https://t.co/dyyJbruk4w
@TilWykes Don’t the workshops & professional training that PIs provide bring such financial gain? That’s Guy Goodwin’s point in the blog
Can we trust the NICE recommendations on CBT for bipolar depression? New @TheLancetPsych paper says NO https://t.co/R8aFf75QAv
.@Mental_Elf True but this is generally peanuts and generally thought to be good as it is dissemination https://t.co/wBCQTfV7jk
@TilWykes Is it peanuts!? (Not for us elves – maybe we’re in the wrong business) I guess that’s why full disclosure is so important :-)
.@Mental_Elf Assure is peanuts often in exchange for free conference reg. Worry indicates no trust in scientists https://t.co/gulsPPevrc
FREE conference registration could be several hundred pounds. Hardly peanuts.
@TilWykes I respect scientists, but I also have a healthy skepticism for methods, especially when there is a history of #bias
Most psychotherapy meta-analyses & RCTs don’t report researcher allegiance. Find out why this matters, a lot https://t.co/R8aFf6OfIX
Is the NICE guideline for bipolar disorder biased in favour of psychosocial interventions? Review by @Mental_Elf https://t.co/yKqAuR8wFe
Is the NICE guideline for bipolar disorder biased in favour of psychosocial interventions? https://t.co/vD0laW7hFC
NICE recommend psychotherapies for bipolar depression. But is this recommendation evidence-based? https://t.co/R8aFf6OfIX
RT @ProfDavidNutt: Is the NICE guideline for bipolar disorder biased in towards psychosocial interventions? https://t.co/kvB5DcokoM via @sh…
Blimey, the author really doesn’t like psychotherapists nor does he seem to know about the potential harms of psychotherapy in general nor the risks to bipolar patients in particular: the sole potential harm he mentions is psychotherapists as sexual predators preying on sexually vulnerable bipolar patients. Psychotherapists can, have been and are sexual predators and sexual predators will go hunting in areas that are likely to have easy pickings, but there’s far more to the harms psychotherapy can do than this. Seems to be both alarmist and inadequate.
I imagine the potential harms of psychotherapy would take up a whole blog of its own; or a series of blogs. There’s certainly not enough space in this one to do it justice.
I agree that it’s a very important subject and vital that we discuss it in this context (i.e. bipolar disorder). As Guy points out in his blog, the NICE guidance seems to view talking treatments as virtually harmless. They gloss over it and give the impression that it’s not an important topic.
We know that any treatment powerful enough to have a positive effect is also powerful enough to cause harm. Unfortunately a large proportion (21-60%) of psychotherapy trials do not measure or report on the harms of treatment: http://www.nationalelfservice.net/treatment/psychotherapy/psychotherapy-trials-should-report-the-side-effects-of-treatment/ This needs to change!
Yes, that does need to change. I’ve personally experienced harm from psychotherapy (and lack thereof) so I have a vested interest in psychotherapy becoming better, and better available where effective.
Perhaps the author deliberately focused on psychotherapists as sexual predators in order to provoke debate. Otherwise it seems unnecessarily inflammatory to single out this one type of harm.
If we have time on Wednesday during our #ElfCampfire, I’ll ask Guy why he singled out this harm in his blog: http://www.nationalelfservice.net/campfire/psychotherapies-for-bipolar-disorder/
A very good piece, so much I agree with but also so many “yes, buts”! It’s complicated! https://t.co/xwBjW3kW9c
Is the NICE guideline for bipolar disorder biased in favour of psychosocial interventions? https://t.co/NZegvqLXrj via @sharethis
From a purely experiential perspective there are a number of massive flaws in logic behind using CBT for bipolar. https://t.co/WINFlE8N11
But that’s the lone talking therapy recommended for me by tertiary services. I feel a blog post coming on… https://t.co/WINFlE8N11
Interesting blog! Looking at both the guidelines and the paper being discussed, mention is also made of Interpersonal Psychotherapy. However, this isn’t discussed in the blog. It is also interesting that no mention is made (in any of the articles) of the very specific, manualized and evidenced based IPSRT (Interpersonal Social Rhythm Therapy) which has (I thought) shown great promise in managing and treating Bipolar Disorder. Care to comment on this?
Mark – we mentioned interpersonal therapy in the paper, but there is not that much to say …certainly no indication of ‘promise’
Although NICE tend to lump multiple psychological interventions, they do analyse ‘Interpersonal Social Rhythm Therapy’ (ISRT) separately and the results are as follows:
NICE/NCCMH conducted 10 meta analyses and 9 had only a single trial (one contained 2 trials) – so, effectively none are meta-analyses in any meaningful sense
Those 10 meta-analyses present data from a total of just 3 trials in over 15 years and only one in nine meta-analyses reveals a significant result (it was a meta of one study)
So, there isn’t much to say about ISRT…except if it had great promise, its not in the data and it hasn’t been followed-up
Thanks Keith! Very clear and informative.
NICE guidance how reliable is it? Has NICE lost credibility? https://t.co/UidhBaQXPV
Have NICE cherry-picked trials to recommend CBT for bipolar depression? https://t.co/R8aFf6OfIX
@mental_elf Congrats Andre !Very smart of you to make an elf out of Pr GuyGoodwin. He must look NICE with your green hat on.Always liked him
Don’t miss: Is the @NICEcomms guideline for #bipolar disorder biased in favour of #psychosocial interventions? https://t.co/R8aFf6OfIX
There’s a great discussion on our blog today https://t.co/R8aFf6OfIX #bipolar #psychotherapy #CBT #NICE #bias Please share your thoughts!
@Mental_Elf really enjoyed reading this & the comments. Thought you made some excellent points on disclosures
Is the NICE guideline for bipolar disorder biased in favour of psychosocial interventions? https://t.co/BrGUMFyeZa
Hi @Sectioned_ Tx for your comment on our psychotherapy for bipolar blog. I’ve responded here: https://t.co/nRwbTtowsv Cheers, André
Clearly written blog High standards to meet Other potential harms from therapy other than sexual though https://t.co/AjXSnfsdHf
NICE has always been biased in the favor of CBT. That’s nothing new. They aren’t really interested in other therapies than CBT or a third generation therapy of CBT. That was their message to a professor who did research about another sort of therapy: ‘we have CBT, that’s enough!. We don’t need new therapies!’
And okay, if it’s really, really necessary, they will admit that some other therapies can do something good too…
And it’s known for years, already, that CBT isn’t an evidence based therapy for bipolar disorder. But NICE disregarded that always.
Most popular blog this week? It’s Prof Guy Goodwin on psychotherapies for bipolar disorder https://t.co/R8aFf6OfIX
RT @121Therapy: Interesting blog! Looking at both the guidelines and the paper being discussed,… https://t.co/uguK3wsbTM
I welcome this paper, but probably not for the same reasons as the authors. The diagnosis of bipolar disorder encompasses a huge spectrum of severity, chronicity, comorbidity, and most importantly, personal stories. There is no one size fits all solution, whether psychological or pharmacological. Some people experience dehabilitating symptoms for their entire life. Others experience transformation towards a fruitful life free of severe difficulties. Most credit medication, psychological therapies, and a variety of other people along the way. We are far from fully understanding how to most effectively and efficiently help people with this diagnosis. I propose that we put far more research into learning from those who have achieved the most successful and long lasting recovery, rather than researching the causes and treatments for a homogenous condition. A fully biopsychosocial framework is required for this. That, in addition to patience, humility, and persistent scientific enquiry. Maybe this article will spur us in this direction.
Do you provide psychotherapy for people with bipolar disorder? Please share your thoughts on our blog https://t.co/R8aFf6OfIX
@Mental_Elf An issue is meta analyses tend to analyse relapse rates for any episode, so unclear if effective for both depression and mania
@DrTomRichardson Yes, amongst their 170 meta-analyses NICE/NCCMH examined relapse rates for ‘depression’, ‘mania’ and ‘all types’ and ‘mixed’ both at end of trial and at follow-up and across a range of psychological interventions – resulting in 30 meta-analyses examining relapse alone! The only indicator that any psychological therapy prevents relapse came for CBT for ‘all types’ – but this was the meta-analysis we described that had inexplicably omitted the largest ever trial by Scott et al (although NICE included it when looking at relapses for both depression and mania).
So, the evidence presented by NICE/NCCMH shows that no psychological therapy (including CBT) prevents mania, depression, any or all types of relapse …at either the end of trial or at follow-up
Thanks @DrTomRichardson Please feel free to expand on this in a blog comment https://t.co/R8aFf6OfIX Cheers, André
RT @Mental_Elf: Have you had psychotherapy for your bipolar disorder? We’d love to hear your experiences on our blog https://t.co/R8aFf6OfI…
RT @Mental_Elf: Is the NICE bipolar guideline biased?
. @matttyrermusic @flatsquid Here Pr Goodwin abt the CBT for bipolar meta-a which shows NICE is the victim to COI https://t.co/p10GYDUB4C
It is intellectually very easy to throw round accusations of bias. We can all do it – it takes little work, and it’s rather uninformative. Indeed, if the standards set down by Goodwin, Laws, Jauhar and McKenna are true, then we are all biased in some way. I’m sure this is true, but does it get us anywhere? For example, Laws and McKenna are well-known for being positively against CBT. Laws has published numerous papers taking issue with it, and has given many talks up and down the country on the matter. Is this declared anywhere? No, and nor should it be, in my opinion. Pecuniary interests are what matters. Eg – if someone is making thousands of pounds of the back of their opinion, then we need to know. But this applies to both non-critics and critics.
RT @Mental_Elf: Join us Wed 2pm
CBT for bipolar depression
#ElfCampfire: https://t.co/z1UsjNSgN8 https://t.co…
I’m always intrigued by one trying to look at #psychotherapy from an evidence based perspective. https://t.co/5pu6KG7WHu
Had psychotherapy for your bipolar disorder? We’d love to hear your experiences https://t.co/R8aFf6OfIX #ElfCampfire https://t.co/8UixMjcx4f
We’ve just finished our Campfire on this blog. You can watch it here: http://www.nationalelfservice.net/campfire/psychotherapies-for-bipolar-disorder/
I was running the Twitter account and my take on how we can improve research and care for people with bipolar disorder was:
1) Listen to expert patients
2) We need to move on from the false dichotomy between psychological and pharmaceutical therapies. Evidence for both should be assessed on an equal footing
3) We need better outcome measures
We’ll be posting more details about the discussion, so check back or follow the chat on Twitter #elfcampfire
Contributors to NICE guidelines declare conflicts of interest at each meeting, which are then reported in the published guidelines.
Where are COI disclosures from Professor Goodwin and other contributors to this discussion? Rather than make ad hominem attacks and post accusations about bias, shouldn’t we share information and let readers draw their own conclusions?
Guy disclosed his interests in the opening paragraph of his blog:
“Disclosures: I am currently completing the third revision of the British Association for Psychopharmacology (BAP) Bipolar treatment guideline. I have advised many pharmaceutical companies doing trials in the bipolar therapeutic area. I was Principle Investigator (PI) for a trial of individual psychoeducation for bipolar disorder (OXTEXT6 – in submission).”
– Guy Goodwin, 5 Feb 2016
I totally agree with your comment that we should “share information and let readers draw their own conclusions”. My concern is that this guideline includes recommendations based on evidence where (as Guy says) “many clinicians … would not come to the same strong conclusions that NICE did”.
I have learnt in my work that frontline practitioners rarely drill-down to the detailed trial information on which NICE recommendations are based (Forrest plots and clear explanations please!), and when they do they struggle to understand them. It’s the responsibility of NICE to summarise the evidence in a transparent and reliable way. Personally I would say that is not always the case and it’s certainly not the case with the bipolar guideline.
Nonetheless, just discussing this particular blog post (and not the Lancet Psych article in question, to which this critique does not apply), your point also applies to people maybe not taking the time to read outside the blog post and build a comprehensive image of an issue. Making a case about how flawed/biased psychotherapy research is is fine, but in the interest of equipoise I would have been nice to see at least one paragraph/ phrase/link about how when it comes to anything from conflict of interests, flaws, sponsorship bias, and even data manipulation, the alternative treatment (pharmacotherapy) is much worse off. I mean light years away. So while it’s fine to be prudish about research allegiance for instance (one of the many documented problems of psychotherapy research), before we get outraged, I think it would be fair to also remember that this is endemic in pharmacotherapy research in the simple formula “who pays, gets the results”. In other words, more corporate than it is individual.
Again, this is just about the blog post and not the content of the article it comments on.
The standard ICMJE disclosure form asks about payment for speaking, shares, board membership, etc. Is Goodwin’s statement that he “advised many pharmaceutical companies” really complete and transparent?
I understand that you and others disagree with the conclusions and recommendations of this guideline, but you’ve not given any evidence that the GDG was biased or that the guideline lacks transparency. The guideline provides a complete and transparent summary of the evidence, and we’ll be happy to answer any questions that you have about it.
There are limitations in the evidence regarding psychological and pharmacological therapies for bipolar disorder, and the guideline states those clearly. It then offers treatment recommendations that consider on the values and preferences and stakeholders and the best current evidence regarding efficacy and cost-effectiveness.
Hibive been on a guideline group, it’s hard work. Please stick to evidence. Both Profs have made sig contributions to care of BPAD
Re COI http://www.ncbi.nlm.nih.gov//pubmed/25456156/?i=9&from=Jauhar%20S%5Bau%5D
I don’t do pharma, and I have significant concerns re this guidelines use of evidence- I look forward to this being addressed
If you look at our Lancet Psychiatry paper, it’s very clear that we all declare no conflicts of interest, but I’m not really sure what you are getting at here or how this addresses any of the questions raised about the guideline
Thank you for offering to answer any questions I have about the guideline.
There was a very important question that was not answered in the #ElfCampfire: Why was largest ever trial of bipolar relapse omitted?
Keith has helpfully shared two PDFs:
1. Appendix 25 from the guideline. That’s the one with all the Psychological and psychosocial interventions forest plots: http://nationalelfservice.net/cms/wp-content/uploads/2016/01/CG185FullGuidelineAppendix25.pdf
2. His slides showing the impact that including the Scott trial has on the overall result: http://nationalelfservice.net/cms/wp-content/uploads/2016/01/Scott.pdf
I look forward to hearing your thoughts on this.
I apologize, I thought we covered this during the call.
We analyzed “relapse” as an outcome “For people who are euthymic at baseline” (protocol on page 251 in the guideline). For information, this is consistent with the reviews of pharmacological interventions in which we distinguished between (i) the treatment of acute episodes and (ii) strategies for long-term management.
Scott 2006 included a mix of participants who were euthymic and participants who were depressed. The published report didn’t distinguish (i) relapse for participants who were euthymic from (ii) effects for participants who were depressed (who could have recovered and experienced new episodes, or they could have continued in their index episodes). We requested disaggregated data from Jan Scott because the GDG wanted to include the subgroup of participants who were euthymic at baseline in the analysis “any relapse” along with other studies in which all participants were euthymic at baseline (page 6 of Appendix 25); she declined to provide the information we requested. The published report also included results for different types of relapse (manic and depressive). Additional analyses in Appendix 25 present the available data about specific types of relapse at different times; the studies assigning people who were euthymic are identified “[M]” and the studies assigning people who were in an acute episode are identified “[A]”.
Readers who want to see the review protocols, statistical code, meta-analyses (including the data as we entered them), study characteristics, risk of bias assessments, GRADE profiles, etc. can find them all on the NCCMH website (http://www.nccmh.org.uk/). In addition to the main report, there are 36 appendices available online.
Please do email me if you have any further questions (firstname.lastname@example.org).
In reply to Evan’s last comment, as I see it there are two problems with the NCCMH relapse meta-analyses. First, the inclusion criteria for the meta-analysis on all relapses were such that they managed to exclude the largest trial to date (168 +168 patients), and one that is widely regarded as definitive. Clearly, in this instance meta-analysis is not doing its job of integrating all relevant evidence properly. Secondly, the meta-analyses of manic relapses and depressive relapses considered separately, which do include Scott 2006, are broadly negative (the sub-analysis for CBT against depressive relapse is borderline significant, but the composite meta-analysis of this and two other therapies is insignificant). Yet, the NICE guideline seems to be in keeping much more with the positive findings of the first (unrepresentative) meta-analysis, while the altogether more gloomy findings of the other two, which give a more complete picture of the evidence, are not mentioned. It all goes back to the point I tried to make in the campfire yesterday – that if you carry out vast numbers of meta-analyses with differing sets of inclusion and exclusion criteria, you end up with findings that are very difficult to interpret in a simple, let alone even-handed way.
This is an astonishingly brazen comment from an author of perhaps the most selective meta-analysis I’ve ever come across: http://www.ncbi.nlm.nih.gov/pubmed/19476688
His meta-analysis also managed to exclude the largest trial of CBT conducted for psychosis
The largest CBT for psychosis trial was done in Germany and hasn’t published its results despite finishing years ago ( burying bad news?) So can’t blame Lynch et al for not including it
I assume you are referring to the Klingberg POSITIVE trial. We have actually written to him twice about the results, but he has not replied. Does anyone out there know anything?
The largest CBT for psychosis trial was done in Germany and hasn’t published its results despite finishing years ago ( burying bad news?)
Sorry for near duplicate posting !
I think that’s a very good point Peter. Would you care to respond to that Evan? Why so many meta-analyses (170), many of them (over half) with just one trial in them?
Another question I have is a follow-up of the data in Keith’s slides yesterday. Why does the NICE guidance advocate psychological therapies for depression, but not mania, when only the latter is significant at follow up?
We will reply in full to the Lancet Psychiatry paper.
Regarding yesterday’s discussion, I think Keith has confused outcomes and inclusion criteria. That is, trials of psychological interventions included people with acute depression and/or people who were euthymic at baseline. Some of those trials measured relapse, including manic relapse (i.e., mania was an outcome, not the condition at baseline). The guideline recommends psychological and pharmacological interventions for people presenting with acute depression and for people in remission (i.e., to prevent relapse), and it recommends pharmacological interventions for mania.
Andre, you need to be a neutral Chair in this. Instead, you appear to have formed a view already, and regrettably seem to find it difficult to reflect critically on the claims of the critics.
This “debate” is becoming more and more unprofessional. As a scientist I like to keep an open mind and generally find the blogs published in Mental Elf to help both inform and challenge my views. I can not ever remember reading an evaluation of any scientific publication in the field of mental health research which reached an unequivocal conclusion. In deed one of my own personal gripes is that authors’ conclude “encouraging results but we need more studies to prove this hypothesis”. As someone who uses NICE mental health guidelines each and every day of my working life, I am acutely aware of their strengths and weaknesses. The scientific studies used to inform these guidelines are carried out under very different conditions than those experienced in the real world of treating real people with mental health disorders. When we add the complexities of delivering psychological treatments to heterogeneous populations of people with broad brush diagnoses such as “bipolar” and “depression”, can anybody really be surprised that it is very difficult to catagorically specifically recommend any treatment protocol? And that is without even mentioning the contentious Dodo effect!
Let’s think about those who are trying to live with mental health disorders and start to put more emphasis on involving them in designing treatment programmes which take the elements of ALL therapies that work, cross train healthcare professionals to offer integrative approaches, tailored to the individual instead of pointlessly trying to prove that a single manualized approach could work for everybody. Finally, stop snipping at each other and try collaborating for the good of the patients – this it’s not about academic egos, it is about alleviating human suffering!
[…] on pharmacological treatments for bipolar disorder. The lead author (Guy Goodwin) wrote a blog post for The Mental Elf earlier this year reviewing an article (Jauhar et al. 2016) describing the […]
[…] earlier blog reviewed a study demonstrating the bias apparent in the handling of the relevant evidence by NICE […]