Using research findings to improve care in mental health should be easy. Notice an understudied area, do a study on it, write a guideline based on your results and watch mental health professionals modify their practice accordingly. Hooray for science.
But in reality, this process is convoluted and murky. Though we’re fairly good at churning out loads of studies performed under tightly-controlled, scientific conditions, we’re less good at boiling all that evidence down into guidelines for use in the real world, and even worse at checking if those guidelines are doing any good. Especially when it comes to severe mental illness.
So how much do we know about how effective guidelines are?
The Cochrane Schizophrenia Group team of Barbui et al recently performed a literature review of studies that aimed to examine:
- how effective guideline implementation strategies were in improving the performance of healthcare providers and patient outcomes and
- which components of those guideline implementation strategies were associated with benefit.
The authors sieved mercilessly through the Cochrane Schizophrenia Group database for relevant studies. Their inclusion criteria were as follows:
- Randomised or quasi-randomised controlled trials
- On adult patients with schizophrenia-spectrum disorders (schizophrenia, schizoaffective disorder, delusional disorder, schizophreniform disorder)
- Comparisons between “active” or “passive” guideline implementation and either no change in care or a different type of guideline implementation.
The team realised that the trials they would find would be measuring a broad range of things, so they chose one primary outcome (practitioner impact) and an additional five secondary outcomes (global state, satisfaction with care, treatment adherence, drug attitude and quality of life) that they thought might be relevant.
The team screened 887 studies, of which only 19 were potentially relevant. Further inspection found that only 5 of the studies met the inclusion criteria.
Two looked into antipsychotic polypharmacy:
- Baandrup (2010) carried out a cluster-randomised controlled trial of a multifaceted intervention aimed at decreasing antipsychotic polypharmacy versus routine care. The participants were people with schizophrenia and related psychotic disorders in outpatient settings. Follow up lasted 12 to 18 months
- Thompson (2008) also conducted a cluster-randomised controlled trial of a multifaceted intervention aimed at reducing antipsychotic polypharmacy. Their participants were inpatients in 19 adult psychiatric units in the south west of England. Follow up lasted 5 to 6 months
- When the results of these two studies were combined, no difference in antipsychotic polypharmacy rates were found between control and experimental groups (n = 1,082, RR 1.10, 95% CI 0.99 to 1.23)
The other three studies all tackled different topics:
- Hamann (2006) conducted a cluster-randomised comparison of a shared decision-making intervention (printed decision aid plus planning talk) versus routine care in a sample of 107 inpatients with schizophrenia. Follow up lasted 12-18 months
- The intervention had no significant effect on psychopathology as measured by the PANSS total score scale (n = 105, MD -1.30, 95% CI -8.21 to 5.61)
- No impact on satisfaction with care as measured by the Patient Satisfaction Questionnaire (n = 83, MD 0.10, 95% CI -1.43 to 1.63)
- No impact on drug attitude as measured by the DAI (n = 57, MD -1.40, 95% CI -2.88 to 0.08).
- Hudson (2008) conducted a cluster-randomised comparison of a multifaceted intervention to promote medication adherence versus basic education in six outpatient psychiatric services. A total sample of 349 participants with schizophrenia were enrolled. Follow up lasted around 6 months.
- Though there was an initial 22.5% increase in adherence rates in the experimental group compared to a 15.1% increase in the control group
- There was no significant difference between the groups at follow up (n = 349, RR 0.87, 95% CI 0.66 to 1.15)
- Osborn (2010) conducted a cluster-randomised comparison of a nurse-led intervention to improve screening for cardiovascular risk factors in people with severe mental illness. Six community mental health teams were randomised. A total of 121 people participated in outcome interviews. Follow up lasted around 6 months
- When corrected for the cluster design, only blood pressure and cholesterol screening rates were found to be significantly improved (blood pressure n = 33, RR 0.10, 95% CI 0.01 to 0.74; cholesterol n =35, RR 0.49, 95% CI 0.24 to 0.99)
- But not glucose, BMI, smoking status or Framingham score (glucose n = 35, RR 0.58, 95% CI 0.28 to 1.21; BMI n = 34, RR 0.18, 95% CI 0.02 to 1.37; smoking status n = 32, RR 0.25, 95% CI 0.06 to 1.03; Framingham score n = 38, RR 0.71, 95% CI 0.48 to 1.03).
The risk of bias across several domains was assessed by the team. A low risk of selective reporting was found, and an unclear risk of bias in blinding. The other domains fell between low and unclear risk.
There appears to be a yawning chasm between the pervasive reverence of guidelines in schizophrenia and the wafer-thin evidence that actually establishes them as effective in improving care.
This is interesting.
It’s easy to assume that applying guidelines, which endorse treatments shown to be efficacious by research, would be of benefit to patients. But the translation of interventions from the sterile, tightly-bound world of science to the busy, downtrodden local clinic – via the agenda-laden land of guideline production – is not seamless.
- The patients are different – there are no exclusion criteria in real life.
- The staff are different – they aren’t singled minded research experts.
- The interventions are different – squeezed into care, jostling with endless other tasks.
So are guidelines still worth following if they’ve not been backed up by evidence of benefit in real-world implementation?
You tell me.
Barbui C, Girlanda F, Ay E, Cipriani A, Becker T, Koesters M. Implementation of treatment guidelines for specialist mental health care. Cochrane Database of Systematic Reviews 2014, Issue 1. Art. No.: CD009780. DOI: 10.1002/14651858.CD009780.pub2.
Baandrup L, Allerup P, Lublin H, Nordentoft M, Peacock L, Glenthoj B. Evaluation of a multifaceted intervention to limit excessive antipsychotic co-prescribing in schizophrenia out-patients. Acta Psychiatrica Scandinavica 2010;122:367–74. [PubMed abstract]
Thompson A, Sullivan SA, Barley M, Strange SO, Moore L, Rogers P, et al. The DEBIT trial: an intervention to reduce antipsychotic polypharmacy prescribing in adult psychiatry wards—a cluster randomized controlled trial. Psychological Medicine 2008;38:705–15. [PubMed abstract]
Hamann J, Langer B, Winkler V, Busch R, Cohen R, Leucht S, et al. Shared decision making for in-patients with schizophrenia. Acta Psychiatrica Scandinavica 2006;114:265-73. [PubMed abstract]
Hudson TJ, Owen RR, Thrush CR, Armitage TL, Thapa P. Guideline implementation and patient-tailoring strategies to improve medication adherence for schizophrenia. Journal of Clinical Psychiatry 2008;69:74-80. [PubMed abstract]
Osborn DPJ, Nazareth I, Wright CA, King MB. Impact of a nurse-led intervention to improve screening for cardiovascular risk factors in people with severe mental illnesses. Phase-two cluster randomised feasibility trial of community mental health teams. BMC Health Services Research 2010;10(61):1-13. [PubMed abstract]
‘@Mental_Elf: What impact do guidelines actually have on patient outcomes for people with schizophrenia?’ http://t.co/FuVESZNR6J <– by me.
@psychiatrysho @mental_elf good points but Evidence based practice can act as a disciplinary micro-facism excluding the ‘ineffable’ (1/2)
@mental_elf @psychiatrysho (2/2) the problem precedes the formulation into guidance. Although as you say this is problematic.
@anarcho_ted @PsychiatrySHO @Mental_Elf I really like the concept of disciplinary-based micro fascism!
@suzypuss @mental_elf @psychiatrysho I’m adapting Foucault/Deleuze/Guattari. Dave Holmes has written some interesting papers on this.
What impact do guidelines actually have on patient outcomes for people with schizophrenia?: Using research fin… http://t.co/8qOQyh2jnz
Mental Elf: What impact do guidelines actually have on patient outcomes for people with schizophrenia? http://t.co/2Cbu06FOmA
@CochraneSzGroup We’ve blogged about your recent review today http://t.co/c1JeRk9GW1 As always, let us know your thoughts.
@PsychiatrySHO flags up recent SR on the implementation of treatment guidelines for specialist mental health care http://t.co/c1JeRk9GW1
New from @Mental_Elf What impact do #guidelines actually have on patient outcomes for people with #schizophrenia? http://t.co/VPbActlrcW
What impact do guidelines actually have on patient outcomes for people with schizophrenia? – The… http://t.co/4Lzm4gZR6l
Guidelines for healthcare professionals did not reduce antipsychotic co-prescribing in schizophrenia outpatients http://t.co/c1JeRk9GW1
Useful blog on whether clinical guidelines make any difference by @PsychiatrySHO for @Mental_Elf http://t.co/4RGy7Q9ze1
Considerable doubt remains about how best to implement guidelines in mental health practice. Read our blog http://t.co/c1JeRk9GW1
@Mental_Elf Shorthand for this: the system just doesn’t work. Stop paying for it.
Do schizophrenia guidelines improve outcomes & reduce antipsychotic misuse? http://t.co/RfOaEGz4eA @Mental_Elf
RT @Mental_Elf: Considerable doubt remains about how best to implement guidelines in mental health practice http://t.co/LbymChEXJb
My latest @Mental_Elf blog, on the evidence for guidelines in schizophrenia. Have your say, make a comment. http://t.co/kExRYSrnIT
@PsychiatrySHO @Mental_Elf Alex, Come and be a NICE scholar for a year and help us position our guidance so that it has effects in practice
@valmooreatpb @Mental_Elf i’ve checked out the website and they look like great posts. highly competitive too, i’d bet.
@PsychiatrySHO @Mental_Elf . getting the application in is half the job done. We would love to see you apply Alex. KIT
@valmooreatpb @Mental_Elf for what it’s worth i dont think the guidance is half bad, i just think audit cycling it more would be useful.
@PsychiatrySHO @Mental_Elf Completely agree and it’s what we are working on, to get more info back from local audit
In case you missed it: What impact do guidelines actually have on patient outcomes for people with schizophrenia? http://t.co/c1JeRk9GW1
How to change practice? @Mental_Elf: Guidelines for HCP did not reduce antipsychotic polypharm in SCZ outpts http://t.co/T8dEqXJ4Ru #hqo
What impact do guidelines actually have on patient outcomes for people with #schizophrenia? http://t.co/m2oWB3HNS3
Most popular blog this week? It’s @PsychiatrySHO on schizophrenia guidelines – do they make a difference? http://t.co/U4LEoMlcIP
What impact do guidelines actually have on patient outcomes for people with schizophrenia? – The Mental Elf http://t.co/LLLdba4zMc
I think one thing that hasn’t been taken into consideration are the consequences of no guidelines at all. A lot of doctors may not find them applicable to their day-to-day jobs but some will, especially those just starting a career in medicine. I have met practitioners from areas of medicine other than schizophrenia that have raved about the use of reviews in their day-to-day decision making. As with anything you read, there will be some bits you find useful and others that you won’t. I don’t think that means that guidelines are ineffective though. In terms of your comment about guidelines being ‘backed up by evidence of benefit in real-world implementation’, yes of course they should be, but as you also said ‘The patients are different’ and ‘The staff are different’ therefore applicability to the real world will always depend on the people using the guidelines no matter the evidence it is based upon. By the way, I work for the Cochrane Schizophrenia Group but these are my own opinions.
What impact do guidelines actually have on patient outcomes for ppl with schizophrenia? http://t.co/U4LEoM33uH #SchizophreniaAwarenessWeek