If a treatment is powerful enough to have a good effect, then it’s powerful enough to have a bad effect. This is well recognised when it comes to medication, with strict regulations in place to ensure adverse outcomes are monitored and measured.

By contrast, psychotherapy has never been as readily associated with the potential to cause harm. Whatever the basis of this assumption, it doesn’t always quite ring true. We already know that several forms of therapy can be potentially hazardous, including critical incident debriefing, ‘boot camps’ and grief counselling for normal bereavement (Lilienfeld 2007, Barlow 2010, Nutt et al 2009, Berk et al 2009). The team working on the huge STAR*D trial noted that even CBT appeared to increase suicidal ideation in some patients (Sinyor et al 2010), and 15 of the 198 patients with psychosis in Klingberg et al’s 2012 trial of CBT versus cognitive remediation for negative symptoms suffered severe adverse events (Klingberg et al 2012).

So with psychotherapy being capable of causing unwanted effects, just like any other potentially effective treatment, it would seem important that such occurrences are monitored, measured and crucially, carefully reported in trials. In a manner similar to how we discuss side effects of drugs, patients are entitled to information about potential side-effects of psychotherapy thus allowing them to make balanced and informed decisions about their care.

The CONSORT guidelines make this process straightforward and widespread for drug trials, but the rate at which adverse events are being monitored in randomised controlled trials of psychotherapy was unclear, that is until two papers published this year (Vaughan et al 2014, Jonsson et al 2014). 

Methods

Vaughan et al performed a Medline search of high impact (impact factor >5) psychiatry and psychology journals for randomised controlled trials of Axis I disorders (all mental health diagnoses except personality disorder and learning disability). They only included phase II, III or IV drug trials and psychotherapy trials of “commonly used” modalities like CBT or supportive therapy. They then chose 15 trials at random from each of the following groups:

• Medication trials
• Psychotherapy trials
• Combined medication and psychotherapy trials

Two of the authors rated each of the trials for mentioning possible or actual adverse events of treatments in the following categories:

• Medication in medication trials (M)
• Psychotherapy in psychotherapy trials (T)
• Medication in combined trials (CM)
• Psychotherapy in combined trials (CT)

Each section of each paper (introduction, methods, results, discussion) was rated, as well as each paper as a whole. When it wasn’t clear if the harm being discussed in a combined trial was in reference to the drug or the psychotherapy, the consideration was ascribed to both groups.

Jonsson et al looked at a much larger sample of studies, but they chose not to include any control group of medication trials. They searched Pubmed for trials of any psychological intervention for any mental or behavioural disorder according to ICD-10 that had been published in the year 2010. One author searched each study for several keywords associated with adverse events using the “find” tool, and then manually as well. A second author then screened the results sections of the papers for a consensus.

Results

Vaughan et al found significant differences between their groups. Whereas 100% of medication trials (M) demonstrated an awareness of possible or actual adverse events, only 60% of psychotherapy trials (T) did so.

In combined trials of medications and psychotherapy, adverse events were discussed in relation to medication 86.7% of the time, but in relation to psychotherapy only 60% of the time (p=0.018). Adverse events were discussed in 100% of the methods and result sections of medication trials, but only in 46.7% of methods (p=0.001) and 40% of results (p=0.020) sections of psychotherapy trials.

When medication (M) and medication in combined trials (CM) were added together, 93% of trials discussed adverse effects, whereas when psychotherapy (T) and psychotherapy in combined trials (CT) were added together, that number was only 60% (OR 9.33, CI 1.87 to 46.66, p=0.007).

Jonsson et al’s search strategy turned up 3,696 studies, all but 132 of which were excluded. Anxiety (23%) and mood (13%) disorders were the most commonly studied problems and CBT (52%) was the most common intervention. Europe and North America both hosted 41% of the trials and 73% of the trials were conducted on working age adults.

Just 28 (21%) trials indicated that harms, adverse events or side effects were monitored.

Four trials included full reports of adverse events and how these were monitored:

• One trial of CBT for childhood PTSD used a checklist
• One trial of behavioural therapy for children with Tourette disorder used a series of structured questions
• One trial systematically assessed PTSD symptoms at the end of every session of exposure therapy
• One 2-year trial of a family intervention in schizophrenia had monthly assessments for relapse by a psychiatrist

Another 24 trials showed some consideration of the potential for adverse events:

• Five trials gave some information about adverse events in their results sections but had missing or incomplete details
• Four trials reported that no adverse events occurred, but gave no more information
• Fifteen trials didn’t report adverse events as such, but did measure deterioration using a variety of tools. Only three of these mentioned how many patients actually deteriorated.

The other 79% of trials gave no indication of having considered adverse events at all, one even stating in the methods section that the treatment was “not deemed harmful”.

Half of the trials of interventions for PTSD considered adverse events, far more than trials on interventions for any other diagnosis (25% at most).

Summary of results

These two important studies suggest that trials of psychotherapeutic interventions report potential and actual harms infrequently. Vaughan et al reported that harms were considered in only 60% of psychotherapy interventions in their sample compared to 100% of medication interventions, and Jonsson et al found that only 21% of psychotherapy trials published in 2010 mentioned harms.

Strengths and weaknesses

Between them, the two trials boast various strengths. Jonsson et al were comprehensive in studying every trial from a single year, lending reliability to their results. By looking at each trial in detail, they discovered that PTSD trials were more likely to consider harms, which is a salient point.

Vaughan et al, in contrast, made an illuminating comparison with medication. By finding that harms were reported significantly less in every section of psychotherapy papers, including the introduction, they suggest that not considering harms is a cultural problem, not just an immediately methodological one. The study of PTSD, as noted above, may be an except to this, because researchers are already primed for the idea that their participants may come to harm – as they already have.

Both papers had flaws though – Jonsson et al lacked a control group, whereas Vaughan et al chose to study a small selection of papers, making generalising their results tricky. Also, by selecting papers from only high impact journals, which have more stringent selection criteria, Vaughan et al may have overestimated the rates of reporting on harms – a possibility that is backed up by the more inclusive Jonsson et al study finding a lower reporting rate. Neither could, of course, account for harms being considered by authors but not being explicitly mentioned in final manuscripts.

Discussion

However you look at it, the results are hard to ignore – psychotherapy trials need to monitor, measure and report on adverse events more frequently. How can this be achieved?

Firstly, there needs to be far better recognition of the problem. Accepting the potential for psychotherapy to cause adverse events, as all effective treatments can, needs to become common parlance. Perhaps, as Vaughan et al suggest, this current situation is due to us:

thinking of the process of therapy in nomothetic or generalized terms rather than idiographic or individualized terms, enabling clinicians to de-emphasize specific cases and outcomes.

Secondly, researchers need tools that help them to differentiate between the various causes of deterioration during psychotherapy, so they can actually spot harms and report them appropriately. The lack of such tools has undoubtedly been a hindrance to reporting adverse events in the past. It’s not as straightforward as identifying drug side effects, where consequences such as rashes and vomiting are obviously attributable to the treatment in question – psychotherapy is a complex, multi-faceted process, the effects of which can prove nearly impossible to disentangle from life events themselves. Michael Linden can help here; he presents definitions of unwanted events, treatment-emergent reaction, adverse treatment reaction, malpractice reaction, treatment non-response, deterioration of illness, therapeutic risk, and contraindications, which could make discerning the difficult types of adverse events easier (Linden 2013). The development of CONSORT-SPI, a psychotherapy equivalent of the structure medication trials adhere to, is ongoing and will surely also prove useful.

At the very least, authors should report if no harm-related data were collected, and both ethics and guideline committees need to be alert to this responsibility.

Monitoring for harms, and therefore making it clear to patients that they may occur, may have an interesting effect on the efficacy of psychotherapy in trials. No longer in the dark about potential harms, some patients may get worse simply by being aware that they might. This nocebo effect is well-known in drug trials, and although it is in constant competition with the beneficial placebo effect, it can only decrease the effectiveness of an intervention. But we must be honest with our patients.

Also of note, reporting on harms might be a free shot at killing the Dodo effect. If some therapies turn out to be more harmful than others, it could seriously influence choice of treatment – just like the weight of side effects with some drugs make them less suitable choices than equally effective alternatives. Drop out rates – often seen as lower than that in drug trials – still need to be factored into these calculations.

Conclusion

In conclusion, to say that psychotherapy trials do not report on harms frequently enough is not to say that psychotherapy is ineffective, unscientific, or necessarily even harmful. It is to say that there are valuable improvements to be made in psychotherapy trial designs. These improvements will result in psychotherapists being able to offer their patients more information about the interventions they propose, to better recognise how and when and which of those interventions are best used, and ultimately to offer more effective, comprehensive and ethically honest care.

Links

Vaughan B et al.  Frequency of reporting of adverse events in randomized controlled trials of psychotherapy vs. psychopharmacotherapy. Compr Psychiatry. 2014 Jan 23. pii: S0010-440X(14)00006-6. doi: 10.1016/j.comppsych.2014.01.001. [Epub ahead of print] [PubMed abstract]

Jonsson U, Alaie I, Parling T, Arnberg FK. Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioral disorders: A review of current practice. Contemp Clin Trials. 2014 Mar 4;38(1):1-8. doi: 10.1016/j.cct.2014.02.005. [Epub ahead of print] [PubMed abstract] [Accepted manuscript PDF]

Lilienfeld SO. Psychological treatments that cause harm (PDF). Perspect Psychol Sci 2007;2:53–70.

Barlow DH. Negative effects from psychological treatments: a perspective (PDF). Am Psychol 2010;65:13–20. [PubMed abstract]

Nutt DJ, Sharpe M. Uncritical positive regard? Issues in the efficacy and safety of psychotherapy (Researchgate PDF, requires registration). J Psychopharmacol 2008;22:3–6. [PubMed abstract]

Berk M, Parker G. The elephant on the couch: side-effects of psychotherapy. Aust N Z J Psychiatry 2009;43:787–94. [PubMed abstract]

Sinyor M, Schaffer A, Levitt A. The Sequenced Treatment Alternatives to Relieve Depression (STAR*D) Trial: A Review. The Canadian Journal of Psychiatry 2010, 55 (3): 126–135. [PubMed abstract]

Klingberg S, Herrlich J, Wiedemann G, Wölwer W, Meisner C, Engel C, Jakobi-Malterre UE, Buchkremer G, Wittorf A. Adverse effects of cognitive behavioral therapy and cognitive remediation in schizophrenia: results of the treatment of negative symptoms study. J Nerv Ment Dis. 2012 Jul;200(7):569-76. doi: 10.1097/NMD.0b013e31825bfa1d. [PubMed abstract]

Linden M. How to define, find and classify side effects in psychotherapy: from unwanted events to adverse treatment reactions. Clin Psychol Psychother. 2013 Jul-Aug;20(4):286-96. doi: 10.1002/cpp.1765. Epub 2012 Jan 18. [PubMed abstract]