Last year (2018) was the 65th anniversary of psychologist Hans Jürgen Eysenck’s paper which stated that psychotherapy was an ineffective treatment for mental disorders (Eysenck, 1952). The paper sent shockwaves through the psychology community, and it took the creation of the meta-analysis method to dispute his study.
Gene Glass, the creator of the meta-analysis, and Mary Lee Smith published a paper in 1977 (Smith & Glass, 1977) that disputed Eysenck’s belief. They found that patients who received psychotherapy were better off than 75% of control group patients, seemingly putting the matter of psychotherapy treatment to bed, 25 years later.
However, a recent study has found that the apparent clear-cut benefits of psychotherapy might not be as straightforward as Glass and Smith initially believed (Cuijpers et al, 2018).
With a focus on adult depression, the study examined how common biases affect the effect size of various psychotherapies.
The authors used a comprehensive database of randomised controlled trials (RCTs) (described in Cuijpers et al, 2008), which included all RCTs in which at least one arm was a psychological treatment for adults with depression according to a diagnostic interview or an elevated level of depressive symptomatology (as indicated by a score above a cut-off score on a validated self-report depression scale).
The authors only used trials that compared a psychotherapy for adult depression with a control group (waiting list, care-as-usual, placebo or other).
They conducted multivariate meta-regression analyses and calculated effect sizes using Hedges’ g.
The psychotherapies examined were:
- Cognitive behavioural therapy (CBT)
- Behavioural activation
- Interpersonal psychotherapy
- Problem-solving therapy
- Third wave therapies
- Supportive counselling
- Psychodynamic therapy
The biases examined and controlled for were:
- Exclusion of waiting list control groups
- Risk of bias (systematic errors)
- Publication bias
Unadjusted effects of psychotherapies for adult depression
The pooled effect size for the psychotherapies examined, when compared to a control group, was g=0.70. When interpreting effect size, this is considered a medium-large effect. This was in line with previous trials and meta-analyses of psychotherapies. There was also no significant difference in effectiveness between each different type of psychotherapy when used to treat adult depression.
Adjusted effects of psychotherapies for adult depression
After limiting the included studies to exclude waiting list control groups, studies with a high risk of bias, and after adjusting for publication bias, the mean effect size dropped to g=0.31. This is considered a low-medium effect. Only 22% of all studies included were considered to have a low risk for bias.
Of the seven psychotherapies included:
- Only CBT and supportive counselling retained a small effect size
- Psychodynamic therapy became no longer significant
- Problem-solving therapy, behavioural activation, third wave therapies, and interpersonal therapy had insufficient evidence to draw a conclusion.
The authors concluded that:
- Previous meta-analyses have considerably overestimated the effects of psychotherapy on adult depression
- However, overall the effect of psychotherapy was small, but significant
- Further research is needed to examine the effects of other biases.
Strengths and limitations
- The study looked at a wide range of psychotherapy options. Previous similar studies have only focused on specific therapies, so this paper stands out due to its broad examination.
- The experimenters were able to include a large number of studies in their final analyses (369).
The study could have been expanded to include a number of other biases that are likely to have a significant effect on effect size. The authors acknowledge that selective outcome reporting, experimenter bias, and unconscious bias are all likely to have a significant impact on effect size, but do not control for them.
Due to the variations in research, the number of papers available for each psychotherapy option varied greatly. CBT was examined in 192 papers, whereas psychodynamic therapy only in 11 papers.
Implications for practice
Overall, it is not expected that this paper will have large implications for current practice and research. Whilst the result was slightly surprising, in the sense that the effects of psychotherapy appear greatly overestimated, it does not suggest ceasing their use.
However, this study may encourage some mental health practitioners to examine multiple treatment types for patients with adult depression. Psychotherapy is significantly successful, but its effect size was overall small, so this might encourage practitioners to seek different, and more effective treatments.
As the biases observed had a significant effect on the effect size, this paper may also encourage future experimenters to more closely examine their own biases when undertaking research into psychotherapy.
The new information provided by this study, specifically the large potential effects of bias on effect size, could stimulate similar research into other treatment methods.
Conflicts of interest
King’s MSc in Mental Health Studies
This blog has been written by a student on the Mental Health Studies MSc at King’s College London. A full list of blogs by King’s MSc students from can be found here, and you can follow the Mental Health Studies MSc team on Twitter.
We regularly publish blogs written by individual students or groups of students studying at universities that subscribe to the National Elf Service. Contact us if you’d like to find out more about how this could work for your university.
Cuijpers, P., Karyotaki, E., Reijnders, M., & Ebert, D. D. (2018). Was Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression. Epidemiology and Psychiatric Sciences, 1-10. doi: 10.1017/S2045796018000057
Eysenck, H. J. (1952). The effects of psychotherapy: an evaluation. Journal of Consulting Psychology, 16, 319-324.
Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760
Cuijpers P, van Straten A, Warmerdam L, Andersson G (2008). Psychological treatment of depression: a meta-analytic database of randomized studies. BMC Psychiatry 8, 36.