Last year (2018) was the 65th anniversary of psychologist Hans Jürgen Eysenck’s paper which stated that psychotherapy was an ineffective treatment for mental disorders (Eysenck, 1952). The paper sent shockwaves through the psychology community, and it took the creation of the meta-analysis method to dispute his study.

Gene Glass, the creator of the meta-analysis, and Mary Lee Smith published a paper in 1977 (Smith & Glass, 1977) that disputed Eysenck’s belief. They found that patients who received psychotherapy were better off than 75% of control group patients, seemingly putting the matter of psychotherapy treatment to bed, 25 years later.

However, a recent study has found that the apparent clear-cut benefits of psychotherapy might not be as straightforward as Glass and Smith initially believed (Cuijpers et al, 2018).

Methods

With a focus on adult depression, the study examined how common biases affect the effect size of various psychotherapies.

The authors used a comprehensive database of randomised controlled trials (RCTs) (described in Cuijpers et al, 2008), which included all RCTs in which at least one arm was a psychological treatment for adults with depression according to a diagnostic interview or an elevated level of depressive symptomatology (as indicated by a score above a cut-off score on a validated self-report depression scale).

The authors only used trials that compared a psychotherapy for adult depression with a control group (waiting list, care-as-usual, placebo or other).

They conducted multivariate meta-regression analyses and calculated effect sizes using Hedges’ g.

The psychotherapies examined were:

• Cognitive behavioural therapy (CBT)
• Behavioural activation
• Interpersonal psychotherapy
• Problem-solving therapy
• Third wave therapies
• Supportive counselling
• Psychodynamic therapy

The biases examined and controlled for were:

• Heterogeneity
• Exclusion of waiting list control groups
• Risk of bias (systematic errors)
• Publication bias

Results

Unadjusted effects of psychotherapies for adult depression

The pooled effect size for the psychotherapies examined, when compared to a control group, was g=0.70. When interpreting effect size, this is considered a medium-large effect. This was in line with previous trials and meta-analyses of psychotherapies. There was also no significant difference in effectiveness between each different type of psychotherapy when used to treat adult depression.

Adjusted effects of psychotherapies for adult depression

After limiting the included studies to exclude waiting list control groups, studies with a high risk of bias, and after adjusting for publication bias, the mean effect size dropped to g=0.31. This is considered a low-medium effect. Only 22% of all studies included were considered to have a low risk for bias.

Of the seven psychotherapies included:

• Only CBT and supportive counselling retained a small effect size
• Psychodynamic therapy became no longer significant
• Problem-solving therapy, behavioural activation, third wave therapies, and interpersonal therapy had insufficient evidence to draw a conclusion.

Conclusions

The authors concluded that:

• Previous meta-analyses have considerably overestimated the effects of psychotherapy on adult depression
• However, overall the effect of psychotherapy was small, but significant
• Further research is needed to examine the effects of other biases.

Strengths and limitations

Strengths

• The study looked at a wide range of psychotherapy options. Previous similar studies have only focused on specific therapies, so this paper stands out due to its broad examination.
• The experimenters were able to include a large number of studies in their final analyses (369).

Limitations

The study could have been expanded to include a number of other biases that are likely to have a significant effect on effect size. The authors acknowledge that selective outcome reporting, experimenter bias, and unconscious bias are all likely to have a significant impact on effect size, but do not control for them.