There is concern that sponsorship bias by pharmaceutical companies may influence the results of research and overstate the effectiveness of psychiatric medication. It’s possible that this kind of financial conflict of interest may also bias the results of research that compares the effectiveness of drug treatments and psychotherapies.

It’s fair enough that people are concerned, after all you probably wouldn’t trust a comparison between a Golf and an Astra if it had been sponsored by Volkswagen.

A new meta-analysis published yesterday in the British Journal of Psychiatry (Cristea et al, 2016) answers some important questions:

1. Does pharma sponsorship (and/or financial conflict of interest of the authors of a study) bias effectiveness comparisons of antidepressants and psychotherapy in treating depression?
2. If so, how big is this effect?
3. And does this effect change the interpretation of the results?

Methods

The authors used a very thorough process to search for and identify relevant medication v psychotherapy comparative trials in the treatment of depression. They used an explicit definition of financial conflict of interest (COI) and pharmaceutical sponsorship. They performed a very thorough double-check using information from journals with mandatory COI declaration to see if authors had declared any potential COI in papers in other journals. The definition of financial conflict of interest is broad:

receipt of financial support or benefits of any type from the industry (employees, direct compensation, revenues from the industry, speaker fees, consultancy, serving on boards, etc)

This is a good thing as an overly restrictive definition may lead to an underestimate of the effect of pharmaceutical sponsorship. However, there was no measurement of financial COI for psychotherapy such as authors writing books or running paid-for workshops describing their models of psychotherapy.

There was measurement of risk of bias in the studies using four criteria from the Cochrane Collaboration. Appropriate statistical techniques were used to check for missing studies, heterogeneity of results (how much the results in different studies varied from each other) and to compare the effectiveness of antidepressants versus psychotherapy.

A reasonable set of exclusion criteria were used (e.g. unable to calculate useable statistics for the analysis) except for one: no studies with inpatients were included. It’s not clear why they were excluded; this type of study would tend to favour antidepressants as people with depression so severe or risky they need admission to hospital are felt by clinicians to struggle to engage with therapy without medication improving mood sufficiently. A reasonable series of sensitivity analyses were performed, such as only looking at recent studies, or studies with low risk of bias, or studies using CBT as the comparison therapy.

• 45 studies comparing medications with psychotherapy were included
  • CBT (24 studies)
  • Interpersonal Therapy (12 studies)
  • SSRI antidepressants (20 studies)
  • Tricyclic antidepressants (12 studies)

Results

• The overall effect size for the comparison between psychotherapy and pharmacotherapy was non-significant, g=-0.01 (95% CI -0.12 to 0.09)
• Studies funded by the pharmaceutical industry had a tiny advantage for antidepressants compared to psychotherapy with effect size g= -0.11 (95% CI -0.21 to -0.02). An effect size of 0.2 is regarded as “small” and this advantage is even smaller than this in magnitude
• Studies not funded by the pharmaceutical industry showed a non-significant tiny advantage for psychotherapy g=0.10 (95% CI -0.09 to 0.29)
• The difference between these two results was statistically significant (p=0.05) although the absolute difference was small
• There was little evidence of publication bias.

Financial conflict of interest

• Studies where at least one author had a financial COI showed a tiny advantage for pharmacotherapy that was non-significant compared to psychotherapy, g=-0.13 (95% CI -0.27 to 0.003)
• For studies where no financial COI was identified, there was a tiny non-significant advantage for psychotherapy, g=0.05 (95% CI -0.08 to 0.19)
• This difference was not statistically significant (p=0.057) and approximately small in magnitude.
• The thorough process to cross-check for financial COI identified 5 studies where financial COI were not declared by the authors.

The different sensitivity analysis generated one interesting result and three statistically significant results. The interesting result was that studies at lowest risk of bias were still more likely to show a non-statistically significant advantage for pharmacotherapy if funded by industry. This could have two explanations: that industry funded studies are more biased towards medication despite high-quality research methodology, or that the apparent bias caused by industry funding is a chance finding as the high-quality research methodology protects against bias (or both).

The three statistically significant findings of increased benefit for medication in industry funded studies were all small in size and included studies where the only funding was free medication, studies aimed at adults and studies using SSRI antidepressants.

Strengths and limitations

Strengths

• The authors used an explicit and thorough process to identify and select studies for inclusion, as well as appropriate statistical methods
• The study benefits from a very broad definition of financial conflict of interest and thorough checks to identify studies where the authors had financial COI
• An appropriate range of sensitivity analyses were performed.

Limitations

• Inpatient studies were excluded and the reason for this was not clear
• There was no measurement of financial conflict of interest for psychotherapy trials.

Implications

This study was informative on the three questions described in my blog introduction and an implied point of clinical effectiveness. The thoroughness of the methodology allowed three reasonable conclusions to be made based on the data:

1. Firstly, that industry sponsorship and/or financial COI of the authors (broadly defined) means that medication is more likely to be slightly more effective than psychotherapy studies measuring effectiveness in outpatients with depression
2. Secondly, the size of this effect is small
3. Thirdly, that any differences between psychotherapy and medication effectiveness in treating outpatients with depression are likely to be negligible (or below small). In other words, the interpretation of the findings remains unchanged.

Based on this meta-analysis, an important point about the treatment of outpatients with depression can be made. If you accept that evidence-based psychotherapies are effective in the treatment of outpatients with depression, then you also accept that medications used for depression such as antidepressants are also effective. Those who claim that antidepressants are a placebo either do not understand what a placebo effect is or are unaware that many medications used in general medicine are regarded as useful and effective despite having only a small advantage over placebo.

Two further questions spring to mind:

• What about inpatients with depression
• What is the effect of financial conflict of interest on the effectiveness of psychotherapy in trials?

This paper is praiseworthy as even though it only showed a little difference (the effects of industry sponsorship and/or author’s financial COI), it was useful to know that this little difference was not enough to cast doubt on medication being effective in outpatient depression.

Links

Primary paper

Cristea IA, Gentili C, Pietrini P, Cuijpers P. (2016) Sponsorship bias in the comparative efficacy of psychotherapy and pharmacotherapy for adult depression: meta-analysis. BJP bjp.bp.115.179275; published ahead of print November 3, 2016, doi:10.1192/bjp.bp.115.179275 [Abstract]

Photo credits

• Petras Gagilas CC BY 2.0
• Always Shooting CC BY 2.0
• Amy CC BY 2.0
• barbara w CC BY 2.0
• Charm 2010 CC BY 2.0