Only two things are clear when it comes to antidepressants: they’re big business, and big business makes everything else less clear.
With an annual market of nearly $10 billion in the USA alone, pharmaceutical companies want to make antidepressants look like they’re worth prescribing. Unsurprisingly, studies published by these companies are more likely to find their own drugs effective than independent research (Lexchin, 2003), and negative antidepressant trials tend to get buried (Eyding, 2010).
Meta-analyses are relatively cheap and easy to do. You don’t need to collect any new data, just statistically combine lots of pre-existing trial data. We already know that meta-analyses of general medical drugs written with pharmaceutical company influence tend to be of worse quality and to report more positive results (Jørgensen, 2008). But antidepressant meta-analyses have never been studied in this way, until now.
Ebrahim et al (2015) have looked into whether industry sponsored meta-analyses mention fewer negative conclusions in their abstracts than non-industry sponsored studies.
The research team searched MEDLINE for all meta-analyses of randomised controlled trials evaluating antidepressants in people with depressive conditions published since 2007. They looked for trials not just of all recognised antidepressants but also of drugs like quetiapine, which aren’t primarily antidepressants, but are approved for the purpose.
They accepted meta-analyses that compared antidepressants to placebo or other antidepressants, but not those that compared them to a non-drug treatment, like CBT. The main focus of the meta-analyses could be benefit, harm, treatment discontinuation or any combination of these, but not ‘predictors of outcomes’, whatever that means.
They identified whether:
- The study had been sponsored (paid for) by a company who were producing one of the drugs being evaluated
- Any of the authors had a conflict of interest, like working for a pharmaceutical company who made one of the drugs being studied or receiving money from them for another project
If they found a paper with no conflicts of interest declared by the authors, they checked three other recent papers by the same lead author to make sure they weren’t declaring anything elsewhere.
Then two of the team were blinded to which meta-analyses had biased authors, before judging whether the abstracts included any negative comments (‘drug X is not effective’). If there were no negative comments, they looked for positive statements. If positive statements were present, comments about the evidence being of low quality or inconclusive were not counted as negative. If there were no positive comments, those statements were taken as negative.
The initial search found 1,111 papers. This was reduced to 259 by looking at abstracts, then another 74 were excluded by looking at full papers. This left 185 meta-analyses. They tended to:
- Be from the US (35%) and Europe (33%)
- Assess benefit as their primary focus (92%)
- Evaluate patients with major depressive disorder only (82%)
- Appear in specialist psychiatric journals (56%)
- Study SSRIs (45%) and SNRIs (30%)
- Compare antidepressants to placebo (74%)
Who paid for them?
- 46 (25%) by a company producing the drug being evaluated
- 20 (11%) by a not-for-profit organisation
- 33 (18%) by a government organisation
- 41 (22%) didn’t have funding
- 51 (28%) didn’t mention funding, but 11 (22%) of these had authors who were industry employees
How many authors were biased?
- 57 (31%) of studies had an author employed by a for-profit organisation, including 54 (29%) who had an author employed by the company producing the drug being evaluated.
- 134 (72%) of studies had an author with a conflict of interest with a for-profit organisation, including 111 (61%) who had an author receiving support from the producer of the drug being assessed.
- This added up to 145 (78%) of studies with biased authors. 122 (66%) meta-analyses had authors employed by or receiving support from the producer of the drug under evaluation.
- 53 of the studies had no authors reporting a conflict of interest, but 13 of these had authors who’d declared them elsewhere.
How many trials had negative conclusions in their abstract?
- 127 (69%) had no negative conclusions
- 20 concluded the drugs didn’t work or worked worse than the comparator
- 20 concluded that the drug had poor safety or toxicity
- 21 said nothing positive and said the evidence was low quality, limited or inconclusive, which was counted as negative
Comparing industry-biased and non-industry-biased trials
Studies with industry-employed authors were less likely to:
- Use a literature review (21% v 91%, p<0.001)
- Use individual-level data (11% v 79%, p<0.001)
- Use an active comparator (not placebo) (5% v 17%, p=0.04)
And industry-influenced trials gave biased results:
- Those with an industry-employed author were 22 times less likely to report a negative conclusion (1/54 [2%] vs 57/131 [44%], p<0.001)
Studies were less likely to have negative conclusions if they were in specialist psychiatric journals (23/103, 23%) compared to general medical journals (9/15, 60%, p=0.004) and if they were published in the US (14/66, 21%) compared to elsewhere (44/119, 37%, p=0.03).
Even if the studies with industry-employed authors were removed from the analysis, studies were still less likely to have a negative conclusion if they had an author with a conflict of interest with the company producing the drug being evaluated (22/68 v 34/60, p=0.007).
No one is surprised that the research funded by pharmaceutical companies appears to be biased. But they might be surprised to find that that bias has already crept so terminally into the realm of meta-analyses.
Nearly a third of antidepressant meta-analyses were authored by drug company employees. Only one of these had a negative conclusion, and even that only concerned side effects. Over another third had authors with less serious conflicts of interest and these reported less negative conclusions than independent studies too. Meta-analyses, the go-to resource of busy doctors around the world, seem to be the latest pharma marketing tool.
You could argue that basing how biased a study is on the conclusions in the abstract is superficial, but professionals only have time to read those very same conclusions.
To play devil’s advocate you might wonder if the authors of non-industry funded trials were also biased, hell-bent on proving that antidepressants are useless. Hardly likely though.
Implications for practice
If you want this paper’s results to improve your practice, you might want to:
- Treat meta-analyses written by authors with conflicts of interest with extreme suspicion
- Always read past the abstract
- Preferentially read independent meta-analyses, like Cochrane reviews
- Remember that a lack of negative conclusions doesn’t mean the drug doesn’t have flaws
Ebrahim S, Bance S, Athale A, Malachowski C, Ioannidis JPA. (2015) Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. Journal of Clinical Epidemiology 2015, doi: 10.1016/j.jclinepi.2015.08.021 [Abstract]
Lexchin J, BeroLisa A, DjulbegovicBenjamin, ClarkOtavio. (2003) Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003; 326 :1167
Eyding D, Lelgemann M, Grouven U, Härter M, Kromp M, Kaiser T. et al (2010) Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ 2010; 341 :c4737
Jørgensen AW, Maric KL, Tendal B, Faurschou A, Gøtzsche PC. (2008) Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: Differences in methodological quality and conclusions. BMC Medical Research Methodology 2008, 8:60
- JawshBeavz via Compfight cc
- KeithRaynor via Compfight cc
Antidepressant meta-analyses: big business and bias https://t.co/mqNK0tknnv via @sharethis #legemidler
@Mental_Elf good to see this important influence highlighted, more on this soon in @advancesDD specifically relating to drug interactions
Antidepressant meta-analyses: big business and bias https://t.co/IcYuyZG2NU #MentalHealth https://t.co/FaFLlwBbVc
#Antidepressant meta-analyses: big business and bias https://t.co/N6PNoU8q1Q Via: @Mental_Elf
@Mental_Elf Treat meta-analyses written by authors with conflicts of interest with extreme suspicion? Powerful and deeply concerning!
RT @PsychiatrySHO: 78% of antidepressant meta analyses have biased authors, with less negative conclusions.
A @Mental_Elf blog by me.
RT @Mental_Elf: Antidepressant meta-analyses: big business & bias https://t.co/0MABRxk2rD Research by @ShanilEbrahim Blog by @PsychiatrySHO
Antidepressant meta-analyses: big business and bias https://t.co/kQooWsE8vv vía @sharethis
Three quarters of meta-analyses about antidepressants have biased authors https://t.co/Dwsuaui3qT
“Meta-analyses, the go-to resource of busy doctors around the world, seem to be the latest pharma marketing tool.”
RT @Mental_Elf: Less than one third of #antidepressant meta-analyses had negative statements in abstract https://t.co/0MABRxk2rD #BadPharma
Ioannidis et al: 72% of antidepressant MAs had an author with a conflict of interest https://t.co/0MABRxk2rD @bengoldacre @PsychiatrySHO
RT @Mental_Elf: US antidepressant annual market = $10 billion. Unsurprisingly, most meta-analyses are biased https://t.co/0MABRxk2rD
Antidepressant meta-analyses: big business and bias https://t.co/YZb5YPazYL via @sharethis
Antidepressant meta-analyses: big business and bias https://t.co/gyYvenemWW @Mental_Elf
Today @PsychiatrySHO on industry involvement in antidepressant research, which leads to bias https://t.co/0MABRxk2rD
RT @Mental_Elf: 1 in 4 antidepressant meta-analyses are paid for by a company producing the drug being evaluated https://t.co/0MABRxk2rD #B…
RT @cgblanch1: Uno de cada 4 meta-análisis sobre antidepresivos los financia la compañía que produce la medicación evaluada https://t.co/AJ…
“Antidepressant meta-analyses: big business and bias” https://t.co/CfbjgE0Fk9
RT @Mental_Elf: Don’t miss: Three quarters of meta-analyses about antidepressants have biased authors https://t.co/0MABRxk2rD #EBP
Meta-analyses of antidepressants done by pharma employees are more likely to show positive results. https://t.co/RwPynckclc via @bengoldacre
RT @Mental_Elf: Grt advice from @PsychiatrySHO Treat meta-analyses written by authors w/ conflicts of interest w/ extreme suspicion https:/…
Not a good look for medical research https://t.co/tfgnqIE5t3 #bias #medication #depression
This is important from @Mental_Elf – big business and bias, antidepressants https://t.co/SaIEidZuiL
Three quarters of meta-analyses about antidepressants have biased authors https://t.co/u7uTHmc3c7 via @Mental_Elf
Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. https://t.co/8qLcm2wqhl
#Antidepressant #metaanalyses: big business & bias https://t.co/xUxEZf0WHA @Mental_Elf looks at industry sponsored #research
Alex – I am surprised that people believe that meta-analyses are ‘objective’ or somehow less prone to bias than regular studies. Indeed, meta analyses are the ideal environment for bias to go un-noticed – meta analyses are a Trojan Horse in science.
Meta-analysis is in essence quite like conducting a regular study – it involves collecting data and inferential statistics and prone to the same Questionable Research Practices (QPRs). Unlike a regular study, however, meta-analytic authors ‘choose’ their own data. They have inclusion and exclusion criteria, but these are their ‘own’ criteria (however well justified or not).
Pharam are a very easy (and justifiable) target – in fact, sometimes I think they carry a big target just to help. But as I’m sure you would agree… the issues raised here about meta analyses (and many others) apply beyond pharma to other interventions including psychotherapy.
Meta analyses in healthcare are not the solution to our problems but a problem in need of a solution
RT @Mark_Sheldon: Don’t miss: Three quarters of meta-analyses about antidepressants have biased authors https://t.co/LBuYtP61NH #EBP (via @…
Exploring incidences of bias in meta-analyses of antidepressants – prevalent is the word https://t.co/tb5C28khD2
Three quarters of meta-analyses about antidepressants have biased authors https://t.co/h8GLm8DIdp via @Mental_Elf
Three quarters of meta-analyses about antidepressants have biased authors https://t.co/5Nxt66l9Gf via @Mental_Elf
Antidepressant meta-analyses: big business and bias https://t.co/R5dxYRd6cV via @sharethis
Most popular blog this week? It’s @PsychiatrySHO Antidepressant meta-analyses: big business & bias https://t.co/0MABRxk2rD Great work Alex!
Only 1/54 antidepressant meta-analyses with industry author report negative result vs 57/131 w/o COI via @Mental_Elf https://t.co/uxMUDlPWjZ