Psychotherapy trials are particularly vulnerable to researcher allegiance, the researcher’s ‘belief in the superiority of a treatment and in the superior validity of the theory of change that is associated with the treatment’ (Leykin and DeRubeis, 2009). Several meta-analyses have consistently found larger effects when researcher allegiance is present in psychotherapy trials.

However, the mechanisms behind researcher allegiance have held a contentious status, with some researchers suggesting it to be the result of a self-serving bias, acting to overestimate the effects of therapies, and others claiming that researcher allegiance to be merely the mark of highly protocol adherent therapy delivered with increased competence.

In a recent study published in BMJ Open, Dragioti and colleagues (Dragioti et al., 2015a) set to systematically investigate the extent to which researcher allegiance is reported in meta-analyses (MAs) of randomised controlled trials (RCTs) of psychotherapy, as well as in the primary RCTs included in these meta-analyses.

Methods

The authors included meta-analyses of psychotherapeutic interventions published in journals with an impact factor higher than 5, based on the Web of Science Journal Citation Report for 2012.

Included MAs could evaluate any form of psychotherapy, as long as they contained studies that were RCTs assessing the direct comparisons of different forms of psychotherapy, different modes of delivery of the same psychotherapy (for example group versus individual therapy), psychotherapy versus control conditions (treatment as usual, waiting list, no treatment) or psychotherapy versus medication. Included MAs also had to provide a clear description of the main outcome or class of outcomes regarding mental or medical disorders. In order to prevent duplication if an RCT appeared in more of the included MAs, it was evaluated only for the first publication that described it.

The authors considered that a study showed the presence of researcher allegiance if one or more of the co-authors:

1. Developed the intervention;
2. Developed the intervention and directly trained the therapists;
3. Developed both the intervention and supervised the therapists;
4. Supervised and/or trained the therapists alone; or
5. Advocated the therapy.

When sufficient information on researcher allegiance was not reported in the full text, the authors also checked the references section to ascertain if a reference was made to previously published research by the same principal investigators showing the efficacy of the therapy as compared to no treatment or its superiority in relation to other treatments.

Results

146 meta-analyses met the inclusion criteria and were included in the analysis. They investigated a large range of psychotherapies: 25% CBT alone, 50% CBT combined with various other psychotherapy treatments, and the remaining 25% included other psychotherapies alone or in combination.

• Each study included a median of 10 allegiant trials (IQR 7-15).
• Evidence of researcher allegiance was present in approximately 40% of the included MAs (56 out of 146). Out of these 56 allegiant MAs, approximately 90% (50 MAs) had an author who had been co-author of an allegiant RCT included in the MA.
• Only about 17% of the included MAs (25 out of 146) discussed or reported researcher allegiance either in the meta-analysis or in the included RCTs. Level of allegiance was rated as high in about half of these 25 MAs. Just 4% of the MAs (6 out of 146) evaluated the presence of allegiance.
• From the 1,198 included RCTs, whose provenance was from the included MAs, around 66% were rated as allegiant, while around 12% as non-allegiant. The remaining 22% were rated as unclear.
• About half of the allegiant RCTs provided evidence of author researcher allegiance in the text, while for the other half researcher allegiance was documented through a cited reference by the same principal investigators to their previous research.
• About 15% of the allegiant RCTs reported a disclosure statement of potential conflicts of interest (COIs) other than allegiance. The number of RCTs that disclosed allegiance was significantly different from the studies that reported any other COIs.

Conclusions

The authors concluded that:

We found that the vast majority of meta-analyses and primary RCTs of psychotherapeutic treatments published in high-impact journals failed to report researcher allegiance.

However, they also emphasised that:

This study suggests that, without a strict reporting policy, researcher allegiance is unlikely to be reported in meta-analyses and RCTs. Psychotherapy should move forward, following what is accomplished with pharmaceutical industry trials and sponsorship biases.

In fact, in another recent meta-analysis (Dragioti et al., 2015b), authors from the same group found there was an inflation in the effects of psychotherapy treatments when researcher allegiance was present. More specifically, they showed effect sizes were 30% higher when one of the authors of the report was an allegiant therapist in the trial.

Limitations

• The analyses was limited to high impact journals (IF>5) and might have missed some other relevant meta-analyses published in journals with a lower impact factor, but nonetheless relevant for psychotherapy research.
• As there is no agreement on what represents allegiance, the authors used one of the existing paradigms for assessing it. However, I would like to add that even if this is an unavoidable limitation, it must also be said that this coding paradigm is not without its own “allegiance”. The developer of this paradigm is Bruce Wampold, one of the champions of the idea that all psychotherapies work through the same common factors and that any observed differences between them are explained by researcher allegiance. Moreover, his strong allegiance is clearly conveyed in his proposal (Laska et al., 2014) to consider his own approach (the “common factors perspective”) as another evidence-based psychotherapeutic approach.

Links

Primary paper

Dragioti E, Dimoliatis I, Evangelou E. (2015a) Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal. BMJ Open 5, e007206. [Full-text]

Other references

Dragioti E, Dimoliatis I, Fountoulakis KN, Evangelou E. (2015b) A systematic appraisal of allegiance effect in randomized controlled trials of psychotherapy. Ann. Gen. Psychiatry 14, 25. [Full-text]

Laska KM, Gurman AS, Wampold BE. (2014) Expanding the lens of evidence-based practice in psychotherapy: a common factors perspective. Psychotherapy 51, 467–481. [PubMed abstract]

Leykin Y, DeRubeis RJ. (2009) Allegiance in Psychotherapy Outcome Research: Separating Association From Bias (PDF). Clin. Psychol. Sci. Pract. 16, 54–65.