We elves spend much of our time on the Internet. Maybe that’s why we’ve given research into Internet-based therapy a fair bit of coverage. We’ve shared positive findings (e.g. Jones, 2014; Steele 2013) as well as less compelling evidence (e.g. Bould, 2015; Dash, 2015). And we’ve also raised questions about the quality of the research (Cristea, 2014).

There’s growing evidence that some Internet-based interventions work for some people. They’re also relatively low-cost, but it doesn’t follow that they are necessarily cost-effective. A new systematic review looks at whether or not they are (Donker et al 2015).

Methods

The authors carried out an extensive search of the medical literature, including Medline, EMBASE and 7 other databases. The search criteria targeted economic evaluations of Internet-based interventions published between 1990 and July 2014, limited to studies in humans and written in English for peer-reviewed journals.

Titles and abstracts were screened independently by 2 different researchers and only full economic evaluations carried out alongside randomised controlled trials were included. Data extraction was also completed independently by two reviewers.

Incremental cost-effectiveness ratios (ICERs; representing cost per unit of outcome) were reported in local currency and also converted into pounds sterling for comparability using 2012 prices.

Identified papers were quality assessed using the Drummond 35-item checklist.

Results

5,846 unique titles were identified by the search, with 236 full texts considered for inclusion. 16 studies met the criteria and were included in the review.

Mental health conditions studied were:

• Depression (n=4)
• Smoking (n=3)
• Social phobia (n=3)
• Alcohol use (n=2)
• Panic disorder (n=1)
• Health anxiety (n=1)
• Anxiety (n=1)
• Suicidal ideation (n=1)

CBT was the most common therapeutic mode evaluated, and the control varied across studies.

The duration for the interventions evaluated was between 4 weeks and 6 months. Most interventions showed some improvement in the primary or secondary outcomes over time, with two studies finding stronger clinical effects for guided compared with unguided Internet interventions.

For depression, the evidence suggests that Internet interventions probably are cost-effective, but only marginally. For anxiety and social phobia, the results are a little more compelling with higher probabilities that the interventions would be cost-effective. A number of interventions were found to ‘dominate’, which means they were both more effective and less costly than the alternative.

A critical feature of economic evaluations is their comparability in order to inform resource allocation decisions. For this reason, they tend to adopt a generic measure of health outcome, namely quality-adjusted life years (QALYs). In general in this review, the use of QALYs showed relatively small benefits in the included studies and therefore modest cost-effectiveness results. All cost-per-QALY analyses used the EQ-5D to measure health status.

The quality of the studies was moderate, with some reporting to a high standard and some with major oversights.

The review covered different treatments for different conditions using different outcome measures. This makes it difficult to summarise the findings. The table below shows the studies that reported cost-effectiveness in terms of cost-per-QALY (NICE’s preferred outcome for health technology assessments). However, it’s important to bear in mind that some other studies simply didn’t report cost-per-QALY because no QALY benefit was found; therefore, this table gives an overly positive picture of the findings.

Conclusions

The authors concluded that

Guided Internet interventions for depression, anxiety, smoking cessation and alcohol consumption demonstrated higher probabilities of being more cost-effective than control at an ICER range of £1,801-33,124.

Discussion

The study’s reporting standards are not high, and there are some details missing. Inclusion and exclusion criteria are defined in an unclear and ambiguous way. There is no PICO statement, which makes it challenging to identify who and what was being evaluated in the included studies. It is also not clear what data were extracted.

When reporting the findings of economic evaluations, the probability that the intervention is cost-effective is not the key finding; it is simply a way of characterising uncertainty. The focus should be on the average (mean) cost-effectiveness in the population.

This review demonstrates that Internet interventions do not offer impressive benefits in terms of QALYs. While there is room for debate about the validity of QALYs in these populations, the interventions also show only modest benefits in terms of clinical outcomes.

So a key question that needs to be answered is whether or not Internet based interventions can be cost-saving. For anxiety in particular, this seems like a real possibility.

Links

Primary paper

Donker T, Blankers M, Hedman E, Ljótsson B, Petrie K, Christensen H. Economic evaluations of Internet interventions for mental health: a systematic review. Psychol Med. 2015;45: 3357–3376. [PubMed]

Other references

Bould H. E-therapy for eating disorders: review finds lack of evidence for digital treatment or prevention. The Mental Elf. 2015.

Cristea I. Internet-based CBT for chronic somatic conditions: problematic meta-analysis offers an overly enthusiastic appraisal. The Mental Elf. 2014.

Dash S. Computerised CBT for depression is no better than usual GP care: the REEACT trial. The Mental Elf. 2015.

Jones A. Internet-based interventions for harmful drinking show small beneficial effects. The Mental Elf. 2014.

Steele D. We all know that the Internet can be a dangerous place for people with eating disorders, but can it also help them get better? The Mental Elf. 2013.

Photo credits

• GotCredit CC BY 2.0
• gratisography.com CC0
• Rosmarie Voegtli CC BY 2.0
• James Cridland CC BY 2.0
• CollegeDegrees360 CC BY-SA 2.0