We elves spend much of our time on the Internet. Maybe that’s why we’ve given research into Internet-based therapy a fair bit of coverage. We’ve shared positive findings (e.g. Jones, 2014; Steele 2013) as well as less compelling evidence (e.g. Bould, 2015; Dash, 2015). And we’ve also raised questions about the quality of the research (Cristea, 2014).
There’s growing evidence that some Internet-based interventions work for some people. They’re also relatively low-cost, but it doesn’t follow that they are necessarily cost-effective. A new systematic review looks at whether or not they are (Donker et al 2015).
The authors carried out an extensive search of the medical literature, including Medline, EMBASE and 7 other databases. The search criteria targeted economic evaluations of Internet-based interventions published between 1990 and July 2014, limited to studies in humans and written in English for peer-reviewed journals.
Titles and abstracts were screened independently by 2 different researchers and only full economic evaluations carried out alongside randomised controlled trials were included. Data extraction was also completed independently by two reviewers.
Incremental cost-effectiveness ratios (ICERs; representing cost per unit of outcome) were reported in local currency and also converted into pounds sterling for comparability using 2012 prices.
Identified papers were quality assessed using the Drummond 35-item checklist.
5,846 unique titles were identified by the search, with 236 full texts considered for inclusion. 16 studies met the criteria and were included in the review.
Mental health conditions studied were:
- Depression (n=4)
- Smoking (n=3)
- Social phobia (n=3)
- Alcohol use (n=2)
- Panic disorder (n=1)
- Health anxiety (n=1)
- Anxiety (n=1)
- Suicidal ideation (n=1)
CBT was the most common therapeutic mode evaluated, and the control varied across studies.
The duration for the interventions evaluated was between 4 weeks and 6 months. Most interventions showed some improvement in the primary or secondary outcomes over time, with two studies finding stronger clinical effects for guided compared with unguided Internet interventions.
For depression, the evidence suggests that Internet interventions probably are cost-effective, but only marginally. For anxiety and social phobia, the results are a little more compelling with higher probabilities that the interventions would be cost-effective. A number of interventions were found to ‘dominate’, which means they were both more effective and less costly than the alternative.
A critical feature of economic evaluations is their comparability in order to inform resource allocation decisions. For this reason, they tend to adopt a generic measure of health outcome, namely quality-adjusted life years (QALYs). In general in this review, the use of QALYs showed relatively small benefits in the included studies and therefore modest cost-effectiveness results. All cost-per-QALY analyses used the EQ-5D to measure health status.
The quality of the studies was moderate, with some reporting to a high standard and some with major oversights.
The review covered different treatments for different conditions using different outcome measures. This makes it difficult to summarise the findings. The table below shows the studies that reported cost-effectiveness in terms of cost-per-QALY (NICE’s preferred outcome for health technology assessments). However, it’s important to bear in mind that some other studies simply didn’t report cost-per-QALY because no QALY benefit was found; therefore, this table gives an overly positive picture of the findings.
|Anxiety||Email-guided ICBT vs active wait-list control
|Alcohol||Chat-guided ICBT+ vs unguided ICBT
|Health anxiety||Email-guided ICBT vs Attention control
|Social anxiety||Email-guided ICBT vs CBGT
|Depression||Chat-guided ICBT vs TAU
|Depression||(a) Email-guided ICBT vs (b) Email-guided IPST vs (c) Wait-list
|£19,371 (a vs c)|
|£9,873 (b vs c)|
|Smoking cessation||(a) Internet-based multiple tailoring & F2F counselling vs (b) Inter-based multiple tailoring vs (c) TAU
|£33,124 (a vs c)|
|£15,096 (a vs b)|
The authors concluded that
Guided Internet interventions for depression, anxiety, smoking cessation and alcohol consumption demonstrated higher probabilities of being more cost-effective than control at an ICER range of £1,801-33,124.
The study’s reporting standards are not high, and there are some details missing. Inclusion and exclusion criteria are defined in an unclear and ambiguous way. There is no PICO statement, which makes it challenging to identify who and what was being evaluated in the included studies. It is also not clear what data were extracted.
When reporting the findings of economic evaluations, the probability that the intervention is cost-effective is not the key finding; it is simply a way of characterising uncertainty. The focus should be on the average (mean) cost-effectiveness in the population.
This review demonstrates that Internet interventions do not offer impressive benefits in terms of QALYs. While there is room for debate about the validity of QALYs in these populations, the interventions also show only modest benefits in terms of clinical outcomes.
So a key question that needs to be answered is whether or not Internet based interventions can be cost-saving. For anxiety in particular, this seems like a real possibility.
Donker T, Blankers M, Hedman E, Ljótsson B, Petrie K, Christensen H. Economic evaluations of Internet interventions for mental health: a systematic review. Psychol Med. 2015;45: 3357–3376. [PubMed]
Bould H. E-therapy for eating disorders: review finds lack of evidence for digital treatment or prevention. The Mental Elf. 2015.
Cristea I. Internet-based CBT for chronic somatic conditions: problematic meta-analysis offers an overly enthusiastic appraisal. The Mental Elf. 2014.
Dash S. Computerised CBT for depression is no better than usual GP care: the REEACT trial. The Mental Elf. 2015.
Jones A. Internet-based interventions for harmful drinking show small beneficial effects. The Mental Elf. 2014.
Steele D. We all know that the Internet can be a dangerous place for people with eating disorders, but can it also help them get better? The Mental Elf. 2013.
Are Internet interventions cost-effective for mental health? https://t.co/8KPyRfd0V9 via @sharethis
Are Internet interventions cost-effective for mental health? https://t.co/K1KhA9M6mf #MentalHealth https://t.co/KqlFE0NR7I
RT iVivekMisra Are Internet interventions cost-effective for mental health? https://t.co/cAZJFTHRmf #MentalHealth https://t.co/wAt4H2xYWW
Today @ChrisSampson87 reports on an SR of economic evaluations of internet-based interventions for mental illness https://t.co/aiOAtghL2P
Are Internet interventions cost-effective for mental health? https://t.co/lqAlU9W1s4
Not clear that internet-based interventions are cost-effective for mental health https://t.co/VDlYgf01RV summary from the fab @Mental_Elf
Internet interventions are supposed to be cheap & so save money but maybe not. https://t.co/uoklAuvlxm via sharethis
@Mental_Elf @AccessCounsell they can be but it depends on what you are counting and how. As with all things
@akamisspink @Mental_Elf agreed together with the purpose of the research question, personally I prefer a grounded theory approach
@AccessCounsell @Mental_Elf exactly, there is inherent bias towards models eg cbt where ‘measures’ provide – sometimes spurious- objectivity
@AccessCounsell @Mental_Elf &rejection of qualitative ‘measures’ reject the experience of clients. It’s Not bad, just not the whole picture
Economic evaluation says guided internet interventions may help with depression, anxiety, smoking & alcohol https://t.co/aiOAtghL2P
@Mental_Elf @ChrisSampson87 good to see economic evaluation as its often lacking yet can be most persuasive to policy makers & commissioners
@Mental_Elf That’s interesting as others (no surprise) seem to have shown that talking therapy much better in person – not online #CBT.
Are Internet interventions cost-effective for mental health? https://t.co/IFNphoHiU7 via @Mental_Elf
Are #Internet interventions cost effective for #mentalhealth? https://t.co/qbzM6kXJ9S A look at the #evidence from a #systematicreview
RT NHFTNHSLibrary Are #Internet interventions cost effective for #mentalhealth? https://t.co/9o7BtQJuVK A look at the #evidence from a #sy…
Are Internet interventions cost-effective for #mentalhealth? via @Mental_Elf https://t.co/63POK2b88V
RT SanazRiahi Are Internet interventions cost-effective for #mentalhealth? via Mental_Elf https://t.co/9o7BtQJuVK
RT @ChrisSampson87: New from me on @Mental_Elf I discuss a review of economic evaluations of Internet interventions for mental health https…
RT @Mental_Elf: Mental health SR finds internet-based treatment most cost-effective for anxiety disorders https://t.co/aiOAtghL2P
Don’t miss: Are Internet interventions cost-effective for mental health? https://t.co/aiOAtghL2P
Are Internet interventions cost-effective for mental health? not especially by the look https://t.co/rxZdnvIPBW
Are Internet interventions cost-effective for mental health? https://t.co/OI90ZNKOfw via @sharethis
Are Internet interventions cost-effective for mental health? https://t.co/F0kvnRgqxf via @sharethis
Are Internet interventions cost-effective for mental health? https://t.co/oHCvEXAnhU #mentalelf #icbt