Major depression is extremely prevalent with at least one in eight people living with the condition. As well as health and health care utilisation, major depression has wider negative impacts on employment through decreased work performance and absenteeism. Consequently, it is in the patient’s and society’s interest to reduce the severity of depressive symptoms.
Standard care for depression typically involves antidepressant medication and/or talking treatments such as CBT, psychotherapy or counselling. However, there are extensive waiting lists for talking treatments for newly referred patients, with waiting times typically being in excess of six weeks. The authors of this paper (Kolovos et al, 2016) examine an intervention that aims to take advantage of this waiting period.
The authors highlight an emerging body of evidence that suggests that internet-based guided self-help therapies can be effective in reducing the severity of depressive symptoms (Andersson and Cuijpers, 2009; Cuijpers et al., 2011; Richards and Richardson, 2012). They have the additional benefit of being easy to access and comparatively cheap to implement. Given the long waiting lists for treatment, the authors felt intervening during the waiting period could be an effective use of time. Should an intervention be successful within this period then it should have the benefit of reducing the quantity of more costly face-to-face treatment, whilst also leading to an improvement in health outcomes.
The aim of the trial was therefore to evaluate the clinical and cost-effectiveness of providing an internet based intervention for major depressive patients waiting for face-to-face treatment.
The study is a two arm randomised controlled trial, both of which received ‘usual care’ after the waiting period in which the intervention took place. The two trial arms can be characterised as follows:
- Intervention arm: During the waiting period, patients receive a structured internet based self-help intervention based on problem solving therapy. The intervention (called ‘Taking Control’) consists of five weekly sessions, each with an assignment from which they get feedback from an appointed coach and is designed to give them the tools to help regain control of their life.
- Control arm: The control arm participants received ‘enhanced usual care’: they received a self-help book (entitled ‘Everything under control: overcome your problems and worries by self-analysis’) during the waiting period.
Participants were recruited from two outpatient clinics that provided services in ten different locations. To be included within the trial, participants had to be/have:
- Aged 18 years or over
- Access to the internet
- Fluent in Dutch
- On a waiting list for face to face treatment
- Met the DSM-IV criteria for major depression.
Those at high risk of suicide or with the presence of bipolar or psychotic disorders were excluded.
Outcomes were measured at baseline, at 8 weeks (immediately post-intervention) and then at six months and 12 months post baseline.
- CES-D: The primary clinical measure used within the study was the Center for Epidemiological Studies for Depression Scale (CES-D) which contains 20 items and gives a score from 0 to 60 with higher scores denoting more severe depression.
- EQ-5D: The primary health economic measure was the EQ-5D-3L which is a generic measure of health related quality of life. The health profiles generated were combined with the Dutch EQ-5D value sets to calculate index values. These values can be combined with length of life to generate quality-adjusted life years (QALYs).
For the primary analysis, costs were measured from a societal perspective. Costs within the trial included:
- Direct costs: primary care, secondary care, mental healthcare and the cost of the intervention.
- Indirect costs: productivity losses (e.g. absenteeism and presenteeism)
Secondary analyses included: Mental healthcare perspective (excluded other healthcare costs and societal costs), national healthcare perspective, and finally, only included those who attended at least three out of five sessions.
Methods of analysis
- An intention to treat principle was adopted
- Multiple imputation was used to address missing data
- Seemingly unrelated regressions were used to estimate cost and effect differences across groups
- Incremental cost-effectiveness ratios were calculated by dividing the difference in cost by the difference in clinical effects
- Non-parametric bootstrapping was used to estimate the confidence intervals
- Cost-effectiveness acceptability curves were generated to characterise uncertainty.
In total, 269 patients were recruited into the trial. 136 were randomised into the intervention arm, whilst 133 were randomised into the control arm. There were no significant differences in composition between the two trial arms.
The adherence to the intervention was poor, but that’s quite common in trials of online interventions. Only 2/3 of the intervention arm completed two of the five sessions, whilst just 1/3 completed three or more sessions. Notably only 12.5% completed the entire intervention as intended. Likewise, 2/3 of the control arm did not use the self-help book.
- As shown in the graph below, those in the intervention group on average received two more face-to-face therapy sessions than those in the control arm (21 vs 19)
- There was no difference in the number of group sessions (14 vs 14)
- Costs were €1,579 greater (not statistically significant (95% CI -1,395 to 4,382)) in the intervention arm compared to the control arm.
- Both arms improved substantially in CES-D score at 12 months
- There was no significant difference in improvement between trial arms in CES-D score at 12 months (mean difference, 0.49 95% CI -4.97 to 3.94)
- Likewise there was no evidence for effectiveness in terms of QALYs (mean difference, 0.01 95% CI -0.05 to 0.06)
- Thus, there appears to be no evidence of effectiveness resulting from the intervention.
Regarding cost-effectiveness, the paper makes a couple of mistakes in their ICER calculations. They report the ICERs as being negative, when in fact, based on their reported costs and effects, they are positive. A negative ICER would also cause interpretability issues that are not discussed. Here I present the ‘corrected results’.
- The point estimate for the cost-effectiveness was €157,900 per QALY and thus failed to demonstrate cost-effectiveness with just a 30% chance that the intervention was cost-effective at a willingness to pay of €30,000 per QALY
- Using the CES-D, there was just 49% chance that the intervention was the more cost-effective at a willingness to pay of €3,000 per CES-D point
- Sensitivity analysis suggested that the results may be more favourable should an alternative (e.g. mental health care only) perspective be taken for costing with a 68% chance the intervention is more cost-effective than usual care
- Considering only those who attended three or more sessions, there was still no evidence of effect in terms of CES-D (mean difference, 1.30 95% CI -4.39 to 6.98). For these individuals, incremental costs were even higher at €2,689 (95% CI -1,215 to 6,814) for the intervention arm.
The results were underwhelming. Rather than reducing the number of face-to-face sessions required, the intervention group actually received more than the control arm. Likewise in terms of outcomes there was no difference between the two arms in terms of CES-D or in terms of QALYs. The authors point towards the sensitivity analyses for positives; in particular in relation to how different perspective may effect costs. Personally, I was not convinced by these.
There are suggestions for why there may not have been any effect, these include:
- Self-help book may have contaminated usual care group
- Although the ‘enhanced usual care’ has potential for contamination, in this instance, I do not think we can give this much credence given how few people actually said they had looked at the book (the authors acknowledge this)
- Low adherence
- The authors suggest that improved adherence could have led to larger clinical effects and thus better cost-effectiveness results. There are issues with this. First, adherence was incredibly low. It is important to consider the generalisability to the wider context. That is, how would the results be reflected in the real word setting? Trials tend to be ‘perfect patients’ and consequently it is usual to see much poorer adherence in the real world setting than in trials. It would therefore be highly unexpected to see better adherence outside the trial. Second, one of the sensitivity analyses examined only those who completed more than 3/5 of the sessions, that is, those with the best adherence. The incremental costs in this group were in fact even greater, but with only slightly improved outcomes. This did not have any significant impact on cost-effectiveness analysis results.
So where does this leave us?
Well, both arms improved significantly in terms of clinical effects which suggests that face-to-face treatment for major depression is effective. The issue is the six weeks of waiting to get face-to-face treatment. There is no evidence to suggest that this intervention works in terms of reducing costs or the 12 month outcomes. What would have been interesting would have been if the authors had presented the outcome data from immediately after the intervention period, that is, to see if the intervention had led to any short term, immediate improvement in outcomes prior to face-to-face treatment. The fact that this data was not presented suggests however that there probably were no improvements.
People with major depression having to wait six weeks for face-to-face treatment is clearly an issue. This intervention in its current form however does not appear to be the answer. Given the apparent effectiveness of standard treatment, there should be increasing pressure on reducing the waiting times for treatments.
- Long waiting times exist for treatment for major depression
- Internet-based self-help interventions can be designed to make use of this waiting time
- This trial compared an Internet-based problem-solving guided self-help treatment enhanced usual care
- The Internet-based treatment had little impact in terms of costs or effectiveness.
Kolovos S, Kenter RMF, Bosmans JE, Beekman ATF, Cuijpers P, Kok RN, van Straten A. (2016) Economic evaluation of Internet-based problem-solving guided self-help treatment in comparison with enhanced usual care for depressed outpatients waiting for face-to-face treatment: A randomized controlled trial, Journal of Affective Disorders, Volume 200, August 2016, Pages 284-292, ISSN 0165-0327. http://dx.doi.org/10.1016/j.jad.2016.04.025.
Andersson G, Cuijpers P. (2009) Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cognit. Behav. Ther. 38, 196–205.
Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. (2011) Self-guided psychological treatment for depressive symptoms: a meta-analysis. PLoS One 6, e21274.
Richards D, Richardson T. (2012) Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329–342.
.@Mental_Elf great piece @captain_canaway looks like you’ve done a lot of work re-analysing the data, thanks for doing that & sharing
@ian_hamilton_ @Mental_Elf Thanks Ian – very kind, there was some head scratching as to how they’d got a negative ICER from two positives!
Internet-based problem-solving guided self-help for depression whilst waiting… https://t.co/FmfuFj7wza #MentalHealth https://t.co/CUX6eI1NZy
Today @captain_canaway on RCT & economic eval of web-based guided self-help for depression
Waiting for talking treatment for depression?
Trial explores if online guided self-help is effective/cost-effective
@Mental_Elf @stephnicallan yet more evidence that more therapists and not online cbt is the answer to waiting times for #depression
#Internet-based #problemsolving guided self-help for #depression whilst waiting for therapy https://t.co/Y8BSe5WKMm #Evidence from an #RCT
Hi @spyros_kolovos We’ve blogged your paper!
Do you have any comments?
@Mental_Elf @spyros_kolovos @captain_canaway For short-term FX on a similar trial, see https://t.co/bAIiTrQ16f, CEA in submission
@robinnkok @Mental_Elf @spyros_kolovos Thanks for the link – interesting…I look forward to the CEA paper.
@Mental_Elf @spyros_kolovos @captain_canaway Also contamination by book was unfortunate, but IRB required us to give ‘something’ to controls
@robinnkok @Mental_Elf @captain_canaway Thanks for blogging! It’s interesting too that the target population was severely depressed
@Mental_Elf @spyros_kolovos @captain_canaway for short term results see https://t.co/DsAHFthsCk
RT @Mental_Elf: Don’t miss
Internet-based guided self-help for depression whilst waiting for therapy
Internet-based problem-solving guided self-help for depression whilst waiting for therapy https://t.co/H5YsADZAv7 #selfhelp
Internet-based problem-solving guided self-help for depression whilst #cbtfordepression https://t.co/kKszspVatK https://t.co/SPP0Pc7Eqn
“Internet-based problem-solving guided self-help for depression whilst waiting for therapy https://t.co/OkBieJhb5p
Internet-based problem-solving guided self-help for depression
@DrSNThomas @DrMikePsych https://t.co/OkBieJhb5p
@captain_canaway – enjoyed your critical analysis. Any qualitative views abt the prob-solving guided self help?
Internet-based problem-solving guided self-help for depression whilst waiting for therapy https://t.co/MfthVprJTc via @sharethis
Internet-based problem-solving guided self-help for #depression https://t.co/frWoPtrP8K https://t.co/HfQ1JyqzRl
[…] Summary of a trial of the clinical and cost-effectiveness of internet-based problem-solving guided self-help intervention, in comparison with enhanced usual care, for outpatients on a waiting list for face-to-face treatment for major depression. […]