Predictors of adherence to digital interventions for psychosis


In July 2008 the App Store opened on the iPhone. This was a first for digital consumer products and promised unprecedented access to software anytime, anywhere (as long as your battery didn’t die after about 10 minutes of use). It had huge implications for games, tools and productivity, with loads of market space for new start-ups to get in on the action; from Angry Birds to Evernote. However, the app revolution didn’t only create opportunities to shoot zombie pigs at range with a macaw, it also represented huge potential for a new type of health care.

The ability to input your own health data, track your progress, and receive advice on one device means that care can be personalised, affordable, and accessible to anyone with a smartphone. Apps can also be used in conjunction with services, acting as a self-care step before seeing a clinician or being used to show your keyworker how you’ve been in the last few weeks at the touch of a button.

Nevertheless, just as with any health care intervention, apps need to be carefully tailored toward those they aim to help. For example, carrying out research into adherence (that is, helping people stick with using the app) may hugely improve a digital intervention’s potential for curbing the symptoms of psychosis.

  • But what are the precise predictors of adherence?
  • Does the severity of symptoms affect the way in which someone interacts with a digital intervention?
  • Does involving service users in the design of the app generate a more appealing piece of software?

A new study from Clare Killikelly and colleagues aimed to answer these questions by systematically reviewing the current scientific literature around potential predictors of adherence to apps for people with psychosis (Killikelly et al, 2017).

This new review explored the factors that

This new review explored the factors that predicted adherence to digital technology in people with psychosis.


Searching through 5 online databases, authors identified studies adhering to the following criteria:

  • Adults aged 18-65 with a schizophrenia spectrum disorder
  • Was an intervention, trial, or observational study. Randomised Controlled Trials (RCTs), feasibility studies, cross-sectional, longitudinal comparison studies (with or without a control group) were all accepted
  • Used a digital mental health app (this could be web- or mobile-based). The app could have either been a psychoeducation, therapeutic, monitoring, self-help or symptom-checking tool
  • Measured adherence in some form.

From 2,639 studies initially identified, authors eventually whittled down 20 studies to include in the final analysis; excluding other studies for not being clinically relevant, having irrelevant research designs (e.g. posters and case-studies), or not including the relevant outcome measures.

RCTs and feasibility studies were assessed separately using the Clinical Trials Assessment Measure (CTAM), and observational studies were assessed using the Downs and Black scale; these are effectively different sets of recommended guidelines to enable a critique of their quality.


The included studies covered 656 participants with schizophrenia spectrum disorders, between the ages of 20-48 years.

There’s a dense range of results reported in the article, so I’m going summarise key findings as best I can:

Study characteristics

6 RCTs, 7 feasibility trials, and 7 observational studies were reviewed. In the full article, the characteristics of the participants in each type of study is broken down, e.g. mean age, n, etc.

Quality assessment

The mean CTAM score for RCTs was 77.3/100, with only this study scoring below 65; the general benchmark score indicating adequate quality.

Due to lack of randomisation, feasibility studies had a lower mean of 44.7. Of the observational studies included, only 3 were deemed ‘good’, and 4 scored ‘fair’, so had a mean quality rating of 20.

Adherence: types of assessment across studies

The figure below probably best depicts the different approaches to adherence that each study recorded –the percentage of the intervention completed (average 83.4%) by participants indicated for the top set of studies, and the percentage of completed interventions (average 73.4%) indicated for the bottom set.

Figure taken from: Killikelly et al, 2017. Improving Adherence to Web-Based and Mobile Technologies for People With Psychosis: Systematic Review of New Potential Predictors of Adherence

Figure taken from: Killikelly et al, 2017. Improving Adherence to Web-Based and Mobile Technologies for People With Psychosis: Systematic Review of New Potential Predictors of Adherence.

Which factors lead to dropout?

  • Age and gender were found to have no effect in most studies
  • Symptoms were concluded not to affect dropout (except in this one study)
  • Cognitive function was looked at in one study, and not found to predict dropout
  • Mobile phone based apps improved adherence vs. text-message-only technologies
  • The degree of social support users had while taking part also improved adherence
  • Service user involvement in the design of the tool improved adherence

Only four studies actually asked participants if they were keen to continue to use the intervention, and this averaged to 73.1% saying ‘yes’.

Strengths and limitations

This is a thorough review and the first of its kind, thereby making a positive contribution to the literature. The review identified important implications, which may help inform the delivery and development of e-health interventions.

Despite the comprehensive search that was undertaken, there were a relatively small number of studies to address each objective. As the authors noted, there was also heterogeneity in terms of the study interventions and outcomes. Overall, these issues made it difficult to make cross-study comparisons and draw more definitive conclusions about the significance of the review findings.

It was also noted that the authors excluded “conference abstracts and theses not published in a peer-reviewed journal”. Including articles from the grey literature could have helped identify novel technologies that have not yet undergone rigorous scientific testing (e.g. RCTs) but are nonetheless potentially valuable. This is particularly relevant to the field of e-health, where the development of technologies is outpacing research (Nilsen et al., 2012). Inclusion of non peer-reviewed sources may also reduce the risk of publication bias (McAuley, Tugwell, & Moher, 2000).


Overall this study was dealing with a mixed bag of evidence: varying levels of quality, differing intervention designs, and a range of outcome measures. Quite rightly, authors write that the review provides an up-to-date and comprehensive look at the field and synthesises preliminary conclusions based on diverse existing research. As it stands, it is noted that future reviews should focus on splitting levels of adherence by intervention, i.e. adherence in web-based technologies vs. app-based technologies.

However, can we truly dismiss the claims that symptoms do not affect adherence to a digital health intervention? While the given evidence doesn’t seem to point toward symptoms having a marked effect, we do not have data on individuals who turned down the intervention (or weren’t selected) and those who dropped out. This might mask conclusions if participants’ symptoms weren’t taken into account upon sample selection, and the authors didn’t point this out clearly in the review or suggest ways in which this could be ascertained.

While differences between interventions were mentioned in the article, I still feel like the authors have missed out a vital aspect of digital technology that could really help users engage. User Experience and User Interface (commonly abbreviated to UXUI) is a huge aspect of design when it comes to commercial apps; they keep users engaged and make the process of navigating the app as low energy as possible. It’s something that I feel is often overlooked or left undiscussed in the clinical sphere, and it’s not clear why. While it is definitely important to get intervention content right, it would be interesting for future research to pursue whether highly developed graphical interfaces can produce increased adherence to, for example, computerised cognitive behavioural therapy compared with simple text-based apps with the same content.

Do we need more User Experience as well as our current focus on User Interface?

Do we need more User Experience as well as our current focus on User Interface?


This comprehensive review has given a great overview of the current literature, with all its variances, and suggested some useful steps to improve the field further. Moving forward, it might also be necessary to use some UXUI tactics to get service users engaged with the hope that it’ll further help the audience it’s intended for.

Conflict of interest

Joe Barnby is currently employed part-time for Thrive Therapeutic Software: a clinical software company developing digital interventions for mental health care. Thrive has not had any part in the writing or reviewing of this article.


Primary paper

Killikelly C, He Z, Reeder C, Wykes T. (2017) Improving Adherence to Web-Based and Mobile Technologies for People With Psychosis: Systematic Review of New Potential Predictors of Adherence. JMIR Mhealth Uhealth 2017;5(7):e94 DOI: 10.2196/mhealth.7088

Other references

McAuley L, Tugwell P, Moher D. (2000) Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?. The Lancet, 356(9237), 1228-1231.

Nilsen W, Kumar S, Shar A. et al (2012) Advancing the science of mHealth. Journal of health communication, 17(sup1), 5-10.

Photo credits

Share on Facebook Tweet this on Twitter Share on LinkedIn Share on Google+
Mark as read
Create a personal elf note about this blog
Profile photo of Joe Barnby

Joe Barnby

Joe is currently a doctoral researcher at King’s College London at the IoPPN researching the neuroscience and psychology of belief and delusion formation, funded through the MRC. He also produces music, loves to write, is a big tech enthusiast, and recently got into bouldering (it’s a steep learning curve). You can often find him on Twitter.

More posts

Follow me here –

Profile photo of Muna Dubad

Muna Dubad

Muna Dubad is a PhD Student at the University of Warwick supervised by Dr Steven Marwaha and Dr Catherine Winsper. The PhD project aims to investigate the use of a mood-monitoring smartphone application (app) to support the assessment, engagement, and empowerment of young people presenting to mental health services with affective instability. Prior to this, Muna completed an M.Res Clinical Psychology Degree at the University of Birmingham and a BSc in Psychology at Coventry University. Muna also worked across different mental health settings as a Support Worker and as an Assistant Psychologist.

More posts

Follow me here –

Profile photo of Tayla McCloud

Tayla McCloud

Tayla is a PhD student in UCL’s Division of Psychiatry. For her PhD she is conducting the longitudinal cohort study SENSE, which is investigating the mental health of UCL students. Previously, she completed an MSc in Clinical Mental Health Sciences and worked as a Research Assistant on various randomised controlled trials. She has a particular interest in the mental health of young people, specifically depression and anxiety.

More posts

Follow me here –