The reason we started this website back in 2011 (other than an unhealthy interest in puns) was to support mental health professionals in keeping up to date with relevant and reliable research.

Back then, we’d spent just over a decade working on various NHS information projects for NICE and other organisations, so we were well aware of the difficulties that health practitioners faced in finding, reading and understanding evidence that they could actually use in practice.

We (Minervation Ltd) are publishing a White Paper today: Keeping up to date with reliable mental health research (PDF). The paper describes the problems faced by practitioners and others trying to find, appraise and use evidence-based research in practice. It quantifies the avalanche of evidence that we all get buried under from time to time, and it explores some of the solutions that we (the National Elf Service) and our comrades in the mental health and evidence-based healthcare communities are coproducing to help drive change.

We already know that:

• Health professionals need evidence more often than they get it (Covell 1985, Osheroff 1991, Hersh 1998, Gonzalez-Gonzales 2007, Magrabi 2008).
• Giving clinicians access to evidence reduces errors and improves outcomes (Ely 2005, Hoogendam 2008, Ely 2007, Marcelo 2013).
• The amount of reading time available to health and social care professionals varies enormously and time for reading research tends to decrease as people move further away from their training (Sackett, 1999).
• Accessibility, usability and reliability are the key to providing effective evidence resources (Durieux 2013, Shibuya 2013).

Methods

Our methodology for producing this White Paper can be divided into two sections:

• The first is a literature review of clinicians’ information seeking behaviour:
  • This builds on a study we conducted back in 2011 (Badenoch & De Brún, 2011), which reviewed the evidence on the information seeking behaviour of clinicians
  • We included the references from this previous review in our new White Paper and also updated the searches to find new research to include
  • The full methods of review are reported in the original study (Badenoch & De Brún, 2011).
• The second is a piece of information science research that we conducted on the PubMed database, which aimed to quantify the amount of high quality mental health treatment research published every year since 1973:
  • We searched PubMed for mental health research by exploding the MeSH “Psychiatry and Psychology Category”. We then used the “Systematic [sb]” subset search to identify Systematic Reviews (SRs) and Meta-analyses (MAs) and the “Randomized controlled trial” publication type to identify RCTs
  • PubMed is one of the leading biomedical databases, but it only provides access to a proportion (16.5%) of all the medical research published globally (Fraser et al, 2010), so we multiplied our PubMed results by 6.06, to gain our best estimate of the total published SRs/MAs and RCTs published worldwide.

Results

Literature review results

• In 2011, our heuristic aggregate yielded an estimate of between 58 and 87% of “evidence needs” being unmet in everyday practice. Reviewing more recent evidence has not changed this estimate substantially; 38-70% of clinical questions went unanswered in the newer studies we included (Del Fiol 2014, Anton 2014, Kloda 2014).
• “Evidence-based” decision tools often fail to help clinicians keep up to date with the latest evidence (Banzi 2011), so there remains substantial demand for evidence-based analysis that responds quickly to new evidence being published.
• Clinicians use the Internet to try to keep up to date, and change their practice based on what they find (Younger 2010).
• One study indicated that although clinicians often retrieved inaccurate information using Internet search engines, they believed the information was reliable enough to use in patient care and subsequently performed poorly in tests of clinical knowledge (Schwartz 2010).
• Many sources of evidence-based summaries also have weaknesses in how they report their content production procedures (Banzi 2010). If clinicians cannot trust these sources, or if the sources cannot be more transparent about their limitations, they may not be used.
• There is consistent evidence that clinicians struggle with databases such as PubMed and the Cochrane Library, which often contain reliable evidence, but are difficult for the uninitiated to use. Clinicians often fail to find answers in these databases. Time, awareness, skills and access are the key barriers (Shibuya 2013).

Information science results

• In the last 40 years (1973-2013) we have seen a huge increase in the quantity and quality of published mental health research (see table 1)

• Between 1973-2013, nearly 1 million (estimated 952,329) SRs, MAs and RCTs relevant to mental health were published worldwide. The proportion of high quality studies has steadily risen (see table 2)

• It is impractical for professionals to keep up to date with all of the high quality research relevant to their field. It was extremely difficult in 1973, but it’s completely impossible now (see table 3)

Conclusions

• The currency of clinicians’ knowledge degrades over time.
• Research papers are inaccessible to the majority of professionals, who don’t have the skills to appraise them or act upon their findings.
• It’s unfeasible for professionals to keep up to date with important and reliable mental health research by reading all of the full-text papers.
• There is clear and compelling evidence that what health professionals need is accessible, usable and reliable summaries of new research, policy and guidance.
• Blogs and social media are essential tools that can help all health and social care professionals find, use and discuss relevant evidence.

Limitations

The evidence in this review is based on a narrative synthesis, due to the complexity of the issues we addressed.

Many of the individual studies of clinical knowledge and behaviour are observational studies, which could be prone to selection bias and ascertainment bias. It could be that the most proactive clinicians are the ones who are most likely to use “new-fangled” evidence-based resources. However, this potential source of bias emphasises further the importance of the user-friendly approach adopted by the National Elf Service.

It is possible that our method of calculating the total worldwide number of published SRs and RCTs is an overestimate because the PubMed database from which we extrapolated our results may contain a greater proportion of these types of studies, compared with other databases.

We think that the balance of evidence is broadly supportive of our hypothesis that current, evidence-based summaries of important new research can help clinicians keep up to date and thereby improve patient care.

Summary

The vast majority of health and social care professionals lack the time and skills needed to find, read and interpret new research. By mobilizing a community of research-savvy bloggers, the National Elf Service is breaking down these barriers and opening up new avenues for disseminating the best available evidence to those who need it.

Feedback from our bloggers

But don’t just take our word for it! Here’s what some of our bloggers think:

“I never considered myself much of a ‘writer’ per se and even less of an ‘academic’ psychiatrist; teaching and clinical work being my clear interests. What blogging has allowed me to do (with the support and structure of The Mental Elf) is demystify wordy, academic pieces into useful, clinically applicable information for anyone to use.”
– Kirsten Lawson, Liaison Psychiatrist

“Blogging helps me think critically about what is written; not just when I blog, but whenever and whatever I read!”
– Mark Smith, IAPT therapist

They say you don’t really understand your subject until you teach it. Well, I’d say you don’t really understand a research article until you’ve blogged it. The discipline of writing a blog post that’s your own, that you can answer for, really cements your understanding of the research. As a non-clinical contributor, I have on occasion written things that people don’t like. But the research world thrives on discussion and debate. If you’re not a part of it, well, you need to ask yourself why not!
– Chris Sampson, Health Economist

It’s a daunting challenge, critically reviewing research. But an important one. Blogging for The Mental Elf has taught me how to read between the lines, and to discern good research practice from bad. For psychologists, this is nothing short of essential. Not only had it been a fantastic resource from a user perspective, it has been a great experience for me at the early stages of my career.
– Patrick Kennedy-Williams, Trainee Clinical Psychologist

As a non-academic and non-clinician, volunteering to blog for the Elf Service about dementia research for a mainly professional audience was very daunting.  But I was really grateful for the opportunity to contribute, and also to give a personal perspective as my mother has dementia.  I always find writing the blogs challenging (what if I get it wrong?), frustrating (how is this research ever going to change anything for the better?), but also very cathartic.
– Caroline Struthers, Carer

Links

Primary paper

Badenoch D, Tomlin A. (2015) Keeping up to date with reliable mental health research: National Elf Service white paper (PDF). Minervation Ltd, Oxford. DOI: 10.13140/RG.2.1.4404.2328

Other references

Anton B, Woodson SM et al. (2014) The persistence of clinical questions across shifts on an intensive care unit: an observational pilot study. J Med Libr Assoc. Jul 2014; 102(3): 201–205.

Badenoch D & De Brún C. (2011) Where’s the evidence for evidence? Review of abstracts of studies of clinicians’ information seeking behaviour. Newsletter of the International Society of Evidence-Based Health Care 2011;4:8-13

Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. (2010) A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010 Jul 7;12(3):e26.

Banzi R, Cinquini M, Liberati A et al. (2011) Speed of updating online evidence based point of care summaries: prospective cohort analysis. BMJ 2011;343:d5856.

Del Fiol G, Weber AI et al. (2014) Clinical questions raised by providers in the care of older adults: a prospective observational study. BMJ Open. 2014; 4(7): e005315.

Durieux, V. (2014) Finding scientific medical information through collaborative tagging systems: a task-oriented evaluation performed by healthcare professionals. 11^th European Conference of Medical and Health Libraries, 2014.

Ely JW, Osheroff JA, Chambliss MA et al. (2005) Answering physicians’ clinical questions: obstacles and potential solutions. JAMIA, 2005, 12(2), pp217-224.

Ely JW, Osheroff JA, Maviglia SM, Rosenbaum ME. (2007) Patient-Care Questions that Physicians Are Unable to Answer. JAMIA 2007;14:407-414.

Fraser Alan G, Dunstan Frank D. (2010) On the impossibility of being expert BMJ 2010; 341:c6815

González-González AI, Dawes M, Sánchez-Mateos J, Riesgo-Fuertes R. (2007) Information needs and information-seeking behavior of primary care physicians. Ann Fam Med. 2007 July; 5(4): 345–352.

Hersh WR. Hickam DH. (1998) How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review. JAMA. 280(15):1347-52, 1998 Oct 21.

Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. (2008) Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. Journal of Medical Internet Research, 2008, 10(4), e29. (2008b)

Kloda LA & Bartless JC. (2014) A characterization of clinical questions asked by rehabilitation therapists. J Med Libr Assoc. Apr 2014; 102(2): 69–77.

Magrabi F, Westbrook JI, Kidd MR, Day RO, Coiera E. (2008) Long-term patterns of online evidence retrieval use in general practice: a 12-month study. J Med Internet Res. 2008 Mar 19;10(1):e6.

Marcelo A, Gavino A et al. (2013) A comparison of the accuracy of clinical decisions based on full-text articles and on journal abstracts alone: a study among residents in a tertiary care hospital. Evid Based Med 2013;18:48-53 doi:10.1136/eb-2012-100537.

Osheroff JA, Forsythe DE et al. (1991) Physicians’ information needs: analysis of questions posed during clinical teaching. Ann Intern Med. 1991 Apr 1;114(7):576-81.

Sackett DL. The Need for EBM.  Centre for Evidence-Based Medicine, 1999.

Schwartz DG, Abbas J et al. (2010) Are internet searches a reliable source of information for answering residents’ clinical questions in the emergency room? IHI ’10 Proceedings of the 1st ACM International Health Informatics Symposium: 391-394.

Shibuya A, Inoue R et al. (2013) An Approach to Medical Knowledge Sharing in a Hospital Information System Using MCLink. Journal of medical systems 2013;37(4):1-9

Younger P. (2010) Internet-based information-seeking behaviour amongst doctors and nurses: a short review of the literature. Health Info Libr J. 2010 Mar;27(1):2-10.