Since its inception, the National Institute of Health Research (NIHR), often described as ‘the research arm of the NHS’ has recognised the importance of involving patients and the public in the research it funds and supports.

This is founded on two main arguments:

• The moral or ethical argument: patients and the public (as citizens) have the right to be involved in research funded from the public purse and where they are potentially on the receiving end of the outcomes.
• The utilitarian argument: research quality is improved when patient and public perspectives are incorporated into the design and delivery of research studies, leading to less waste on research that has limited impact (Chalmers and Glasziou, 2009).

Despite the long history of patient and public involvement (PPI) in research (INVOLVE, as a national advisory group pre-dates the NIHR, being first set up by the Department of Health in 1996), there is limited evidence of what difference involvement makes to research; does it in fact improve quality, relevance and impact? However, the desire to measure the impact of PPI is not without its critics. Purtell and Wyatt (2011) have asked who this measurement is for and who will benefit from it; if involvement is a moral right, what is the justification for evaluating impact?

But if PPI is to become ‘the rule not the exception’ (Professor Dame Sally Davies, 2009) in health research, evidence is also needed to understand how to do it well; what works, for whom and in which context? Therefore, we can welcome the outcomes from the RAPPORT study, along with the other studies funded by NIHR Health Services and Delivery research programme in partnership with INVOLVE (Evans, 2014; Gamble, 2015).

The authors of the RAPPORT study state that PPI is a multifaceted, social process making it difficult to pinpoint impact within this complexity. This difficulty is compounded by a lack of robust and systematic reporting of PPI outcomes by the research community (making me wonder about the number of research reports summarised in this woodland that have reported on their PPI?).

Therefore, the study aimed to identify what PPI approaches apply across different domains (and which are context-specific) and whether different types of involvement achieve different outcomes.

By the way, RAPPORT is an acronym taken from the full study title: ReseArch with Patient and Public invOlvement: a RealisT evaluation (hmm…)

Methods

The study was underpinned by realist evaluation, adopted mixed methods and conducted in three stages.

The first two stages were a scoping study and survey of research registered on the UK Clinical Research Network portfolio. This register includes all NIHR funded research and other studies funded by research councils, industry, charities and other bodies (subject to an assessment of quality), although commercially funded studies were excluded from this study. A complex sampling frame (factors included recency of study, specified topics and geographical areas) resulted in 838 studies identified, of which 478 were included in the scoping work and 360 in the survey.

The scoping study (38% response rate) used a framework to assess the nature of PPI in studies working from documents either publicly available or provided by the research teams, while the survey (28% response rate) aimed to describe researchers’ experiences and perceptions of PPI outcomes, and benefits and challenges of PPI.

The third stage comprised in-depth, realist evaluation of 22 research projects to understand the context, mechanisms and outcomes in specific research settings. The primary method used was telephone interviews, with a total of 206 completed across the 22 case studies, including follow-up ‘tracking’ interviews. Interviewees included the PPI members, lead/senior researcher, co-applicants, researchers, clinical partners and sometimes PPI coordinators in the host organisation.

Results

Scoping study and survey

PPI was not found to be universal across the studies examined. The scoping work found evidence of PPI in 51% (n=92) studies, while the follow up survey found PPI in 79% (n=80) of studies. Funder stipulated requirements for PPI and the study design were the best predictors of involvement. The most common way that patients and the public were involved with research was through steering group membership and reviewing participant information leaflets.

Case studies

From the case studies, the authors developed a dynamic framework for PPI. They described it as dynamic because PPI developed and shifted with time and the research process; PPI in studies was rarely static. They identified three distinct approaches:

1. One-off model: involvement in a limited, researcher-identified task, usually by an established external PPI panel
2. Outreach model: regular points of contact between researchers and lay people, usually by PPI representatives acting as a bridge to the wider community
3. Fully intertwined model: researchers and lay people working alongside each other throughout the research process.

By applying Normalisation Process Theory, a model developed which had sufficient explanatory power to predict which case studies had PPI embedded as normal practice. The model included four work areas:

1. Coherence: making sense of PPI, for example, higher levels of agreement in the research team of the purpose of PPI usually led to more embedded PPI
2. Participation: relational work to build and sustain a community of practice for PPI, for example, time invested in social time for the lay people as well as involvement activity
3. Collective action: the operational work to enact PPI practices, for example, flexible approaches and regular communications
4. Reflexive monitoring: the appraisal work to evaluate PPI, for example, keeping track and recording the impact of PPI on the research.

Observable evidence of impact was difficult to find. However, the detailed examination of the range of research studies showed that case studies with the most embedded PPI were likely to demonstrate the greatest number of PPI-related outcomes. They were also more likely to demonstrate outcomes aligned with the moral perspective for involvement, such as the personal development for the people getting involved. Researchers limited the potential outcomes (read ‘positive benefit’) by adopting what could be viewed as the ‘minimally acceptable’ level of involvement; one-off involvement confined to a specific task.

Conclusions

The authors conclude that it is not a specific approach that leads to the effectiveness of PPI, but rather the achievement of six salient actions. These were:

1. Having a clear purpose, role and structure for PPI
2. Ensuring diversity
3. Whole research team engagement with PPI
4. Mutual understanding and trust between the researchers and lay representatives
5. Ensuring opportunities for PPI throughout the research process
6. Reflecting on, appraising and evaluating PPI within a research study.

However, the creation and sustenance of meaningful relationships between PPI representatives, specific researchers and the research team as a whole proved to be the most important enabling context for positive outcomes for PPI in research. This was a key finding in the RAPPORT study. The authors state:

Relational work appears to be the foundation stone for effective PPI and, while relationships are acknowledged to be part of the architecture of PPI, their central importance above other processes and resources has not been previously highlighted. These relationships are framed by trust, mutual respect and responses between researchers and lay representatives being framed by synergistic team membership.

Strengths and limitations

This is a large study, based on a research registry which includes a wide range of study types; funders, study designs, different stages in the ‘bench to bedside’ process. While the authors necessarily had to sample from this registry, they included a diverse range of studies. But we do have to note that studies/researchers still had to opt into the research so it’s likely that those holding largely positive attitudes towards PPI will have taken part in this study.

Another strength of this study is the extensive data collection, including interviews with people at the sharp end of PPI; the involved people. Insights straight from the horse’s mouth, as it were.

However, it’s a highly complex study, with extensive findings to report and digest (hence the rather long blog post!). And this may make the outcomes from the study difficult to discern and make use of. At present the authors have only published one paper covering the scoping work and survey (Mathie et al, 2014). I hope there are others in the pipeline, which will concisely report on the findings from the case studies, so these reach a wider audience.

Summary

The study provides us with good evidence of what works, for whom and in what context to achieve effective PPI. As someone who has worked hard to make sure people with lived experience of mental health problems play a key role in how research is prioritised, designed, delivered and disseminated, it’s great to have this evidence to support what I do.

Pertinent to my experience is the emphasis on relationships and these being the key enabling factor for effective PPI. In practice, it’s this aspect of PPI which is most often underestimated when working out how to do involvement and also how much time will be needed; that’s the time required for the PPI coordinator (whether a dedicated role or one of the research team) to build relationships and the time needed for relationships to develop. Sometimes it feels like chief investigators think involvement can be conjured up in a trifle (and often within days of the grant application deadline!).

It is also important to recognise the emotional aspect of this relational work. While it’s not a therapeutic relationship per se, the quality of the individual researcher; lay person relationship can be critical in enabling a person to get involved, especially when involvement is part of a person’s route to recovery.

Links

Primary paper

Wilson P, Mathie E, Keenan J, McNeilly E, Goodman C, Howe A, et al. ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study. Health Serv Deliv Res 2015;3(38)

Other references

Chalmers I, Glasziou P  (2009) Avoidable waste in the production and reporting of research evidence.  The Lancet, 374 (9683), 86-89 [Abstract]

Davies S,  Foreword in Staley, K (2009) Exploring Impact: public involvement in NHS, public health and social care research INVOLVE, Eastleigh

Evans D, Coad J, Cottrell K, Dalrymple J, Davies R, Donald C, et al. (2014) Public involvement in research: assessing impact through a realist evaluation. Health Serv Deliv Res, 2(36)

Gamble C, Dudley L, Allam A, Bell P, Buck D, Goodare H, et al. (2015) An evidence base to optimise methods for involving patient and public contributors in clinical trials: a mixed-methods study. Health Serv Deliv Res, 3(39)

Mathie E, Wilson P, Poland F, McNeilly E, Howe A, Staniszewska S, Cowe M, Munday D, Goodman C.  (2014) Consumer Involvement in health research: a UK scoping and survey International Journal of Consumer Studies;35-44 [Abstract]

Purtell, R. A. and Wyatt, K. M. (2011), Measuring something real and useful in consumer involvement in health and social care research. International Journal of Consumer Studies, 35: 605–608. [Abstract]

Photo credits

• Pete Markham CC BY 2.0
• Chip Griffin CC BY 2.0
• Kelly Teague CC BY 2.0
• Mark CC BY 2.0
• deux-chi CC BY 2.0
• Sebastiaan ter Burg CC BY 2.0