The quality and outcomes framework (QOF) was introduced in the NHS in 2004 as part of the new General Medical Services contract (Health and Social Care Information Centre, 2016). Its purpose was to improve the quality of NHS primary care clinical services and also to retain general practitioners (GPs).

It has been more costly than predicted but arguably the NHS has retained and even increased the number of GPs. If one looks at the high level of attainment of these QOF indicators among GP practices you could say that the quality of clinical care is better insofar as the indicators themselves measure it.  The real question for someone like me is whether it makes any difference at all to the medium or long term outcomes of people with mental health problems.

Full disclosure: while I do understand the need for indicators I am always very skeptical about the way they are selected, particularly in mental health. I am also very suspicious of anything that looks like a process indicator (having a care plan, having an annual review) rather than what I consider a true hard outcome (being back at work, ability to live without support, the presence of comorbidities, mortality).

Trying to get some clarity on this issue I came across a paper by Nils Gutacker and colleagues from the Centre for Health Economics at University of York (Gutacker et al, 2015). They looked at the relationship between QOF attainment in severe mental illness (SMI) and hospital admissions. They used retrospective analysis of routine data at GP practice level estimating the effect of four QOF indicators on psychiatric admissions to hospital using random effects Poisson regression. Their hypothesis (looking at results from similar studies in other chronic conditions) was that admissions will be reduced for practices that perform better on these measures. I’ll mention some details of the design that I’ll comment on later.

Methods

Sample

The dataset included all GP practices in the English NHS between 2006 and 2011. The authors excluded practices with less than 1,000 registered patients and practices with fewer than 5 patients with SMI, but they did not apply this exclusion criterion to bipolar affective disorder as the numbers are very small. They also excluded practices reporting inconsistent numbers of people with SMI.

Data sources

They used the QOF data set and multiple other sources for the SMI statistics at practice level. More interestingly they only used the Hospital Episodes Statistics (HES) to calculate admission rates.

QOF indicators used

They used these two indicators for all individuals with SMI:

1. MH6: ‘The percentage of patients on the register who have a comprehensive care plan documented in the records agreed between individuals, their family and/or carers as appropriate.’
2. MH9: ‘The percentage of patients with schizophrenia, bipolar affective disorder and other psychoses with a review recorded in the preceding 15 months. In the review there should be evidence that the patient has been offered routine health promotion and prevention advice appropriate to their age, gender and health status.’

They used 2 others for those individuals on Lithium (which the authors thought might only apply to those with Bipolar Affective Disorder).

Exception reporting

The authors went to great lengths to properly adjudicate and control exception reporting (an allowable practice under QOF to enable practice to exclude registered patients that refuse to engage) as it could be potentially exploited by practices.

Covariates

The authors controlled for various known confounders (socio-economic factors, access to care, etc) and used pre-study data to control for unobserved practice-specific covariates.

Analysis

They used random effects Poisson regression models to relate psychiatric admissions per practice to QOF achievement, conditioning on potential confounding factors and a normally distributed GP practice random effect with zero mean and constant variance. I thought the assumptions were reasonable and the researchers conducted a great number of robustness checks for their assumptions.

Results

They had in their sample 8,234 GP practices that treated people with SMI during the 5-year period (38,774 practice-year observations; mean follow-up 4.8 years). The median number of people with SMI per practice was 39 (interquartile range (IQR)= 22 to 64) and the median number of people with bipolar disorder was 6 (IQR= 3 to 10). The median number of annual admissions per practice was 3.5 (IQR= 1 to 5) for SMI, and 1.1 (IQR= 0 to 2) for bipolar disorder.

For the main analysis there was a consistent positive association between QOF achievement rates and hospital admissions for all indicators apart from MH6 (documented comprehensive care plan). For MH9 (annual review), an additional 1% in achievement rates was associated with an average increase in the practice admission rate of 0.19%.

The researchers did various sensitivity analyses (including looking at the effect of exception reporting) and the results remained essentially the same.

Conclusions

Comments on design

The exclusion criterion for number of people with SMI would easily exclude practices that actively avoid having to deal with individuals with SMI, which is something that I have personally encountered many times. However this would not necessarily affect the research question at all, as they would not be engaging in QOF.

There was an assumption that the Lithium indicators would only apply to individuals with bipolar affective disorder, however Lithium is also used in schizoaffective disorder.

One of the most surprising things is that the way they defined SMI was essentially schizophrenia and bipolar affective disorder. They did not include severe depression with or without psychotic symptoms or any of the more severe and disabling forms of anxiety disorder. The number of individuals with schizophrenia-like disorders in their sample was around 7 times greater than the number of people with bipolar affective disorder, even though schizophrenia affects 1% of the population compared to 2.5% for bipolar affective disorder. This makes me wonder how accurate was the coding system they relied on and whether the sample really contained individuals with SMI as defined by the authors.

Using HES excludes all patients who are inpatients at independent healthcare provider hospitals who would be registered in the community with a GP and attending the surgery. The GPs would be reporting on them for QOF, but they would not be at risk of admission to hospital as they would be inpatients already. Also residing in a specialised mental health care home was not included anywhere as a covariate.

Weaknesses identified by the authors

Admission is not necessarily an adverse outcome. Annual reviews and having a care plan may prompt closer monitoring and a better identification of individuals at risk leading to admission rather than a very poor community outcome such as severe self-neglect, self-harm, violence, etc.

Individuals with difficult to manage SMI would gravitate to those practices that take achieving QOFs in SMI more seriously and provide a better service to these individuals. This again is true in my personal experience; patients do leave GPs that they perceive as incompetent or unsympathetic to their problem. I have encouraged them at times to do precisely this when it was clear they had a bad experience and I knew a more mental health friendly GP.

Are QOFs worth having from the patient perspective?

This is a worthwhile study attempting to answer an important question. I have problems with their definition of SMI and I am unsure about what their sample actually contained. The QOFs included where clearly process ones (care plan and annual review – I am not including the Lithium-related ones as they are so narrow and not relevant to even all individuals with bipolar affective disorder).

Are these QOFs worth having from the patient perspective? I believe they are for the following reasons:

• QOF achievement went up and exception reporting went down during this study which means to me that GPs are progressively taking a greater interest in people with SMI 

• Admissions went up in those paying more attention to people with SMI. Having experienced the at times insurmountable barriers some individuals experience when accessing inpatient care I believe this is just those GPs being better advocates for their patients to access what is becoming a staggeringly scarce resource

Are QOFs likely in themselves to improve outcomes?

That is far less clear. I am worried about focusing on avoiding admission as a goal for QOF, I can easily see that leading to perverse incentives and unintended consequences. I think other outcome measures such as functioning, quality of life and independent living would be far more useful than avoiding admissions.

Links

Primary paper

Gutacker N, Mason AR, Kendrick T, et al. (2015) Does the quality and outcomes framework reduce psychiatric admissions in people with serious mental illness? A regression analysis. BMJ Open 2015;5: e007342. doi:10.1136/bmjopen-2014-007342

Other references

Health and Social Care Information Centre (2016) Quality and Outcomes Framework. Website last accessed, 5 Jan 2016.

Photo credits

• Andrew Bowden CC BY 2.0
• Ilmicrofono Oggiono CC BY 2.0
• That Guy DouG CC BY 2.0
• Darron Birgenheier CC BY 2.0
• Kate Ter Haar CC BY 2.0