Have you ever tried to compare the results of different clinical trials only to find that different outcome measures have been used in each and often in each study more than one outcome measure is reported. This is a common problem for systematic reviewers. When faced with more than one outcome measure which should be selected for analysis? Analysing the results of clinical trials where different outcome measures have been used can be like comparing apples and oranges.
In 1996 the Outcome Measures in Rheumatology (OMERACT) Conference agreed that measures of ‘pain’, ‘disability’ and ‘patient global assessment’ should be reported in future osteoarthritis (OA) trials. However it did not recommend which outcome measurement tools should be used to assess these. It was with interest that we noticed a newly published review that aimed to develop a prioritised list for extracting Patient Reported Outcome Measures (PROs) on ‘pain’ and ‘disability’ for meta-analyses.
Here’s what they did
The authors undertook a systematic search for Randomised Controlled Trials (RCTs) using two or more PROs to measure pain and/or disability in knee OA trials. Their search was restricted to studies published in the top ten highest impact factor journals of the previous five years in general and internal medicine and rheumatology.
The Patient Reported Outcomes (PROs) were ranked for responsiveness for ‘pain’ and ‘disability’ using Standardised Mean Difference (SMD).
SMD was estimated as the difference in mean change between intervention and control group divided by the pooled standard deviation (combining the different groups in any particular trial); the greater the SMD, the more responsive the measure. T he mean rank was used to estimate responsiveness across trials. PROs used in at least 5 trials were then listed according to mean rank. Subgroup and sensitivity analyses were applied to assess the robustness of findings.
Here’s what they found
Of the 402 publications identified only 38 studies were included. The most commonly used PROs were Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) subscales “pain” or “function”, pain scores using a visual analogue scale (VAS), or Short Form 36 (SF-36) “physical functioning”.
The final prioritised list for pain was:
- WOMAC pain subscale (Likert/100mm)
- Pain during activity (VAS)
- Pain during walking (VAS)
- Global knee pain (VAS)
- Pain at rest (VAS)
- SF-36 (bodily pain subscale)
- Health Assessment Questionnaire (pain subscale) Lequesne algofunctional index, Arthritis Impact Measurement Scale, Knee-Specific Pain Scale, McGill Pain Questionnaire, Arthritis Self Efficiency Scale, SES
- Pain at night (VAS), pain during activity (NRS), pain on walking (NRS), number of painful day (days)
- WOMAC subscale function (Likert/100mm)
- SF-36 (subscale physical function)
- Physical composite score (PCS) based on SF-36, SF-12 or SF-8
- HAQ (disability subscale), PDI (pain disability index), ASES (disability subscale)
The authors suggested including the Knee injury and Osteoarthritis Outcome Score (KOOS) which contains both the WOMAC ‘function’ and ‘pain’ subscales and has been shown to be equally responsive.
The Musculoskeletal Elf’s view
There is an extensive array of outcome measures available to researchers and clinicians which makes the decision as to which to use difficult. The notion of using a prioritised list is attractive as it may standardise reporting of results and simplify comparisons for systematic reviewers, clinicians and patients. However, the authors of this review note that choosing the most favourable patient-reported outcomes (PROs) from individual trials can overestimate the effect compared with a systematic approach. To reduce systematic review authors’ likelihood of biased selection of PROs in meta-analyses it is recommend using a prioritised list as presented in this study.
The recommendations of this review are based on PROs used in at least 5 trials, however, it must be noted that the popularity does not guarantee scientific robustness. Several systematic reviews including some by Team Musculoskeletal Elf note that few outcome measures have undergone any systematic method of scientific testing and gaps in measurement properties still exist for most.
- Have you found it difficult to compare the results of RCTs because different outcome measures were used?
- How useful are the results of RCTs to you? Do they make sense? Are they applicable?
- What about outcome measures?
Send us your views on this blog and become part of the Musculoskeletal Elf community.
- Juh C, Lund H, Roos E M, Zhang W, Christensen R. A hierarchy of Patient-Reported Outcomes for meta-analysis of knee osteoarthritis trials: empirical evidence from a survey of high impact journals, Arthritis 2012
- Gray H, Adefolarin AT & Howe TE 2011, A systematic review of instruments for the assessment of work-related psychosocial factors (Blue Flags) in individuals with non-specific low back pain, Manual Therapy, vol. 16(6), 531-543.
- Howe TE, Dawson LJ, Syme G, Duncan L, Reid J. Evaluation of outcome measures for use in clinical practice for adults with musculoskeletal conditions of the knee: a systematic review. Man Ther. 2012 Apr;17(2):100-18. Epub 2011 Aug 26.
- Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: An international initiative to improve outcome measurement in rheumatology, Trials. 2007 Nov 26;8:38
- Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC)
- Short Form 36
- Knee injury and Osteoarthritis Outcome Score (KOOS)