Patient-Reported Patient-Reported Outcomes Outcomes

Measurement GroupMeasurement Group

Elizabeth GibbonsElizabeth GibbonsSenior Research OfficerSenior Research Officer

elizabeth.gibbons@dphpc.ox.elizabeth.gibbons@dphpc.ox.ac.ukac.uk

http://phi.uhce.ox.ac.uk/http://phi.uhce.ox.ac.uk/

OverviewOverview

PROM Group OxfordPROM Group Oxford Outline of what PROMs areOutline of what PROMs are Criteria for selectionCriteria for selection ExamplesExamples UK contextUK context DiscussionDiscussion

History of groupHistory of group

DPHPCDPHPC Part of National Centre for Health Part of National Centre for Health

Outcomes Development Outcomes Development (NCHOD) and funded by the Department of Health through the NHS Information Centre for Health and Social Care.

Now core funding from DH

Web resourcesWeb resources

systematic systematic reviews of PROMs relevant to of PROMs relevant to specific disease and population groups specific disease and population groups

bibliographic database (up to 2005)bibliographic database (up to 2005) comprising comprising over 16000 records relating to PROMs with keyword over 16000 records relating to PROMs with keyword search facility search facility

general general informationinformation on instrument selection on instrument selection linkslinks to related websites including instruments, to related websites including instruments,

resources, organisations, research groups, and resources, organisations, research groups, and journals journals

information about the information about the Oxford Orthopaedic scoresOxford Orthopaedic scores including PDFs of the questionnaires and guides to including PDFs of the questionnaires and guides to their usage. their usage.

What are PROMs?What are PROMs?

Patient’s or public’s reports of healthPatient’s or public’s reports of health

Obtained by questionnaire:Obtained by questionnaire: Self-completed Self-completed InterviewInterview Computer terminalComputer terminal InternetInternet Palm held devicesPalm held devices Mobile phonesMobile phones

DimensionsDimensions Physical functionPhysical function SymptomsSymptoms Global judgments of healthGlobal judgments of health Psychological well-beingPsychological well-being Social well-beingSocial well-being Cognitive functionCognitive function Role activitiesRole activities Personal constructs (eg stigma, satisfaction with Personal constructs (eg stigma, satisfaction with

bodily appearance)bodily appearance) Satisfaction with careSatisfaction with care

Fitzpatrick R., Davey C, Buxton M., Jones D., Evaluating patient-based outcome Fitzpatrick R., Davey C, Buxton M., Jones D., Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment 1998; 2: (14)1-measures for use in clinical trials. Health Technology Assessment 1998; 2: (14)1-74.74.

Type of PROMType of PROM GenericGeneric

-SF-36-SF-36 Condition-specificCondition-specific

-Audit of Diabetes-Dependent Quality of -Audit of Diabetes-Dependent Quality of Life/ADDQoLLife/ADDQoL

Population-specificPopulation-specific-Child Health Questionnaire-Child Health Questionnaire

Dimension-specificDimension-specific -psychological well-being: Patient Health -psychological well-being: Patient Health

Questionnaire (PHQ-9)Questionnaire (PHQ-9) Utility measureUtility measure

-EQ-5D -EQ-5D Individualised measureIndividualised measure

-Patient Generated Index-Patient Generated Index

Generic vs SpecificGeneric vs Specific

Generic/utilityGeneric/utility

relevant to broader groups of relevant to broader groups of patientspatients

Condition-specificCondition-specific

more sensitive to specific problemsmore sensitive to specific problems

Commonly recommended to use both Commonly recommended to use both

What are patient-What are patient-reported outcome reported outcome

measures for? measures for?

Individual patient careIndividual patient care~ ~ ?? Disease-specific Disease-specific

Clinical effectiveness-RCTsClinical effectiveness-RCTs~ generic and/or disease-specific~ generic and/or disease-specific

Cost effectiveness / cost utilityCost effectiveness / cost utility~EQ-5D utility value to construct a QALY~EQ-5D utility value to construct a QALY

Evaluation of services~ Evaluation of services~ ??

Patient-reported Patient-reported outcome measuresoutcome measures

Systematic search of 11 major databasesSystematic search of 11 major databases

1275 separate instruments in 20021275 separate instruments in 2002

3215 separate instruments in 20073215 separate instruments in 2007

Evaluation of Evaluation of bibliographybibliography

0500

1000150020002500

Rheu/

mus

c

Cance

r

Men

tal h

ealth

Neuro

logica

l

Elderly

Paed/

adol

Cardio

vasc

ular

Respir

ator

y

Diabet

es

Population or health condition

Cu

mm

ula

tive

nu

mb

er

of

rec

ord

s

1999

2005

Instrument type 1999 & Instrument type 1999 & 20052005

Number of records per instrument type 1999 & 2005

0100020003000400050006000700080009000

Generic Specific Utility Dimension-specific

Individualised

Instrument type

Nu

mb

er o

f re

cord

s

1999

2005

Choosing an instrumentChoosing an instrument

A potentially complex taskA potentially complex task Structured reviews of evidenceStructured reviews of evidence Selection criteriaSelection criteria Web-based advice: Web-based advice:

http://phi.uhce.ox.ac.uk/http://phi.uhce.ox.ac.uk/http://www.proqolid.org/http://www.proqolid.org/

NHS Guidance: NHS Guidance: currently currently PROMs for PROMs for elective procedures in acute serviceselective procedures in acute services

Structured reviewsStructured reviews

Chronic conditions (2006; 2009)Chronic conditions (2006; 2009) Breast and prostate cancerBreast and prostate cancer KidneyKidney ChildrenChildren Mental healthMental health Elective proceduresElective procedures

Review methodologyReview methodology Fitzpatrick R., Davey C, Buxton M., Jones D., Fitzpatrick R., Davey C, Buxton M., Jones D.,

Evaluating patient-based outcome measures for Evaluating patient-based outcome measures for use in clinical trials. Health Technology use in clinical trials. Health Technology Assessment 1998; 2: (14)1-74.Assessment 1998; 2: (14)1-74.

LSHTM (2004) LSHTM (2004) Patient-Reported Outcome Measures (PROMs) for routine use in Treatment Centres:recommendations based on a review of the scientific evidence.

Terwee (2007) Quality criteria were proposed Terwee (2007) Quality criteria were proposed for measurement properties of health status for measurement properties of health status questionnaires. questionnaires. Journal of Clinical Journal of Clinical EpidemiologyEpidemiology 60 (34-42) 60 (34-42)

Methods for establishing Methods for establishing a PROMa PROM

• Clarify goal of proposed instrumentClarify goal of proposed instrument• Generate items from patient-Generate items from patient-

oriented interviews, focus groupsoriented interviews, focus groups• Item selection Item selection • Formatting and testing of format of Formatting and testing of format of

instrumentinstrument• Studies to examine measurement Studies to examine measurement

propertiesproperties

Measurement propertiesMeasurement properties

Reliability:Reliability: Does the instrument Does the instrument produce results that are produce results that are reproducible and internally reproducible and internally consistent?consistent?

ValidityValidity:: Does the instrument Does the instrument measure what it claims to measure?measure what it claims to measure?

Content validity: Content validity: Qualitative Qualitative evidenceevidence

Measurement propertiesMeasurement properties

Construct validityConstruct validity:: High correlations between the scale High correlations between the scale

and relevant constructs preferably and relevant constructs preferably based on a priori hypothesis with based on a priori hypothesis with predicted strength of correlationpredicted strength of correlation

Statistically significant differences Statistically significant differences between known groups and/or a between known groups and/or a difference of expected magnitudedifference of expected magnitude

Measurement propertiesMeasurement properties

Responsiveness: Responsiveness: Does the instrument Does the instrument detect changes over time that matter to detect changes over time that matter to patients?patients?

~ Statistically significant changes on ~ Statistically significant changes on scores from pre to post-treatment and/or scores from pre to post-treatment and/or difference of expected magnitudedifference of expected magnitude

Precision Precision How precise are the scores of How precise are the scores of the instrument?the instrument?

~ Floor/ceiling effects for summary scores ~ Floor/ceiling effects for summary scores <15% <15%

Measurement propertiesMeasurement properties

InterpretabilityInterpretability

How interpretable are the scores of How interpretable are the scores of the instrument?the instrument?

Clinically important difference- Clinically important difference- minimally important differenceminimally important difference

Operational Operational characteristicscharacteristics

Acceptability:Acceptability: Low levels of Low levels of incomplete data or non-responseincomplete data or non-response

Feasibility/burdenFeasibility/burden:: Reasonable Reasonable time and resources to collect, time and resources to collect, process and analyse the dataprocess and analyse the data

AppraisalAppraisal

Rate quality of evidence for each Rate quality of evidence for each instrument identified in relation to instrument identified in relation to each of key criteriaeach of key criteria

Reliability, validity, responsiveness, Reliability, validity, responsiveness, interpretability, acceptability, interpretability, acceptability, feasibility feasibility

AppraisalAppraisal

Each aspect for each instrument Each aspect for each instrument rated on 5 point scale:rated on 5 point scale:

00 No evidence No evidence _ Negative evidence_ Negative evidence+ Limited evidence in favour+ Limited evidence in favour++ Some good evidence in favour++ Some good evidence in favour+++ Good evidence in favour +++ Good evidence in favour

ExamplesExamples

Patient Health Questionnaire Patient Health Questionnaire (PHQ-9)- (PHQ-9)- QOF-DEP2QOF-DEP2

Audit of Diabetes-Dependant Audit of Diabetes-Dependant Quality Of Life (ADDQoL)Quality Of Life (ADDQoL)

Euroqol-EQ-5DEuroqol-EQ-5D

Scoring PHQScoring PHQ

Scores range from 0 to 27 with a three Scores range from 0 to 27 with a three point scale for the 9 items. point scale for the 9 items.

Mild depression is considered with Mild depression is considered with scores of 5 to 9scores of 5 to 9

Moderate for scores between 10 and 14Moderate for scores between 10 and 14 Severe, 20 to 27 Severe, 20 to 27

(Spitzer et al., 1999). (Spitzer et al., 1999).

Views of patients and Views of patients and GPsGPs

Routine implementation of measures of the Routine implementation of measures of the severity of depression (PHQ, HADS, BDI) severity of depression (PHQ, HADS, BDI)

GPs were cautious about the validity and utility and doubtful about GPs were cautious about the validity and utility and doubtful about the motives behind their implementation. GPs considered that the motives behind their implementation. GPs considered that clinical experience was more important in identifying cases and clinical experience was more important in identifying cases and assessment of the severity of depression. Objective measurement assessment of the severity of depression. Objective measurement reduced the humanistic aspect of the consultation as well as de-reduced the humanistic aspect of the consultation as well as de-skilling doctorsskilling doctors

Patients on the other hand were positive about the utility of the Patients on the other hand were positive about the utility of the measures and viewed the process as a structured adjunct to the measures and viewed the process as a structured adjunct to the consultation. GPs were more attentive to their problems. Some consultation. GPs were more attentive to their problems. Some patients reported that the questionnaires were useful in helping patients reported that the questionnaires were useful in helping them think about their depression and enable them to express them think about their depression and enable them to express themselves better. themselves better.

(Dowrick et al., 2009)(Dowrick et al., 2009)

OthersOthers

ADDQoL-conceptual issuesADDQoL-conceptual issues

EQ-5DEQ-5D

Brevity vs comprehensiveness of itemsBrevity vs comprehensiveness of items

UK DH PolicyUK DH Policy Lord Darzi’s Interim Report on the future of the Lord Darzi’s Interim Report on the future of the

NHS recommends that (PROMs) should have a NHS recommends that (PROMs) should have a greater role in the NHS. ‘greater role in the NHS. ‘It could be useful to It could be useful to build on recent advances in measuring build on recent advances in measuring outcomes as assessed by patients themselves, outcomes as assessed by patients themselves, and make these patient-reported outcome and make these patient-reported outcome measures a stronger part of the approach to measures a stronger part of the approach to quality’.quality’.

The new Standard NHS Contract for Acute The new Standard NHS Contract for Acute Services includes a requirement to report on Services includes a requirement to report on evidence from PROMs from April 2009. evidence from PROMs from April 2009.

Current policyCurrent policy

For the first-time For the first-time payments to NHS payments to NHS hospitals will be hospitals will be adjusted adjusted according to according to patient patient satisfaction and satisfaction and health outcomes.health outcomes.

LTC Pilot studyLTC Pilot study

Pilot evaluation of the feasibility of Pilot evaluation of the feasibility of collecting PROM data in primary carecollecting PROM data in primary care

Six LTCs: asthma, COPD, diabetes, Six LTCs: asthma, COPD, diabetes, epilepsy, heart failure, strokeepilepsy, heart failure, stroke

ReviewsReviews

Postal survey Postal survey

Interviews with non-respondersInterviews with non-responders

Interviews with GPs. commissionersInterviews with GPs. commissioners

Impact of PROMs: the Impact of PROMs: the commissioners of commissioners of

servicesservices PROMs in the context of routine measuring of PROMs in the context of routine measuring of quality of NHS trustsquality of NHS trusts

Will raise an number of interesting Will raise an number of interesting methodological challenges if they are to inform methodological challenges if they are to inform decisions about qualitydecisions about quality

PROMs less clearly understood outside the PROMs less clearly understood outside the context of research-based RCTs where role is context of research-based RCTs where role is clearerclearer

The OfficeThe Office

Department of Public HealthDepartment of Public Health