Patient-Reported Questionnaires in Clinical Practice: Just Another Laboratory Test David Cella, Ph.D. Professor and Chair Department of Medical Social

Patient-Reported Questionnaires in Clinical Practice: Just Another Laboratory Test

David Cella, Ph.D.Professor and Chair

Department of Medical Social SciencesFeinberg School of Medicine

Health Reform 2010

• Major Commitment of the Obama Administration

• Sweet Spot:• Right treatment to the right people• Avoid unnecessary / inappropriate

treatments• Universally available and affordable

• Chronic diseases• Major expense• Rarely “cured”• Most important outcome is often quality of

life

Health care providers do not routinely use health-related quality of life (HRQL) data to guide diagnosis, treatment, or performance improvement.

…if they did, would it make a difference?

…how should HRQL data be presented?

The Problem

Measurement Science: Not the Problem

In chronic illness care, the problem is not the lack of HRQL measures • a large selection of generic and

disease-specific measures exist

The problem is the unhelpful way the data are presented to clinicians

5

Continuum of Disease-specific and Generic Health Measures

5

Clinical Markers

Specific Symptoms

Impact of Disease-specific

Problems

Generic Functioning,

Well-being and Evaluation

Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995

(1) (2) (3) (4)

66

Clinical Markers

Specific Symptoms


Problems




(1) (2) (3) (4)

dd

Spirometry

Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all

Shortness of Breath


77

Clinical Markers

Specific Symptoms


Problems




(1) (2) (3) (4)

dd


How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely

Spirometry Shortness of Breath

Respiratory-specific


88

Clinical Markers

Specific Symptoms


Problems




(1) (2) (3) (4)

dd


How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely

In general, would you say your health is…

Excellent Very good Good Fair Poor

Spirometry Shortness of Breath

Respiratory-specific

Generic


Can Patient Reported Outcomes (PROs) be Treated as Just Another

Lab Result?

9

10

What Do We Need to Launch a Useful Clinical PRO Lab

• Outcomes that matter to patients and providers

• Practical (brief) and inexpensive• Covers a range, including normal• Greater precision• Comparability of scores• Ease of interpretation• Reference values / normal ranges

SyMon-L Symptoms Tracked• Pain• Fatigue (lack of energy)• Shortness of breath• Coughing• Chest tightness• Difficulty breathing• Nausea• Poor appetite• Weight loss• Bothered by treatment side effects• Emotional distress• Cognitive dysfunction• Dissatisfaction with HRQL

Symptom Summary Report

Results: Clinical intervention responses to symptom alerts

Types of intervention N %

Medical/clinical (e.g., medications, MD appt) 233 20%

Education 420 36%

Support, coping 455 39%

Coordination of care (e.g., referrals) 50 4%

Acuity of Interventions

Critical (e.g., PE, ER visit, admission) 22 3%

Important (e.g., change in meds, clinic visit, IV therapy) 103 16%

Standard of care (e.g., reassessment, education) 374 59%

Non-essential (e.g., follow-up) 98 15%

System-related (e.g., study questions) 37 6%

Symptom Monitoring Report

QoL predicts survival in NSCLCStudy What predicts?

Stanley (1980) Initial PS

Kaasa, Mastekaasa & Lund (1989) General sx (pain, fatigue)Psychosocial well being

Ganz et al. (1991) Overall QoL

Loprinzi et al. (1994) Physician-rated PSPatient-rated PS

Ruckdeschel (1994) Patient-reported QOL

Herndon et al. (1999) Pain

Montazeri et al. (2001) Pre-diagnosis overall QoL

Eton et al (2003) Patient-reported QoL

Symptoms(LCS)

Emotional

Physical

Social

Functional

Quality of Life FACT-L

Baseline patient characteristics predicting survival

Order of entry

FACT TOIMetastatic symptomsECOG PSStage (IIIB vs IV)Paclitaxel (Arm A vs B/C)

Risk ratio*

0.981.441.571.390.81

p value

<0.001<0.001<0.0010.0160.039

*Risk of deathOther possible explanatory variables included age, sex, g-csf, weight loss, disease symptoms, systemic symptoms, comorbidity and other FACT subscales

<58, no

<58, yes

>58, no

>58, yes

Initial, improved?

Proportion surviving

1.0

0.8

0.6

0.4

0.2

0.00 1 2 3 4 5 6 7 8 9 10 11 12

Initial TOI and improvement at 6 weeks* (n=352)

*Patients with missing QoL excluded

Eton et al, JCO, 2003

LCS Mean Change By WeekBy Objective Response, Trial 39

PRSDPD

LCS

Sco

re C

hang

e

-8

-6

-4

-2

0

2

4

6

8

10

12

14

16

Weeks from Randomization0 2 4 6 8 10 12 14 16 18 20 22 24 26

Survival by Disease-related Symptom ImprovementSubset surviving > 8 weeks

At Risk Failures Medianin Months

Yes 84 35 .No 100 85 4.8

Prob

abili

ty

0.0

0.2

0.4

0.6

0.8

1.0

Months from Randomization

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Survival by Response & Symptom ImprovementSubset surviving > 8 weeks


PR 22 3 .LCS imp. 84 35 .Other 99 85 4.8

Prob

abili

ty

0.0

0.2

0.4

0.6

0.8

1.0


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Survival by Symptom Improvement in SD SubsetSubset surviving > 8 weeks *

* All patients with an objective response of SD survived > 8 weeks.


SD + LCS imp. 44 19 .SD 62 32 9.4SD - LCS imp. 18 13 4.6

Prob

abili

ty

0.0

0.2

0.4

0.6

0.8

1.0


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

23

Content of Widely-Used Patient-Reported Outcome Measures

Source: Adapted from Ware, 1995

Reported health transition l l l

SIP = Sickness Impact Profile (1976)HIE = Health Insurance Experiment surveys (1979)NHP = Nottingham Health Profile (1980)QLI = Quality of Life Index (1981)COOP = Dartmouth Function Charts (1987)DUKE = Duke Health Profile (1990)MOS FWBP = MOS Functioning and Well-Being

Profile (1992)

MOS SF-36 = MOS 36-Item Short-Form Health Survey (1992)

QWB = Quality of Well-Being Scale (1973)EUROQOL = European Quality of Life Index (1990)HUI = Health Utility Index (1996)SF-6D = SF-36 Utility Index (Brazier, 2002)

Psychometric Utility RelatedSIP HIE NHP COOP DUKE MOS

FWBPMOSSF-36

QWB EURO-QOL

HUI SF-6DCONCEPTS

Physical functioning l l l l l l l l l l lSocial functioning l l l l l l l l l lRole functioning l l l l l l l l l lPsychological distress l l l l l l l l l lHealth perceptions (general) l l l l l lPain (bodily) l l l l l l l l lEnergy/fatigue l l l l l l lPsychological well-being l l l lSleep l l l lCognitive functioning l l l lQuality of life l l l

PROMIS = Patient Reported Outcomes Measurement Information System

= Quality of Well-Being Scale (1973)

PROMIS

llll

ll

l

PROMIS integrates the fields of…

InformationTechnology

PROMIS

QualitativeResearch

SurveyResearch

Psychometrics

Psycho-metricTesting

Item Bank(IRT-calibrated items reviewed forreliability, validity, and sensitivity)

0.0

0.5

1.0

1.5

2.0

2.5

-3 -2 -1 0 1 2 3

Theta

Info

rmati

on

0.0

0.2

0.4

0.6

0.8

1.0

-3 -2 -1 0 1 2 3

Theta

Pro

ba

bil

ity o

f R

esp

on

se

Short FormInstruments

CAT

Items fromInstrument

A

Item Pool


B


C

NewItems

Questionnaireadministered to largerepresentative sample

SecondaryData Analysis

CognitiveTesting

FocusGroups

Content ExpertReview

Self-reported Health

Mental Health

Physical Health

Social Health

Physical Function*

Symptoms

Emotional Distress

Cognitive Function

Social Function

Positive Psychological

Function

Social Relationships

Upper Extremities: dexterity

Lower Extremities: mobility

Central: neck and back

Activities: Instrumental Activities of Daily Living

Pain

Fatigue*

Other

Quality

Behavior

Impact*

Experience

Impact

Anxiety*

Depression*

Anger

Substance Abuse

Negative Impact of Illness

Alcohol Abuse

Positive Impact of Illness

Mastery & Control (self-efficacy)

Subjective Well-Being (positive affect)

Ability to Participate*

Satisfaction with Participation*

SatisfactionSatisfaction

Satisfaction

Satisfaction

Meaning & Spirituality

Stress Response

Self-concept

Sleep/Wake Function*

Sexual Function

Social Roles

Discretionary Social Activities

Social Roles

Discretionary Social Activities

Meaning & Spirituality

Coping

Self-concept

Social Connection

Social Isolation

Social Isolation

Quantity of Social Support (Integration)

Quality of Social SupportInstrumental/Informational

Companionship

Emotional Support

Sleep Disturbance

Wake Disturbance

CaPS bank in development

PROMIS v1.0 bank

PROMIS area tested but no bank developed for v1.0

Area addressed (in part) by bank within lineage

Area not addressed yet

* = Additional cancer-specific PROMIS bank

PROMIS Domain Framework

27

What Do We Need to Launch a Useful

Clinical PRO Lab?



PROMIS Profile Short Forms

28* reduced set (the full bank has 112 items) was used for real data simulation purposes

Anxiety29

Depression28

Fatigue95

Pain Impact41

Sleep Disturbance

27

Physical Function86*

Social Role14

46

8

Mental

Physical

Social

PROMIS Profile Short Forms

29

PROMIS Profile Subscales: Test Information Functions

Anxiety

T-score

rel.=.90

rel.=.95

0

5

10

15

20

25

10 30 50 70 90

Depression

T-score

rel.=.90

rel.=.95

0

10

20

30

10 30 50 70 90

Fatigue

T-score

rel.=.90

rel.=.95

0

10

20

30

10 30 50 70 90

Pain Impact

T-score

rel.=.90

rel.=.95

0

10

20

30

40

50

60

10 30 50 70 90

Physical Function

T-score

rel.=.90

rel.=.95

0

10

20

30

40

10 30 50 70 90

Sleep Disturbance

T-score

rel.=.90

rel.=.95

0

5

10

15

10 30 50 70 90

Social Role

T-score

rel.=.90

rel.=.95

0

10

20

30

40

10 30 50 70 90

SF 8SF 6SF 4

30

PROMIS Profile Subscales: Correlations with Full Banks

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Anxiety

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Depression

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Fatigue

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Pain Impact

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Physical Function

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Sleep Disturbance

Length

Cor

rela

tion

1 2 3 4 5 6 7 8

0.80

0.85

0.90

0.95

1.00

Social Role

Length

Cor

rela

tion

32

“Ceiling Effect”




Clinical PRO Lab?

PROMIS-57 Profile by General Health Rating:In general, would you say your health is:

33

3040

5060

7080

Full Bank

T-S

core

8070

6050

4030

AN

X

DE

P

FA

T

PA

I

SLP

PH

F

SO

R

PoorFairGoodVery goodExcellent

3040

5060

7080

SF - 8 Items

In general, w ould you say your health is:

T-S

core

8070

6050

4030

AN

X

DE

P

FA

T

PA

I

SLP

PH

F

SO

R


3040

5060

7080

SF - 6 Items

T-S

core

8070

6050

4030

AN

X

DE

P

FA

T

PA

I

SLP

PH

F

SO

R


3040

5060

7080

SF - 4 Items

T-S

core

8070

6050

4030

AN

X

DE

P

FA

T

PA

I

SLP

PH

F

SO

R


PROMIScore - PSF

34

35



What Do We Need to Launch a Useful Clinical PRO Lab?

g

0

0.1

0.2

0.3

0.4

0.5

0.6

-2.5 -1.5 -0.5 0.5 1.5 2.5

Computerized Adaptive Testing (CAT)

• Estimates location (severity; capability) of a person on a domain (concept) by selecting questions based on that person’s prior answers

• Iteratively estimates a person’s standing on the domain (e.g., depressive symptoms) and administers only the most informative items, achieving precision with a minimum possible number of questions.

Beginning of CAT

01

23

45

Item 15

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=56

20 40 60 80

T-Score

Po

ste

rio

r D

istr

ibu

tion

20 30 40 50 60 70 80

T-Score = 50 SE = 10

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15

T-Score: 50 SEM: 10

Best Item-I felt depressed

0.0

0.2

0.4

0.6

0.8

1.0

Item 15

T-Score

Pro

ba

bili

ty

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Po

ste

rio

r D

istr

ibu

tion

20 30 40 50 60 70 80

I felt depressed1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 52 SE = 4

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15

T-Score: 52 SEM: 4 01

23

45

Item 10

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=57

Next Best Item-I felt like a failure

0.0

0.2

0.4

0.6

0.8

1.0

Item 10

T-Score

Pro

ba

bili

ty

20 30 40 50 60 70 80

12345

I felt like a failure1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 53 SE = 3

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10


23

45

Item 1

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=59

Next Best Item-I felt worthless

0.0

0.2

0.4

0.6

0.8

1.0

Item 1

T-Score

Pro

ba

bili

ty

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Po

ste

rio

r D

istr

ibu

tion

20 30 40 50 60 70 80

I felt worthless1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 55 SE = 2

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10,1


23

45

Item 3

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=58

Next Best Item-I felt helpless

0.0

0.2

0.4

0.6

0.8

1.0

Item 3

T-Score

Pro

ba

bili

ty

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Po

ste

rio

r D

istr

ibu

tion

20 30 40 50 60 70 80

I felt that nothing could cheer me up1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 55 SE = 2

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10,1,3,21,2,5

T-Score: 55 SEM: 2

20 40 60 80

T-Score

Po

ste

rio

r D

istr

ibu

tion

20 30 40 50 60 70 80

IRT expands the range of what we can measure

Rose et al, J Clin Epidemiol 2007 (accepted)

SE = 0.32rel = 0.90

SE = 0.22rel = 0.95

0

0.1

0.2

0.3

0.4

0.5

0.6

0 10 20 30 40 50 60 70 80

SF-36 items

CAT 10 itemsFull Item Bank

measurement precision (standard error)

normed theta values

HAQ items

SF-12 items

representativesample

rheumatoid arthritis patients

US-Representative Sample

0.4

0.5

0.3

0.2

0.1

-4 -3 -2 -1 0 +1 +2

PROMIS Fatigue Short Form

Garcia SF et al, J Clin Onc, 2007. © 2007 Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group

In the past 7 days … Never Rarely Some-times

Often Always

FATEXP 20

How often did you feel tired? 1

2

3

4

5

FATEXP 5

How often did you experience extreme exhaustion? 1

2

3

4

5

FATEXP 18

How often did you run out of energy? 1

2

3

4

5

FATIMP 33

How often did your fatigue limit you at work (include work at home)?

1

2

3

4

5

FATIMP 30

How often were you too tired to think clearly? 1

2

3

4

5

FATIMP 21

How often were you too tired to take a bath or shower? 1

2

3

4

5

FATIMP 40

How often did you have enough energy to exercise strenuously?

1

2

3

4

5

0

0.1

0.2

0.3

0.4

0.5

0.6

-2.5 -1.5 -0.5 0.5 1.5 2.5

7-item Short-form7-item CAT

98-item Bank

No Fatigue Severe Fatigue

SE=0.32 (r=0.90)

SE=0.22 (r=0.95)

Comparison of Measurement PrecisionFull-length Item Bank vs. Legacy vs. CAT vs. Short-form

Sta

ndar

d E

rror

45

CAT

Assess Health Dynamically When Necessary

Patient scores

here

CAT = Computerized Adaptive Testing

46




Clinical PRO Lab?

47

We Need the Health Equivalent of a

Two-Sided Tape Measure

52 centimeters = 20.5 inches

…and easy-to-use conversion tables with reference data and action thresholds

PROMIS Depression Bank and CES-D

Depression

CE

SD

-2 -1 0 1 2 3 4

01

02

03

04

05

06

0

r =0.84

-2 -1 0 1 2 3 4

08

0

Depression

Depression

CE

SD

-2 -1 0 1 2 3 4

01

02

03

04

05

06

0

01

02

03

04

05

06

0

0 100 250

CE

SD

Fatigue

• Experience and Impact• 95-item bank

• Legacy Instruments• SF-36 Vitality Scale (4 items)

• FACIT-Fatigue Scale (13 items)

0

0.1

0.2

0.3

0.4

0.5

0.6

-2.5 -1.5 -0.5 0.5 1.5 2.5

4-item SF36/Vitality4-item CAT

13-item FACIT-Fatigue13-item CAT98-item Bank

No Fatigue Severe Fatigue

SE=0.32 (r=0.90)

SE=0.22 (r=0.95)

Comparison of Measurement PrecisionFull-length Item Bank vs. Legacy vs. CAT vs. Short-form

Sta

ndar

d E

rror

52

What do the results mean?

• Outcomes that matter to patients and providers• Practical (brief) and inexpensive• Covers a range, including normal• Greater precision• Comparability of scores• Ease of interpretation

• Reference values / normal ranges


Clinical PRO Lab?

53

What Does a Change in Score Mean?

30 40 50

Chronic Lung Disease

Physical Component Summary (PCS)

DiabetesType II

Congestive Heart Failure

Average Adult

Asthma After

Rx

Asthma Before

Rx

Treatment

Average Well Adult

50% reduction in disease burden 33% reduction in hospitalization Substantial increase in work productivity Subsequent cost savings

LowLow HighHigh

PRO Bank Person ScorePRO Bank Person Score

Interpretation AidsInterpretation Aids

Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q Q Q Q Q Q Q Q Likely Likely UnlikelyUnlikely

Item LocationItem Location

Q Q QQ Q QQ Q QQ Q Q Q Q QQ Q Q

Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q QQ Q QQ Q Q

Q Q QQ Q QQ Q QQ Q QQ Q QQ Q Q

Q Q Q Q

Q Q Q Q

Q Q Q Q

People and Items Distributed on the Same Metric: FatiguePeople and Items Distributed on the Same Metric: Fatigue

0.0

Ceiling effect

0.0

Ceiling effect

People with more fatigue

Items less likely to be endorsed

Items more likely to be endorsed

People with less fatigue

LowLow HighHigh



Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q Q Q Q Q Q Q Q Likely Likely UnlikelyUnlikely

Item LocationItem Location

Q Q QQ Q QQ Q QQ Q Q Q Q QQ Q Q

Q Q QQ Q Q Q Q QQ Q Q Q Q QQ Q QQ Q QQ Q Q

Q Q QQ Q QQ Q QQ Q QQ Q QQ Q Q

Q Q Q Q

Q Q Q Q

Q Q Q Q

LowLow HighHigh



30 40 50 60 70

M = 50, SD = 10

LowLow HighHigh

Interpretation Aids: Cancer example


Fatigue Score=60Fatigue Score=60

30 40 50 60 70

This patient’s fatigue score is 60, significantly worse than average (50). Cancer patients who score 60 on fatigue tend to answer questions as follows:

…”I have been too tired to climb one flight of stairs: VERY MUCH

…”I have had enough energy to go out with my family: A LITTLE BIT

Click here if you would like to see this patient’s individual answers

LowLow HighHigh



Fatigue Score=40Fatigue Score=40

30 40 50 60 70

This patient’s fatigue score is 40, significantly better than average (50). People who score 40 on fatigue tend to answer questions as follows:

…”I have been too tired to climb one flight of stairs: SOMEWHAT…”I have had enough energy to go out with my family: VERY MUCH

Click here if you would like to see this patient’s individual answers

60

How Long Per Concept? It depends

• Population monitoring • 1-2 questions

• Group-level outcomes monitoring • 5-7 questions

• Patient-level measurement/management• 10-12 questions or CAT

61

Matching Methods to Applications

1

2

3

4

5

6

7

1

3

5

7

Single-Item

1

2

3

4

5

Multi-ItemScale

PopulationMonitoring

Group-LevelOutcomesMonitoring

Patient-LevelManagement

“Item Pool” (CAT Dynamic)

7

6

Most Functionally Impaired

NoisyIndividual

Classification

Very AccurateIndividual

Classification

www.nihpromis.org

PROMIS Family of Delivery Platforms (2009 )

InteractiveVoice

Recognition

InternetAdministered

Personal Interview

Self Administered

Telephone Interview

Hand-held

Device

MultipleAssessment

Options

TV

Conclusions• Patient-reported outcomes are:

• Well characterized• Easily measured• Increasingly interpretable• Rarely used in Clinical Practice

• Going forward, clinician engagement is the key to setting and applying outcome targets and practice standards

Documents

Patient-Reported Questionnaires in Clinical Practice: Just Another Laboratory Test David Cella, Ph.D. Professor and Chair Department of Medical Social