Characterizing Patients with Patient Acceptable Symptom ... · Chi-square analysis (2x2 table) 2. …ability of generic PRO’s (PROMIS and GRNF) to discriminate between positive

Health Measures Conference 2017

Characterizing Patients with Patient Acceptable Symptom State (PASS) after Selected Foot and Ankle Orthopedic

Procedures using Patient Reported Outcomes

Jeff Houck, PT, PhD, George Fox University

Micheal Anderson, MD, University of Rochester

A.Samuel Flemister, MD, University of Rochester

Irvine Oh, MD, University of Rochester

Benjamin DiGiovanni, MD, University of Rochester

John Ketz, MD, University of Rochester

Judy Baumhauer, MD University of Rochester


Background• Patient Outcomes

• Patient Acceptable Symptom State (PASS)• Well versus healed?

• Pubmed (77 studies) Cataracts, Arthritis (RA, OA, AS), generic musculoskeletal problems, and others

• PASS versus MCID

• Success/Failure • Patients definition of success dependent on expectations

• related however distinct


Generic PRO’s(PROMIS Scales & Global Rating of Normal Function)

• PROMIS• T-score is abstract• MCID versus PASS • Benchmarks that are useful

• Global Rating of Normal Function (GRNF)• Used widely in psychology • Common question on legacy instruments (e.g. FAAM)


Purpose/Objectives

Determine

1. …association of patient outcomes PASS and Success

2. …ability of generic PRO’s (PROMIS and GRNF) to discriminate between positive and negative patient outcomes (aggregate)

3. …ability of generic PRO’s to predict positive and negative patient outcomes using preoperative assessment (PROMIS only) and at follow up

4. …proportion patients identified by accurate thresholds for positive/negative outcomes.


PARAMETERS NUMBER MEAN SD RANGEAGE 90 53.0 16.5 15-80GENDER

FEMALEMALE

49/90 (54.4%)41/90 (45.6%)

FOLLOW-UP (MONTHS) 90 17.0 5.9 7.1-28.8

PROCEDURECPT 28899- subtalar arthrodesis

CPT 28296- bunion correction

CPT 27698- reconstruction ankle ligaments

CPT 27691- deep tendon transfer

17/90 (18.9%)15/90 (16.7%)29/90 (32.2%)29/90 (32.2%)

OUTCOMESPASS (YES %)SUCCESSFULGRNFPROMIS PFPROMIS PIPROMIS D

66/90 (73.3%)75/90 (83.0%)90909090

7.0 2.346.4 9.551.3 10.145.1 9.4

0-1027.2-73.338.7-74.134.2-69.5

Sample

90 Patients reached by phone (7.1-28.8 months post surgery)

4/10 most common F&A surgeries

Pre-operative PROMIS data available for 64 patients


Patient OutcomesConcept Dichotomous Question

Success/Failure Which of the following applies to you?

_____ I consider my surgery a success

_____ I consider my surgery a failure

Patient Acceptable

Symptom State

(PASS)

Taking into account all the activity you have during your daily life, your

level of pain, and also your functional impairment, do you consider that

the current state of your foot and ankle is satisfactory?

_____ Yes

_____ N0


Generic Patient Reported Outcomes (PRO’s)

PRO Rating/Instrument

Global Rating of

Normal Function

(GRNF)

How would you rate the function of your foot and ankle on a scale of 0-10 with 10 being normal, excellent function and 0 being the inability to perform any of your usual daily activities which may include sports?

Patient Reported

Outcome

Instrumentation

System (PROMIS)

Pain Interference CAT

Physical Function CAT

Depression CAT


Analyses • Determine


Chi-square analysis (2x2 table)

2. …ability of generic PRO’s (PROMIS and GRNF) to discriminate between positive and negative patient outcomes (aggregate)

Two Way ANOVA (PROMIS Scales) & Mann-Whitney U test (GRNF)

3. …ability of generic PRO’s to predict positive and negative patient outcomes using preoperative assessment (PROMIS only) and at follow up

ROC analysis, thresholds nearest 95 % sensitivity/specificity

4. …proportion of patients identified by accurate thresholds for positive/negative outcomes for each generic measure.

Chi-square analysis 3x2 tables (predicted Positive outcome, negative outcome, and ambiguous)



Success Failure Total Chi Square, p value

PASS

No 11 (12.2%) 13 (14.4%) 26.7%

Yes 64 (71.1%) 2 (2.2 %) 73.3% 33.1, p < 0.001


2. ...discriminate between positive/negative patient outcomes

0

10

20

30

40

50

60

70

PROMISPF PROMISPI PROMISDepression

t-score

PROMISScales

AverageFollowUpAveragePROMISScoresforPASSYes/No

PASSNO

PASSYES

0

10

20

30

40

50

60

70

PROMISPF PROMISPI PROMISDepressiont-score

PROMISScales

AverageFollowUpAveragePROMISScoresforSuccessYes/No

SuccessNO

SuccessYES

9.7

-8.0

-7.610.6

-9.9

-7.3


3. …to predict positive/negative patient outcomes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Sensitivity

1-Specificity

RecieverOperatorCurvesofPre-OperativeandFollowUpPROMISScoresPredictingPASS

PFFollowUp

PIFollowUp

DepressionFollowUp

PFPre-Operative

PIPre-Operative

DepressionPre-Operative

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

Sensitivity

1-Specificity

RecieverOperatorCurvesforPre-OperativeandFollowUpPROMISScoresPredicting"Success"

PFFollowUp

PIFollowUp

DepressionFollowUp

PFPre-Operative

PIPre-Operative

DepressionPre-Operative


Thresholds for Positive/Negative Patient Outcomes

PARAMETER PASS YES PASS NO SENSITIVITY SPECIFICITY

GRNF >7 <5 97.1 % 93.8 %

PF >50.0 <33.5 95.7% 95.8 %

PI <46.9 >65.6 95.7% 95.8 %

DEPRESSION <44.0 >60.3 95.6 % 91.3 %

PARAMETER SUCCESS FAILURE SENSITIVITY SPECIFICITY

GRNF >6* <5 94.7 % 73.3 %*

PF >45.3 <33.2 96.0% 92.3 %

PI <47.3 >64.5 94.7 % 92.3 %

DEPRESSION <45.8 >60.3 95.6 % 85.7 %

*note using >7 as the cut off yields a specificity of 100%


4. …proportion of patients identified

PROMIS PI SUCCESS FAILURE TOTAL CHI

SQUARE,

P VALUE

>64.5 FAILURE

47.3-64.5AMBIGUOUS

<47.3 SUCCESS

4 (4.4%)

39 (43.3%)

32 (35.6%)

6 (6.7 %)

8 (8.9%)

1 (1.1%)

11.1%

52.2%

36.7%

17.9, p< 0.001

GRNF SUCCESS FAILURE TOTAL CHI

SQUARE, P

VALUE

<5 FAILURE

5-7 AMBIGUOUS

>7 SUCCESS

4 (4.4%)

14 (15.6%)

57 (63.3%)

8 (8.9%)

3 (3.3%)

4 (4.4%)

13.3%

18.9%

67.8%

26.1, p < 0.001

All proportions (categorizations) based on generic PRO’s were significant 43-81.1% of patients were accurately identified (95% Sn/Sp)Positive outcomes (PASS yes or Success) were more successfully predicted than negative outcomes


Summary

• The PASS and Success definitions of patient outcome are similar but not the same

• Generic PRO’s assessed discriminate groups based on patient outcome

• PROMIS and GRNF were predictive of current patient status (Follow Up), however, pre-operative scores did NOT predict Follow Up data.

• High proportions (43-81%) were identified by ONLY taking into account PRO assessments.


Notable Limitations

• Sample was too small to derive and validate thresholds

• Number of surgical failures was small – so models are likely overfit

• Specific diagnoses may give different results

• Representative sample (contacted versus not-contacted patients – same mean age and % female) ?


Questions

Documents

Characterizing Patients with Patient Acceptable Symptom ... · Chi-square analysis (2x2 table) 2. …ability of generic PRO’s (PROMIS and GRNF) to discriminate between positive