Rates of Patient Safety Indicators (PSIs) Rates of Patient Safety Indicators (PSIs) among VA Patients in the First Two Years among VA Patients in the First Two Years Following ACGME Resident Hour ReformFollowing ACGME Resident Hour Reform

Amy Rosen, Ph.D.

Center for Health Quality, Outcomes and Economic Research, VA Center of Excellence, Bedford VAMC

Boston University School of Public Health, Health Policy and Management Department, Boston, MA

AcademyHealth ● June 2008

Supported by VA HSR&D IIR# 04.202.1 and NHLBI ROI HL 082637

Co-Investigators

Susan Loveland, M.A.T.1,2 Patrick Romano, M.D., M.P.H.3

Kamal Itani, M.D.4 Paul Rosenbaum, Ph.D.5 Jeffrey Silber, M.D., Ph.D.5,6,7,8 Orit Even-Shoshan, M.S.6,7 Michael Halenar, B.A.6,9

Yun Teng, M.S.7 Jingsan Zhu, M.B.A.6

Kevin Volpp, M.D., Ph.D.5,6,7,8,9

1 Center for Health Quality, Outcomes and Economic Research, VA Center of Excellence, Bedford VAMC 2 Boston University School of Public Health, Health Policy and Management Department 3University of California at Davis, School of Medicine, Division of General Medicine and Center for Healthcare Policy and Research 4Boston VA Healthcare System and Boston University, Departments of Surgery 5University of Pennsylvania, The Wharton School 6University of Pennsylvania, School of Medicine 7Center for Outcomes Research, The Children’s Hospital of Philadelphia 8The Leonard Davis Institute of Health Economics, The University of Pennsylvania 9Center of Health Equity Research and Promotion, VA Center of Excellence, Philadelphia VAMC

ACGME duty hours regulations implemented on July 1, 2003 to improve safety

Do duty hour regulations improve or worsen patients outcomes in more vs. less teaching-intensive hospitals?

Previous work shows slight improvement in mortality for VA patients with common medical conditions

Little evidence on the effect of ACGME on other outcomes

Background

To investigate whether rates of the AHRQ Patient Safety Indicators (PSIs) changed following ACGME duty hour reforms in more vs. less teaching-intensive hospitals in VA.

Hypotheses:(1) Rates of technical skill-based PSIs (e.g. iatrogenic pneumothorax) would improve(2) Rates of PSIs related to continuity of care (e.g. postoperative sepsis) would worsen(3) Rates of PSIs reflecting collaborative care (e.g. postoperative hip fracture) would not change

Objectives and Hypotheses

All unique patients (n=828,534) admitted to acute-care VA hospitals (n=131) between July 1, 2000 and June 30, 2005

Index admission: first admission for which there was no prior admission eligible for a given PSI composite within 5 years

Exclusions from VA inpatient data: Admissions to hospitals outside the US (n=41,928) Transfers from non-VA hospitals (n=31,049) Admissions spanning July 1, 2003 (n=6,809) Admissions for patients older than 90 years (n=11,398)

Study Population

VA inpatient data files (VA Patient Treatment File) for diagnoses, age, gender, and discharge status

Veterans Health Administration Support Service Center Occupancy Rate Reports for number of beds per hospital

VA Office of Academic Affiliations for number of residents

Pre-reform period: pre-year 3 (7/01/00-6/30/01), pre-year 2 (7/01/01-6/30/02), and pre-year 1 (7/01/01-6/30/03)

Post-reform period: post-year 1 (7/01/03-6/30/04) and post-year 2 (7/01/04-6/30/05)

Data Sources/Years

Measured by the resident-to-bed ratio (RTB ratio)Calculated as the number of interns plus

residents divided by the mean number of operational beds

Minor teaching: RTB ratio >0 - <0.25

Major teaching: RTB ratio >0.25 - <0.60

Very major teaching: RTB ratio >0.60

Teaching Intensity

Technical Care (795,306 admissions) Iatrogenic pneumothorax Foreign body left in during procedure Accidental puncture or laceration Postoperative hemorrhage or hematoma

Continuity of Care (339,504 admissions) Postoperative physiologic or metabolic derangement Postoperative pulmonary embolism/deep vein thrombosis

(PE/DVT) Postoperative sepsis

Collaborative Care (653,270 admissions) Postoperative hip fracture Postoperative respiratory failure Selected infections due to medical care

PSI Composites

Multiple time series research design

Conditional logistic regression models adjusted for age, gender, comorbidities, secular trends, baseline severity, and stratifying on hospital site

Odds ratios measure the degree to which PSI composite rates change differently in more vs. less teaching-intensive hospitals

Tested stability by eliminating patients admitted to hospitals in NY State and those patients admitted from nursing homes

Analysis

Years Pre and Post Duty Hour Reform of 7/1/03

Pre-3 Pre-2 Pre-1 Post-1 Post-2

Academic YearNumber Admissions

2000-1178,207

2001-2184,544

2002-3185,835

2003-4192,176

2004-5197,516

Age (Mean) 63.21 63.40 63.31 63.45 63.47

Mean # of Comorbidities 1.43 1.46 1.50 1.58 1.63

COMPOSITE RATES

Technical Care PSIs PSI Events Total Eligible Admissions % Rate

751154,233

0.49

789157,594

0.50

876157,685

0.56

923161,141

0.57

873164,653

0.53

Continuity of Care PSIs PSI Events Total Eligible Admissions % Rate

70465,687

1.07

81066,392

1.22

80066,425

1.20

91969,300

1.33

93871,700

1.31

Collaborative Care PSIs PSI Events Total Eligible Admissions % Rate

544129,005

0.42

543129,936

0.42

497128,224

0.39

594132,117

0.45

539133,988

0.40

Description of VA Sample

Non-teaching(0)

MinorTeaching (>0-0.25)

MajorTeaching(0.25-0.6)

Very MajorTeaching

(>0.6)

N (%) offacilities

19(14.50%)

32 (24.43%)

40 (30.53%)

40 (30.53%)

N (%) ofadmissions

36,634(3.90%)

144,294(15.38%)

387,667(41.32%)

369,683(39.40%)

Hospital Characteristics by Teaching Status in VA

VA TECHNICAL PSI COMPOSITE UNADJUSTED RATES BY ACADEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5

Pre3 Pre2 Pre1 Post1 Post2

Academic Year

Rat

es (

%)

Group 1 ( 0 ) Group 2: (>0.0 - <=0.25) Group 3 (>0.25 - <=0.6) Group 4 ( >0.6 )

Changes Over Time in Unadjusted PSI Composite Rates in More vs. Less Teaching-Intensive VA Hospitals

Group 1 ( 0 ) Group 2: (>0.0 - <=0.25) Group 3 (>0.25 - <=0.6) Group 4 ( >0.6 )

Changes Over Time in Unadjusted PSI Composite Rates in More vs. Less Teaching-Intensive VA Hospitals

VA CONTINUITY OF CARE PSI COMPOSITE UNADJUSTED RATES BY ACADEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5

Pre3 Pre2 Pre1 Post1 Post2

Academic Year

Rat

es (

%)

VA COLLABORATIVE CARE PSI COMPOSITE UNADJUSTED RATES BY ACADEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5

Pre3 Pre2 Pre1 Post1 Post2

Academic Year

Rat

es (

%)

Group 1 ( 0 ) Group 2: (>0.0 - <=0.25) Group 3 (>0.25 - <=0.6) Group 4 ( >0.6 )

Changes Over Time in Unadjusted PSI Composite Rates in More vs. Less Teaching-Intensive VA Hospitals

Technical CareComposite

Continuity of Care

Composite

Collaborative CareComposite

Odds Ratio(95% CI)

Odds Ratio (95% CI)

Odds Ratio (95% CI)

Resident/bed ratio*post1 a

1.09(0.78 - 1.51)

P= 0.62

1.01(0.70 - 1.46)

P= 0.95

1.18 (0.75 - 1.85)

P= 0.48

Resident/bed ratio*post2 a

1.05(0.75 -1.45)

P= 0.79

1.39(0.97 - 1.99)

P= 0.08

1.60(1.01 - 2.53)

P= 0.04

Number of cases 795,306 339,504 653,270

Adjusted Odds of PSI Composite Events After Duty Hour Reform in More vs. Less Teaching-Intensive VA Hospitals

Residents get more sleep, but there are also increased handoffs

Residents can still work 30 hours straight

Compliance may not be high

Possible Explanations for No Major Changes in PSI

Rates

StrengthsNational sample of patients from the VADifference-in-difference approach to reduce

likelihood of confoundingUtilization of standardized measures of patient

safety

LimitationsNo clinical data for risk adjustmentLimitations in using PSIs to measure patient safetyUnmeasured confounding

Strengths/Limitations

Rates of Technical Care PSI composite did not improve between more vs. less teaching-intensive hospitals over time

Rates of Continuity of Care PSI composite did not worsen - we saw a slight trend in worsening rates in post-reform year 2 in more teaching-intensive hospitals

Rates of Collaborative Care PSI increased slightly over time (only in post-reform year 2) in more teaching-intensive hospitals, marginally significant increase (p=0.04)

Conclusions

Implementation of ACGME had no overall systematic impact on patient safety in more vs. less teaching-intensive hospitals

Lack of improvement in any of the PSI composites suggests that more work is needed to improve safety

Need to examine the effects of ACGME beyond first 2 years

Need more data on facilitators and barriers to improvements in clinical outcomes

Implications

Thank you!