Caroline E. Reinke, MD MSPH, Rachel R. Kelz , MD MSCE,

Trimming the Fat:Optimizing Overall Educational Value by Defining Factors Associated with Overall Educational Value

and Service to Education Ratio

Caroline E. Reinke, MD MSPH, Rachel R. Kelz, MD MSCE, Lori Pray, BA, Noel Williams, MD, Joshua Bleier, MD,

Kenric Murayama, MD, Jon B. Morris, MD

Division of Surgery Education Hospital of the University of Pennsylvania

Philadelphia, PA

Nothing To Disclose

Optimizing Overall Educational Value

Background

• Accreditation Council for Graduate Medical Education (ACGME) responsible for assessing and advancing the quality of resident physicians’ education– Founded in 1981– Requires programs to achieve a balance between education and

service

• With decreasing work hours, defining which characteristics are educational is critical to effective education

• Little is known about which modifiable rotation characteristics impact perceived service and education


ACGME Resident Survey:

In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands?

How often has your clinical education been compromised by excessive service obligations?

Specific Aims

AIM 1: To identify rotation characteristics associated with service to education ratio and overall educational value

AIM 2: To understand the relationship between the perceived overall educational value and the service to education ratio


AIM 3: To understand how these relationships differ between junior (CY1-2) and senior residents (CY3-5)

Data Source

• Responses to an 11-item Web-based survey during FY2011– Distributed to all residents at the end of each rotation

• Exclusion criteria:– Non-clinical rotations (simulation month)– Subspeciality-specific rotations (ie. plastic surgery)

• Survey responses were linked to additional rotation data:– Length of the rotation in number of weeks– Number of attending surgeons on each service– Presence or absence of fellows


Rotation Characteristics 1 2 3 4 5Volume of Operative Experience Poor Fair Adequate Very Good Exceptional

Complexity of Operative Experience Too Simple Basic Appropriate Challenging Beyond my Capacity

Autonomy of Operative Experience Minimal Occasional Commesurate with ability Frequent Excessive

Patient Care Participation Minimal Occasional Adequate Frequent Excessive

Autonomy in Patient Care Minimal Occasional Commensurate with ability Frequent Excessive

Time for Reading/Cognitive Development Minimal Would

prefer more Sufficient Would prefer less Excessive

Educational Value of Fellows and Speciality Residents Detrimental Inefficient Irrelevant Marginal Exceptional

Ratio of Educational to Service Activity

0% Educ, 100% Service





Overall Educational Value None Fair Adequate Very Good Outstanding

Duration of Rotation 2 months too long

1 month too short Just right 1 month too

long2 months too long

Teaching Faculty are supportive of ACGME Duty Hour Rules Never Rarely Occasionally Frequently Always

Rotation Survey


Study Population

57 general surgery training program rotations were evaluated by 85 residents

Excluded rotations in which less than three residents in a given CY evaluated the rotation (n=9)

Final study population = 48 general surgery rotations with 578 resident evaluations (range 3-36)

Overall survey response rate = 87%


Statistical Analyses• Linear regression used to assess the relationship between

rotation characteristics (RC), rotation data (RD), and– Overall Educational Value (OEV)– Service to Education Ratio (SER)

• Linear regression used to assess the relationship between OEV and SER

• Multivariate linear regression was used to examine the relationship between SER and OEV after controlling for significant RC and RD

• Subgroup analyses performed for junior (PGY1-2) and senior (PGY3-5) residents


Relationship between OEV and SER


12

34

5

Ove

rall

Ed

uca

tion

al V

alu

e

1 2 3 4 5

Autonomy of Operative Experience

Operative Autonomy vs. Overall Educational Value



12

34

5O

vera

ll E

duc

atio

nal

Va

lue

1 2 3 4 5

Autonomy of Operative Experience

Operative Autonomy vs. Overall Educational Value

Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.03 0.71 Number of attending surgeons -0.11 0.02 Fellow -0.02 0.93 Other Rotation Characteristics (RC) Volume of operative experience 0.37 <0.001 Complexity of operative experience 0.69 <0.001 Autonomy of operative experience 0.58 <0.001 Patient care participation -0.36 0.32 Autonomy in patient care 0.58 0.005 Time for reading and cognitive development 0.85 <0.001 Educational value of fellows on the rotation 0.35 0.002 Duration of the rotation -1.63 <0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.02 <0.001


RC, RD and the relationship with OEV










Rotation Characteristics (RC) of interest β Coefficient p-value Service to Education Ratio (SER) -- -- Additional Rotation Data (RD) Rotation Length -0.7 0.87 Number of attending surgeons* 1.29 0.06 Fellow 0.13 0.45 Other Rotation Characteristics (RC) Volume of operative experience -1.62 <0.001 Complexity of operative experience -0.63 <0.001 Autonomy of operative experience -0.81 <0.001 Patient care participation* 0.19 0.03 Autonomy in patient care* -0.29 0.06 Time for reading and cognitive development -0.74 <0.001 Educational value of fellows on the rotation* -0.51 0.06 Duration of the rotation 0.25 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules -0.53 <0.001


RC, RD and the relationship with SER









12

34

5O

vera

ll E

duc

atio

nal

Va

lue

(O

EV

, m

ean)

1 2 3 4 5

Service to Education Ratio (SER, mean)

Senior (CY 3-5) Junior (CY 1-2)

Service to Education Ratio (SER) vs. Overall Educational Value (OEV)



12

34

5O

vera

ll E

dcu

atio

nal

Va

lue

(O

EV

, m

ean)

1 2 3 4 5

Service to Educaiton Ratio (SER, mean)

Senior (CY 3-5) Junior (CY 1-2) Fitted values

Service to Education Ratio (SER) vs. Overall Educational Value (OEV)


The relationship between SER and OEV

Rotation Characteristics (RC) of interest

Overall Educational Value

β Coefficient p-value Adj. R-squared

Overall educational value (OEV) -- -- --

Service to education ratio (SER) -1.23 <0.001 0.68


The relationship between SER and OEV

Rotation Characteristics (RC) of interest

Overall Educational Value

β Coefficient p-value Adj. R-squared



After adjustment for other RC & RD



Subgroup Analysis of Junior and Senior Rotations

Junior Rotations (n=28)Overall Educational Value

β Coefficient p-value Adjusted R-squared


Service to education ratio (SER), unadjusted -1.27 <0.001 0.71

Service to education ratio (SER), adjusted -1.27 <0.001 0.85

Senior Rotations (n=20)


Service to education ratio (SER), unadjusted -1.28 <0.001 0.65

Service to education ratio (SER), adjusted -0.46 0.15 0.90

Summary

• We identified an inverse relationship between the service to education ratio and overall educational value, which was not fully explained by other rotation characteristics

• This relationship differed significantly for junior and senior rotations

• Individual rotation characteristics varied in their relationship with overall educational value– The value of fellows varied, but their simple presence or

absence was not predictive of OEV– Number of attending surgeons was negatively correlated

with overall educational value


Thank You


PENN Surgery

Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length -0.11 0.29 Number of attending surgeons -0.09 0.10 Fellow 0.01 0.96 Other Rotation Characteristics (RC) Volume of operative experience 0.31 0.006 Complexity of operative experience 0.55 0.02 Autonomy of operative experience 0.48 0.02 Patient care participation -0.55 0.22 Autonomy in patient care 0.78 0.003 Time for reading and cognitive development 0.86 <0.001 Educational value of fellows on the rotation 0.76 0.001 Duration of the rotation -1.80 0.001 Teaching faculty are supportive of ACGME Duty Hour Rules 1.04 0.001


RC, RD and the relationship with OEV – Junior

Rotation Characteristics (RC) of interest β Coefficient p-value Overall educational value (OEV) -- -- Additional Rotation Data (RD) Rotation Length 0.22 0.08 Number of attending surgeons -0.15 0.11 Fellow 0.01 0.90 Other Rotation Characteristics (RC) Volume of operative experience 0.60 <0.001 Complexity of operative experience 1.44 <0.001 Autonomy of operative experience 1.18 <0.001 Patient care participation 0.10 0.89 Autonomy in patient care 0.28 0.43 Time for reading and cognitive development 1.04 0.04 Educational value of fellows on the rotation 0.32 0.03 Duration of the rotation -1.52 0.003 Teaching faculty are supportive of ACGME Duty Hour Rules 1.56 0.03


RC, RD and the relationship with OEV – Senior

Background – ACGME Resident Survey

• In your opinion, how often do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands?

• How often has your clinical education been compromised by excessive service obligations?


Documents

Caroline E. Reinke, MD MSPH, Rachel R. Kelz , MD MSCE,