Development and Evaluation of an Objective and Simulation-Based

Core Curriculum for Surgery Residents

Orlando C. Kirton, M.D., F.A.C.S., F.C.C.M., F.C.C.P.Ludwig J. Pyrtek, M.D. Chair in Surgery

Director of Surgery, Chief Division of General SurgeryHartford Hospital

Professor of SurgeryProgram Director, Integrated General Surgery Residency Program

Vice Chair, Department of SurgeryUniversity of Connecticut School of Medicine

Essential ACGME Requirement

• Create a knowledge-based and clinical curriculum based on six educational competencies:- Medical Knowledge

- Patient Care

- Interpersonal and Communication Skills

- Professionalism

- Practice-Based Learning and Improvement

- Systems-Based Practice

The Perfect Storm faced by Program Directors

• Restricted resident duty hours– I.e.;80 hours; PGY 1- no more than 16hrs

• Falling reimbursements for hospitals and faculty

• Less non-clinical time available to the Teaching Faculty

• An increasing # of educational requisites– The ACGME milestone project, robust

simulation curriculum, patient safety and quality objectives

The University of Connecticut Integrated General Surgery Residency

Program

• 1 of 8 general surgery residency training programs in Connecticut

• Integration of 3 separate training programs in 1990

• 46 Residents ; 6 chiefs• 5 Hospital integrated program• 110 teaching Faculty

– 60% private practice

The Residents’ Concerns with the Curriculum

Program Director (October 2009)• Diminished available time for residents• Less non-clinical time available to teaching faculty• Resident dissatisfaction with the didactic curriculum• Regurgitation from textbooks• Lack of audience interaction• Session taught by resident lacked depth and scope• Inadequate simulation exposure• Inadequate preparation time (learner & presenter)

Methods• A core curriculum steering committee of residents and

faculty:– Program Director– Incoming 2 Administrative Chief Residents– A peer–elected categorical resident representative

(PGY 1-3)– A faculty member from each teaching hospital

• A simulation curriculum steering committee also created– Similar composition + simulation coordinator and

education specialist• Starts meeting in February; Curriculum completed by

May

The Core Curriculum

• Focuses on specific goals and objectives– 2 year bundled curriculum of 85 standing (annual) and 25 bi-

annual learning topics• Utilizes online teaching materials

– SCORE Modules, ACS Fundamentals of SurgeryTM Curriculum– Various on-line texts : e.g.; Schwartz, Sabiston, etc

• Interactive lecture format– An audience response system

• Compulsory simulation curriculum• SCORE system-based practice modules• Professionalism in Surgery: Challenges and ChoicesTM, ACS

Division of Education (2008)

Chief Resident – Teaching FacultyModerator Team (Monthly Curriculum)

• Define content and format

• Identify and confirm presenters

• Utilize the ABSITE program report

• Moderate interactive sessions

– Vignettes, case scenarios, question/answer

– Focus on critical knowledge; decision making

• Exhaustive literature review prohibited

The Monthly Curriculum

Week 1 Interactive presentations

Week 2 Simulation/ skills sessions

Week 3 Interactive presentations; competency lecture, journal club

Week 4 Interactive presentation, resident research (11:00 AM)

Week 5 Dedicated to a 2nd simulation/ skills or competency presentation

Fridays; 8:30 AM – 11:00 AM

Intern Boot CampJuly / August

• Fluids and electrolytes

• Cardiac• Pulmonary• GI• MS• Wounds

• Nutrition• Pain management• Post-op emergencies• Fever work -up• Hypotension• Pre-op evaluation• Safety

Wednesday 2:00 – 4:30 PM; Friday 8:30 – 11:00 AM

- Fundamental of Surgery Curriculum – 25 essential modules- Clinical Skills Lab

Simulation / Skills ModulesPGY I

• Asepsis and instrument identification

• Knot tying; tissue handling

• Latex allergy; anaphylaxis• Chest tube and

thoracentesis• Basic laparoscopy• CVP and foley placement• Patient hand-off• FLS

PGY II-V• Sim man• Cholecystectomy/

Advanced laparoscopy• EGD; colonoscopy• Vascular anastomosis• Biolab/fresh tissue lab• Robotics

ACS e-learning site

Evaluation• Anonymous Survey Questionnaire

– Baseline (June, 2010); 6 months (December, 2010); 1 year

(June, 2012)

– Fifteen, 5-point Likert-type items• Conferences in relation to ACGME competencies

– PBI, SBP, IC, Professionalism

• The learning objectives of the curriculum

• Quality of teaching presentations

– Faculty and Residents

• Quality of the supplementary teaching materials

– Chi-square tests of proportions; Kruskal Wallis to

compare full distribution of responses

Survey QuestionsI. Conferences

Please rate the following aspects of the Friday morning core curriculum for the entire year using the scale below.

The core curriculum conferences taught me the value of practice based learning.

The core curriculum conferences positively impacted my ability to effectively communicate with patients, families and other health-care professionals.

The core curriculum conferences positively impacted my ability to act in a professional and ethical manner.

The core curriculum conferences provided knowledge of the healthcare system, beyond the confines of the hospital setting, which enables me to feel comfortable calling on other resources for assistance.

II. Learning Objectives & Goals

There is the opportunity to suggest curriculum changes with the core curriculum for the Integrated Surgical Residency Program.

The ACGME core competencies were adequately addressed during the academic year.

.

Survey Questions (con’t)

III. Supervision & Training – Attending Presentations

Preparedness of the faculty speakers.

Quality of the presentations by faculty speakers.

Do faculty regularly assigns specific reading at least 2 weeks prior to the presentations?

Do faculty presenters ask residents in the audience about key content?

IV. Supervision & Training – Resident Presentations

Preparedness of the resident speakers?

Quality of the presentations by resident speakers.

Were the weekly assigned reading and video materials relevant to the topics presented?

Do the assigned reading and video materials prepare you sufficiently for the weekly topics?

Survey Questions (con’t)

V. Materials

Weekly reading and videos relevant.

Weekly reading and videos sufficient preparation.

The Following Likert-Scales Were Used for Survey Questions

I. Conferences

0 – Not Applicable1 – Strongly Disagree2 – Disagree

3 – Neutral4 – Agree5 – Strongly Agree

II. Learning Objectives & Goals

0 – No Interaction

1 – Never

2 – Infrequently

3 – Sometimes

4 – Most of the time

5 – Always

The Following Likert-Scales Were Used for Survey Questions (con’t)

III. Supervision & Training (Attending & Resident Presentations)

0 – No Interaction

1 – Not at All

2 – Not Usually

3 – Usually

4 – Most Always

5 – Always

IV. Material

0 – Not Applicable

1 – Not Valuable

2 – A Little Valuable

3 – Valuable

4 – Very Valuable

5 – Extremely Valuable

Results

Percent of Residents Responding:Two Most Positive Categories

Conferences June 2010 1 Year

December2011

6 months

June 2011

1 Year

X2(p)

Taught value of practice based learning 52.5 78.8 93.3 .005

Positively impacted ability to communicate 37.5 72.7 86.7 .001

Positively impacted ability to act professionally/ethically 45.0 60.6 73.3 n.s.

Provided knowledge of health care system 42.5 60.6 80.0 .034

Learning Objectives and Goals

Opportunity to suggest curriculum changes 70.0 78.8 87.5 n.s.

ACGME core competencies were adequately addressed 72.5 87.9 93.3 n.s.

Supervision and Training - Attendings

Faculty speakers prepared 87.5 93.9 100 n.s.

Quality of the faculty presentations 50.0 78.8 73.3 .028

Sufficient lead time for reading assignments 12.5 66.7 73.3 .001

Faculty identifies key required content 50.0 75.8 86.7 .012

Faculty ask residents about key content 37.5 72.7 73.3 .004

6/10 – 42 respondents; 12/10 – 38 respondents; 6/11 – 22 respondents

Percent of Residents Responding: Two Most Positive Categories (con’t)

ConferencesJune 2010

1 YearDecember

20116 months

June 2011

1 Year

X2(p)

Supervision and Training – Resident Moderators

Resident speakers prepared 80.0 84.8 100 n.s.

Quality resident presentations 25.0 51.5 40.0 n.s.

Materials

Weekly reading and videos relevant 34.6 63.6 66.7 .046

Weekly reading and videos sufficient preparation 32.0 54.5 53.3 n.s

6/10 – 42 respondents; 12/10 – 38 respondents; 6/11 – 22 respondents

American Board of Training Inservice Examination [ABSITE] Percentage Scores

(Lower Quartiles; Top Quartile)

Exam Level

Lower Quartiles(0-75%)

Top Quartiles(76-100%)

Junior ABSITE Year 2009 89.5% 10.5%

(PGY I, II) 2010 89.5% 10.5%

2011 81.0% 19.0%

2012 57.1% 42.9%

Senior ABSITE Year 2009 95.0% 5.0%

(PGY III, IV, V) 2010 90.9% 9.1%

2011 88.9% 11.1%

2012 84.2% 15.8%

American Board Of Surgery Intraining-Examamination

Proportion of Residents in Top Quartile (76-100%)

0

10

20

30

40

50

2009 2010 2011 2012

All residents Junior Level Senior Level

Introduction of new curriculum

Study Limitations

• Unequal # of respondents at the 3 times of the survey

• Unable to separate the survey data by PGY years or categoricals vs. preliminaries

• ? Factor of survey fatigue

The Residents’ Concerns prior to the new approach to Core

Curriculum Development• Diminished available time for residents• Less non-clinical time available to teaching faculty• Resident dissatisfaction with the didactic curriculum• Regurgitation from textbooks• Lack of audience interaction• Session taught by residents lacked depth and scope• Inadequate simulation exposure• Inadequate preparation time (learner & presenter)

Residents’ comments post implementation

• All lectures more structured and beneficial– Presenters and learners better prepared

• Presentations much more interactive and engaging• The best lectures were those that invite audience participation• The intern boot camp is an extremely valuable component of the

core curriculum– Taught us from Day 1 important concepts

• Audience response system engaging and great prep for ABSITE• Avoid resident presentations all together• I enjoyed the practical portions that have been included in the

curriculum– Pig lab, lap trainer– Great chance to practice new skills in an environment not quite

so high stress as the OR

Conclusions

• The systematic collaborative approach (faculty and residents) to curriculum development with interactive, objective competency-based presentations, robust simulation, use of online teaching tools, engaged teaching faculty resulted in:– Resident satisfaction with the curriculum and their

self-reported clinical and academic abilities

– Increase in the number of residents scoring in the top quartile in the ABSITE

– Effectively addresses the ACGME competencies

Thank you!