1
RESULTS: A statistically significant improvement in overall per- formance from pre- to post-test was observed for each one of the 5 different core skills (paired t-test, p<.001; Table 1) showing improvement in surgical skills. CONCLUSIONS: A dedicated four week proficiency-based simula- tion curricula for PGY1 surgical residents significantly improves basic and intermediate surgical skills. These data served as a frame- work to build our training-schedule curricula and helped us opti- mize the time spent in the simulation center. The New GI Mentor Express: Validity Evidence for a Portable, Lower Cost Platform for the Fundamentals of Endoscopic Surgery Carmen L Mueller, MD, Pepa Kaneva, MSc, Gerald M Fried, MD, FACS, John D Mellinger, MD, FACS, Jeffery M Marks, MD, FACS, Brian J Dunkin, MD, FACS, Melina C Vassiliou, MD, FACS McGill University Health Centre, Montreal, Quebec, Southern Illinois University School of Medicine, Carbondale, IL, University Hospitals, Cleveland, OH INTRODUCTION: Ensuring proficiency in flexible endoscopy is a priority for training programs. The Fundamentals of Endoscopic Surgery (FES) examination measures the knowledge and skills required to perform endoscopy. The skills component of FES was created and validated on the GI Mentor II virtual reality endos- copy simulator (Simbionix LTD, Israel). A limitation to dissemina- tion of the FES test is the size and cost of this simulator. A more compact and lower cost alternative (GI Mentor Express) was devel- oped to offset these issues. The purpose of this study is to assess equivalence of scores measured by the two platforms so they can be used interchangeably for testing. METHODS: General surgery and gastroenterology residents at all levels of training and attending staff who regularly perform colo- noscopy at four training institutions participated. Each completed the five FES manual skills tasks on both simulator platforms in random order, with 3-14 days between tests. Scores were determined by a standardized computer-generated scoring system and compared using Pearson’s correlation coefficient. RESULTS: A total of 44 participants were enrolled (mean age: 32; 80% male) with a broad range of endoscopic experience. The mean (95% confidence interval) FES scores were 68 (61:74) on the GI Mentor II and 73 (67:78) on the Express. The correlation between scores on the two platforms was 0.85 (0.73:0.91; p<0.0001). CONCLUSIONS: There is a strong correlation between FES manual skills scores measured on the original platform and the new Express, providing validity evidence to support use of the GI Mentor Express for FES testing. A Deliberate Postoperative Debriefing Process Can Effectively Provide Formative Resident Feedback Mackenzie R Cook, MD, Jeffrey S Barton, MD, Sarah Brown, BS, Jennifer M Watters, MD, FACS, Karen E Deveney, MD, FACS, Laszlo N Kiraly, MD, FACS Oregon Health and Science University, Portland, OR INTRODUCTION: The American Board of Surgery initiated re- quirements for resident operative assessments in June 2012. We designed a process for operative feedback and hypothesized this would generate formative resident feedback and guide curriculum development. METHODS: Residents initiated a postoperative feedback discus- sion with their attending, immediately after a case. After this dis- cussion, both completed a Procedure Feedback Form (PFF). Statements describing strengths and weaknesses were analyzed us- ing grounded qualitative analysis. Comparisons were made between junior (PGY1&2) and senior (PGY3-5) residents. RESULTS: Over 6 months, 347 individual operations involving 48 different residents were analyzed using PFFs. This allowed iden- tification of 6 themes; Flow, Technique, Synthesis/Decision, Out- comes, Knowledge and Communication/Attitudes, in addition to numerous sub-themes. Thematic saturation was achieved. Attend- ings and residents consistently identified the need for additional experience, more pre-case preparation and improved quality of resi- dent operative communication. Technical feedback to senior resi- dents was more specific and nuanced than to junior residents; focusing on dissection technique, use of the non-dominant hand and accurate suture placement. Feedback for the most advanced residents incorporated team leadership and teaching. Attendings routinely commented upon a positive resident response to feed- back. Few comments were made upon the overall case outcome. CONCLUSIONS: This is the first description of a widely appli- cable, routine postoperative feedback process with written PFFs. Attendings noted resident improvement in response to feedback and provided more nuanced and specific feedback to senior resi- dents, though comments on case outcomes are lacking. Identified areas of weakness will be the target of future didactic and simula- tion curriculum development. Table 1. Results: PGY1 Surgical Skills Curriculum Core skills No. of tasks Overall score (% pretest) Overall score (% post-test) p Value Basic suturing and knot tying 11 15 85 < .001 Intermediate suturing and knot tying 6 21 89 < .001 FLS + 5 4 77 < .001 Flexible endoscopy 2 12 86 < .001 Robotics 7 62 83 < .001 + Fundamentals of Laparoscopic Surgery. Vol. 219, No. 3S, September 2014 Surgical Forum Abstracts S117

A Deliberate Postoperative Debriefing Process Can Effectively Provide Formative Resident Feedback

Embed Size (px)

Citation preview

Vol. 219, No. 3S, September 2014 Surgical Forum Abstracts S117

RESULTS: A statistically significant improvement in overall per-

formance from pre- to post-test was observed for each one of the5 different core skills (paired t-test, p<.001; Table 1) showing

improvement in surgical skills.

Table 1. Results: PGY1 Surgical Skills Curriculum

Core skillsNo. oftasks

Overallscore

(% pretest)

Overallscore

(% post-test) p Value

Basic suturingand knottying 11 15 85 < .001

Intermediatesuturing andknot tying 6 21 89 < .001

FLS+ 5 4 77 < .001

Flexible endoscopy 2 12 86 < .001

Robotics 7 62 83 < .001+ Fundamentals of Laparoscopic Surgery.

CONCLUSIONS: A dedicated four week proficiency-based simula-

tion curricula for PGY1 surgical residents significantly improvesbasic and intermediate surgical skills. These data served as a frame-work to build our training-schedule curricula and helped us opti-mize the time spent in the simulation center.

The New GI Mentor Express: Validity Evidence for aPortable, Lower Cost Platform for the Fundamentals ofEndoscopic SurgeryCarmen L Mueller, MD, Pepa Kaneva, MSc,Gerald M Fried, MD, FACS, John D Mellinger, MD, FACS,Jeffery M Marks, MD, FACS, Brian J Dunkin, MD, FACS,Melina C Vassiliou, MD, FACSMcGill University Health Centre, Montreal, Quebec, SouthernIllinois University School of Medicine, Carbondale, IL,

University Hospitals, Cleveland, OH

INTRODUCTION: Ensuring proficiency in flexible endoscopy is a

priority for training programs. The Fundamentals of EndoscopicSurgery (FES) examination measures the knowledge and skillsrequired to perform endoscopy. The skills component of FES

was created and validated on the GI Mentor II virtual reality endos-copy simulator (Simbionix LTD, Israel). A limitation to dissemina-tion of the FES test is the size and cost of this simulator. A morecompact and lower cost alternative (GI Mentor Express) was devel-

oped to offset these issues. The purpose of this study is to assessequivalence of scores measured by the two platforms so they canbe used interchangeably for testing.

METHODS: General surgery and gastroenterology residents at alllevels of training and attending staff who regularly perform colo-noscopy at four training institutions participated. Each completed

the five FES manual skills tasks on both simulator platforms inrandom order, with 3-14 days between tests. Scores were

determined by a standardized computer-generated scoring systemand compared using Pearson’s correlation coefficient.

RESULTS: A total of 44 participants were enrolled (mean age: 32;

80% male) with a broad range of endoscopic experience. The mean(95% confidence interval) FES scores were 68 (61:74) on the GIMentor II and 73 (67:78) on the Express. The correlation between

scores on the two platforms was 0.85 (0.73:0.91; p<0.0001).

CONCLUSIONS: There is a strong correlation between FESmanual skills scores measured on the original platform and thenew Express, providing validity evidence to support use of the

GI Mentor Express for FES testing.

A Deliberate Postoperative Debriefing Process CanEffectively Provide Formative Resident FeedbackMackenzie R Cook, MD, Jeffrey S Barton, MD, Sarah Brown, BS,Jennifer M Watters, MD, FACS, Karen E Deveney, MD, FACS,Laszlo N Kiraly, MD, FACSOregon Health and Science University, Portland, OR

INTRODUCTION: The American Board of Surgery initiated re-

quirements for resident operative assessments in June 2012. Wedesigned a process for operative feedback and hypothesized thiswould generate formative resident feedback and guide curriculumdevelopment.

METHODS: Residents initiated a postoperative feedback discus-

sion with their attending, immediately after a case. After this dis-cussion, both completed a Procedure Feedback Form (PFF).Statements describing strengths and weaknesses were analyzed us-

ing grounded qualitative analysis. Comparisons were made betweenjunior (PGY1&2) and senior (PGY3-5) residents.

RESULTS: Over 6 months, 347 individual operations involving48 different residents were analyzed using PFFs. This allowed iden-

tification of 6 themes; Flow, Technique, Synthesis/Decision, Out-comes, Knowledge and Communication/Attitudes, in addition tonumerous sub-themes. Thematic saturation was achieved. Attend-

ings and residents consistently identified the need for additionalexperience, more pre-case preparation and improved quality of resi-dent operative communication. Technical feedback to senior resi-dents was more specific and nuanced than to junior residents;

focusing on dissection technique, use of the non-dominant handand accurate suture placement. Feedback for the most advancedresidents incorporated team leadership and teaching. Attendings

routinely commented upon a positive resident response to feed-back. Few comments were made upon the overall case outcome.

CONCLUSIONS: This is the first description of a widely appli-cable, routine postoperative feedback process with written PFFs.

Attendings noted resident improvement in response to feedbackand provided more nuanced and specific feedback to senior resi-dents, though comments on case outcomes are lacking. Identified

areas of weakness will be the target of future didactic and simula-tion curriculum development.