University of Rochester Medical Center

Structuring M & M Conferences for Educational Effectiveness

M.J. Kim, F.J. Fleming, J.H. Peters, R.S. Salloum, J.R.T. Monson, M. Eghbali Department of Surgery University of Rochester, Rochester, NY April 22, 2010

University of Rochester Medical Center 2

Background

- Morbidity & Mortality Conferences:• Required by ACGME for surgical residents

• “Practice Based Learning and Improvement”

1. Analyzing patient complications

2. Identifying causes

3. Proposing potential solutions

University of Rochester Medical Center 3

Background

- Content/format discrepancies Aboumatar et al, American Journal of Medical

Quality 2007

- Mixed lessons learnedBender et al, American Journal of Surgery 2009

- Structuring M&M presentations can improve• Understanding of complications

• Learner satisfactionRisucci et al, Current Surgery 2003

Muryama et al, American Journal of Surgery 2002

University of Rochester Medical Center 4

Objectives

1) Evaluate effectiveness of existing M&M conferences

• Content/Delivery

• Resident learning

• Perceptions of effectiveness

2) Develop a standardized presentation format

•Missing content

• Streamline discussion

3) Determine effectiveness of intervention

University of Rochester Medical Center 5

Definitions

Educational Effectiveness

- Specificity of the cause of the complication

- “OR staff not notified that potentially unstable

patient in ED” vs. “Patient had delay to OR”

- Specificity of the potential solution/change

- “Designate contact person in OR for

potential emergency cases” vs. “Need

better communication”

University of Rochester Medical Center 6

Methods

Study Design

•Pre/Post intervention evaluation

Study Sample

•Conferences in five divisions

•All presentations by residents

•Audience of faculty, residents, students,

and staff

University of Rochester Medical Center 7

Methods

- Conference Observation (TEACHING)•Two independent observers

•Consensus regarding conference practices

- Resident Questionnaires (LEARNING)•2 Questions + any comments

- Surveys of Learners (PERCEPTIONS)•Online, anonymous

•Last M&M conference attended

Results: Observation

Observation Item %Yes (n = 40)

1) Is the complication clearly stated? 80%

2) If so, is it stated by the resident

presenter?55%

3) Is a potential cause clearly

established?65%

4) Is the cause proposed by the

presenter?18%

5) Are strategies for practice change

discussed during each case?85%

6) Are the changes proposed by the

presenter?8%

University of Rochester Medical Center 9

Results: Questionnaires

Specificity of cause of complication (n=26)•Mean rating (±SD) = 3.11 (±1.48)

• 1 = Vague

• 5 = Specific

Specificity of a practice change (n=26) •Mean rating = 3.42 (±1.50)

• 1 = No practice change stated

• 3 = Vague practice change

• 5 = Specific practice change based on current discussion

Results: Survey

Question %Yes (n = 19)

1) Is the current format of M&M

presentations effective for analyzing

complications?

63%

2) Is the current format effective for

learning?53%

3) Are specific complications clearly

stated?47%

4) Are specific causes clearly stated? 21%

5) After presentation, is it clear how to

avoid this complication in the future?32%

University of Rochester Medical Center 11

Presentation Format

Case Presentation (< 5 minutes)• Introduction•Clinical Information•Analysis

Supporting Information (< 5 mins)•Literature/Background Review•Take Home Points (Maximum 2!)

Discussion (< 5 mins)

= Total Time < 15 minutes per case

University of Rochester Medical Center 12

Presentation Format

Format based on•Suggestions from surveys•Previous publications

Introduced to residents•Sample presentation demonstrated• Template slides with instructions

Introduced to faculty•Format/process discussed at faculty meeting

Template, sample, instructions sent to all residents, fellows, and faculty

University of Rochester Medical Center 13

Results: Observation

* p<0.01

Observation Item Pre-Format

%Yes (n = 40)

Post-Format, %Yes (n = 35)

1) Is the complication clearly stated? 80% 100%*

2) If so, is it stated by the resident

presenter?

55% 100%*

3) Is a potential cause clearly established? 65% 91%*

4) Is the cause proposed by the presenter? 18% 57%*

5) Are strategies for practice change

discussed during each case?

85% 89%

6) Are the changes proposed by the

presenter?

8% 54%*

University of Rochester Medical Center 14

Results: Questionnaires

Pre-Format

Mean (±SD)

Post-Format

Mean (±SD)

Specificity of the

cause of complication 3.11 (±1.48) 4.56 (±1.03)*

Specificity of a future

practice change 3.42 (±1.50) 4.31 (±1.40)*

*p<0.05

University of Rochester Medical Center 15

Results: Survey

Question Pre-Format,

%Yes (n = 19)

Post-Format,

%Yes (n = 18)

1) Is the current format of M&M

presentations effective for analyzing

complications?

63% 89%

2) Is the current format effective for

learning?

53% 89%

3) Are specific complications clearly

stated?

47% 89%

4) Are specific causes clearly stated? 21% 72%

5) After presentation, is it clear how to

avoid this complication in the future?

32% 78%

University of Rochester Medical Center 16

Conclusions

1) Structuring presentations leads to more

specific lessons taught during M&M

2) More specific lessons from resident

presenters leads to better learning by peers

3) Structuring presentations is a practical

intervention to improve M&M conferences

University of Rochester Medical Center 17

University of Rochester Medical Center 18

University of Rochester Medical Center 19

University of Rochester Medical Center 20

Results: Survey

Suggested areas for changes:

•Focused discussion period - 79%

•Established presentation format - 63%

•Time-limited presentations - 53%

•Resident-led discussion - 21%

•Other (specify improvements, streamline) - 21%

University of Rochester Medical Center 21

Results: Observation

* p<0.01

Observation Item Pre-Format,

%Yes (n = 40)

Post-Format,

%Yes (n = 35)

1) Is the complication clearly stated? 80% 100%*

1b) Within 5 minutes? 20% 100%*

2) If so, is it stated by the resident presenter? 55% 100%*

3) Is a potential cause clearly established? 65% 91%*

3b) Within 10 minutes? 10% 71%*

4) Is the cause proposed by the presenter? 18% 57%*

5) Are strategies for practice change discussed

during each case?

85% 89%

6) Are the changes proposed by the presenter? 8% 54%*

University of Rochester Medical Center 22

Results: Extent of use

Did residents use the new format?• Mean = 3.57 ±1.40 (1 = No organization,

5 = Followed each step)

Was there literature review?• Mean = 2.71 ±1.71 (1 = No attempt at supporting info,

5 = clearly applicable studies cited)

Did faculty wait to ask questions?• Yes = 17%

Was each case <15 minutes?• Yes = 51%

University of Rochester Medical Center 23

Future Directions

1) Evaluate long-term (6-8 months)

adherence to format changes

2) Study when/why format is not used

3) Compare with other assessments to

evaluate extent of benefit