Enhancing formative assessment in critical care training:
The use of a virtual patient
Timothy Willett, RCPSC Rakesh Patel, uOttawa Bernie McDonald, uOttawa Rick Hodder, uOttawa Angele Landriault, RCPSC Pierre Cardinal, RCPSC & uOttawa
Background
• Critical care: • Little time for decision-making during crises • High-stakes decisions • Few opportunities for safe practice with observation
and feedback • Formative assessment (Simulation and
feedback): • Standardized patients limited procedures • High-fidelity simulation limited fidelity
1. Rushton A. Formative assessment: a key to deep learning? Medical Teacher. 2005; 27(6):509-513.. 2. Yorke M. Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education. 2003; 45(4):477-501. 3. Aggarwal R, Mytton OT, Derbrew M, et al. Training and simulation for patient safety. Quality & safety in health care. 2010;19(Suppl 2):i34-43.
4
Alternative: Virtual Patients
• Can offer opportunities for deliberate practice
• Safe
• Reusable
• Best suited for clinical reasoning & decision-making
Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Medical education. 2009;43(4):303-11.
5
Objective
• Using a virtual patient for formative assessment during critical care training
• Feasibility • Perception of value for the use of RCsim as a tool
for formative assessment - Critical Care Fellow perspective - Facilitator perspective
6
Qualitative approach
• Post session questionnaires for fellows • 2 cohorts (total n=13) • 4 open ended questions • completed immediately following the session
• Interviews for facilitator
• Responses anonymously transcribed
• Thematic analysis
11
Results
• Feasibility: • 1 hour session scheduled back to back
- Administrative assistant • Academic half day (3 or 4 back to back) • Available intensivist for facilitation • Available operator familiar with the application
and cases • Cases
12
Fellows’ perspective
• Realism: • “The case was realistic” (EF) • “Videos very useful. Case realistic” (EF) • “Not as realistic as the patient is not in front of you
at all times” (IF)
• EF = End of session Feedback • IF = Integrated feedback (throughout the session)
13
Fellows’ perspective
• Flow: • “Runs like real scenario” (EF) • “Allowed you to work through on your own, even
when stuck to challenge you to see what you know” (EF)
• “A lot of interruptions” (IF) • “Feedback is effective because it makes you think
out loud” (IF)
14
Fellows’ perspective
• Outcomes: • “I gained insight into the speed at which info
gathered/things ordered” (EF) • “Importance of closing the loop” (EF) • “Benefit of reasoning out loud to involve the
knowledge of the rest of the team” (EF)
• “[Will] make me a better teacher” (IF) • “To be more quick in decisions” (IF)
15
Fellows’ perspective
• Value: • “One of the best teaching activities I had” (EF) • “Different from other learning
opportunities…better prep. you for the real situation” (EF)
• “Overall a very useful learning experience” (IF) • “OK for a junior or senior resident but not a
fellow” (IF)
16
Facilitator s’ Perspective
• Insight into how residents would manage a critical situation on their own (EF)
• Additional opportunity to see them manage a case, which is infrequent in real life (EF)
• Fellows were safe & thorough, with good clinical reasoning, but were hesitant to commit to action (IF)
• Hesitancy could be because of exam-like formative assessment model (IF)
17
Conclusion
• VP sessions are generally perceived as realistic
• Sessions are enjoyable for learners and facilitators
• Learners can receive practical feedback that they can put into practice
• Unique opportunity for facilitators
• Further research is needed to determine strengths and weaknesses of the two facilitation/feedback models
18