Using online video annotation to develop communication and self-reflection skills in

medical students: A pilot study

Dr Steve Gallaghera Dr Janet Rountreeb, Professor Bernadette Drummondb, Dr Jane Millichampa, Dr Maria Stubbec, Jayde Fletta

aDepartment of Psychological Medicine, University of Otago, Dunedin, New Zealand; bFaculty of Dentistry, University of Otago, Dunedin, New Zealand;

cPrimary Health Care and General Practice, University of Otago, Wellington

Background

• Many professional courses provide opportunities to review filmed consultations (either real or acted)

• You can learn a lot by watching yourself, and reflecting on how you might do things differently

Conventional Face-to-Face ApproachUndergraduate Medical Students(4th Year Communication Skills)• Simulated consultation:

– Video-recorded– Actor provides written feedback for discussion between

student & tutor– Student immediately takes video to tutor to watch and

review– Student and Tutor discuss– Tutor simultaneously coaches, consoles, guides and

assesses (without having previously seen the video)

Issues with traditional approach

– Logistics – Low motivation – Confounding reflection with assessment– The cringe factor – Tutor Load – Decreased opportunities for reflection

The action-reflection cycle

Image: http://celt.ust.hk/teaching-resources/action-research

Schön (1983) – The Reflective Practitioner

Argyris (e.g., 1991) – Teaching Smart People to Learn

- Reflective Practice is:• the ability to learn from personal

experience• an important attribute for a

competent healthcare professional *

* Mann, Karen, Gordon, Jill, & MacLeod, Anna. (2009). Reflection and reflective practice in health professions education: a systematic review. Advances in Health Science Education, 14(4), 595-621.

Need, Solution, and Aims• Need: a system that addresses issues with current approach and

promotes opportunities for reflection

• Solution: An online environment for students to:– review filmed clinical consultations– annotate video– share comments for feedback

• Aims:1. Increase flexibility2. Increase motivation3. Enhance teaching opportunities4. Capture evidence of reflective practice

Univerisity of Otago CALT Grant 2013

• Pilot an online system for reviewing filmed patient interactions:– undergraduate medical students (4th year, n=20)– postgraduate paediatric dental students (n=6)

Online Approach• The students accessed a private

website https://vbr.siliconcoach.com– Good video analysis tools – Granular privacy

Our objectives were to establish the acceptability of this online-enhanced process and compare it with immediate face-to-face (F2F-only) review

Video Based Reflection (VBR)Adapted existing tool (SiliconCoach) in partnership with Bracken Learninghttp://www.brackenlearning.com/

Dentistry – 2 cameras

Did we enhance learning with this approach? - Medicine

• The good– “I really liked it. So valuable to be able to watch it yourself and think

about yourself first”– “…More efficient. ..(my tutor) could see if there was actually any gaps

in my reflection rather than me just agreeing with things he pointed out.”

– “More beneficial as a learning experience.”

• The not-so-good– “The gap between doing the scenario and discussing it, forget real

issues that came up during it. Took too much time and effort to complete it online, quicker and more useful if discussed straight after.”

– “The whole process takes longer.”

Evaluation

• Medical students evaluated both methods • Majority (69%) indicated a preference for

online-enhanced reviews. • Mean satisfaction rating = 2– (on a 5-point scale was 2, where 1 indicated “very

happy”).

Aim 1

• Increase flexibility • “It was more flexible in terms of being able to complete it in your own

time.”

Tutors: took longer initially to review, but:– Became faster with experience– Allowed overall comments to be completed in advance– Led to a much more satisfactory, higher quality and less pressured

feedback session (n=2!)

Aim 2

• Increase motivation: decouple reflection and assessment, tacit oversight

• Tutor perspective: assessment could be completed early, leaving more time for coaching and discussion

• Some prefer to be unmotivated

Photo by Gene Trindl - © 1983 Gene Trindl - Image courtesy mptvimages.com Sourced from www.imdb.com

Student “Mr T” (paraphrased)“I preferred the old method”“Why’s that?” I ask.“This takes more time.”(pause)“And I couldn’t just turn up and listen.”

Enhance teaching opportunities: create a digital version of interactions that can be easily shared with others for teaching purposes (with consent)

Aim 3

Aim 4• Capture evidence of reflective practice

• “Had time to think. Bring up points to discuss”

(in progress, thanks to Jo Hilder from University of Otago, Wellington, for coding and analysis.Criteria adapted from Mezirow’s Levels of Reflectivity)

Where to next

• 2014– Rollout to all 80 4th year students in Dunedin School of

Medicine– Trial in University of Otago, Wellington for 4th Year

General Practice formative assessments– Interest from 2nd and 3rd year medical teaching (n=280

students per year)– Planning further work in Dentistry– Future possibilities in developing ePortfolios, CME for

practitioners, and academic publication

Acknowledgements

• The incredibly helpful team at Bracken Learning

• Colleagues and students• Faculty of Medicine (for funding)

Contact: steve.gallagher@otago.ac.nz