Memory Science in Medical Education

Frank Lau, MD Co-Founder & Chief Medical Officer

GunnerTraining.com

“I feel like every time I review something or learn something new, I

forget something else.”

– C.C., 2nd year medical student, University of Michigan Medical School

The Challenge for Medical Students

The Challenge Persists for Residents   Resuscitation, July 2009

  70 Johns Hopkins pediatrics residents   66% failed to start timely CPR (basic life support)

  95% of the residents had received this level of training

  83% made errors during defibrillation (shocking the patient)   80% of the residents had been trained in defibrillation

  More years of training did not improve physician performance

Medical knowledge is routinely forgotten, even by the best and the brightest.

The Challenge Persists for Attending Physicians

  Journal of Trauma, June 1996   60 practicing trauma physicians took the standard Advanced

Trauma Life Support course, followed by an examination   National course for managing traumas (car accidents, gunshots, stab

wounds, etc)

  Immediate testing: 86% average score   6 months later, 50% failed   13 years after this study, no novel approaches to learning and

retaining medical knowledge

Basic, critical knowledge is routinely forgotten, even by experienced specialists.

Who Should Care?   Medical Students   Physicians-in-training (residents)   Educators   Foreign medical graduates   Practicing physicians

  Patients   Healthcare policy makers   Patient safety advocates

The 2 Phases of Knowledge Acquisition

Mastering the material

Maintaining that knowledge

Mastering the Material   We think we’re good at this

  We did well on the MCAT, we pass our exams

  But these are graded on a curve!   Patient outcomes don’t work on a curve

  In reality, we do poorly at judging how well we’ve mastered specific topics

Mastery: The Problem

32% of the

reviewed material is

immediately forgotten.

Mastery: The Problem

Maintaining Your Knowledge   We rarely focus on knowledge maintenance

  Johns Hopkins study, the ATLS study   Affects patient outcomes

  For Step 1, the review process involves re-learning a lot of material that you once knew but have since forgotten

Intro to Gunner Training

  The first Knowledge Management platform   Created to help students master and maintain their Step 1

knowledge base

  Personalized: users choose which subjects to study can correlate with school's curriculum

  Dynamic: custom review schedule changes according to student's forward learning

  Mobile:   iPhone/mobile browser compatible   No software installation required

Intro to Gunner Training   Launched April 2009:

  Hundreds of users   Thousands of data points

  We use this data and user feedback to continually validate and improve Gunner Training

The GT Way 1.  Learn the material from flashcards •  High-yield, interactive

2.  Immediate testing on that material •  Open-ended study questions

3.  Spaced Learning algorithms create a personalized review schedule for that material

•  Incorporates Step 1 style multiple choice questions •  Short intervals help you master that material •  Longer intervals maintain your knowledge

4.  Exam simulation •  Eases anxiety •  Increases test-taking stamina

The GT Way 1.  Learn the material from flashcards •  High-yield, interactive

2.  Immediate testing on that material •  Open-ended study questions

3.  Spaced Learning algorithms create a personalized review schedule for that material

•  Incorporates Step 1 style multiple choice questions

•  Short intervals help you master that material

•  Longer intervals maintain your knowledge

4.  Exam simulation •  Eases anxiety •  Increases test-taking stamina

Apply 19 principles of

efficient, effective learning

GT: Flashcard Learning Principles 1.  Contiguity Effects. Ideas that are associated should be

presented contiguously in space and time   One flashcard per topic, rather than one-fact-per-flashcard

2.  Dual Code and Multimedia Effects. Visual + multimedia form richer representations vs. a single medium

  Hundreds of relevant images: gram stains, CT scans, X-rays, clinical photos

3.  Coherence Effect. Materials should explicitly link related ideas

GT: Flashcard Learning Principles 4.  Segmentation Principle. A complex lesson should be broken

down into manageable subparts   Two levels of bulletpoints

5.  Manageable Cognitive Load. The information presented to the learner should not overload working memory

  Second bulletpoints are initially hidden

6.  Cognitive Flexibility. This improves w/ multiple viewpoints linking facts and deep conceptual principles

  Material presented in several locations   Ex: TB is covered under Microbiology, Pulmonology, Immunology

7.  Anchored Learning. Materials are anchored in real world problems that you care about

  Clinical correlation

Contiguity, Dual code/Multimedia, & Coherence Effects

Segmentation & Manageable Cognitive Load

Cognitive Flexibility

Anchored Learning

GT: Study Question Principles 8.  Imperfect Metacognition. We rarely have an accurate

knowledge of our cognition need assistance with calibrating comprehension, learning, and memory

  Immediate testing solves this

9.  Generation Effect. Learning is enhanced when learners produce answers (vs. recognizing answers)

  This is why GT employs open-ended study questions

10.  Testing Effect. Testing enhances learning, especially when the tests are aligned with important content

GT: Study Question Principles 11.  Feedback Effects. Students benefit from immediate

feedback on their performance 12.  Negative Suggestion Effects. Learning wrong information

can be reduced when feedback is immediate 13.  Deep questions. More benefit from answering questions

that elicit explanations (e.g., why, why not, how, what-if)

Imperfect Metacognition, Generation Effect & Testing Effect

Feedback Effects & Negative Suggestion Effects

GT: Review Schedule Principles 14.  Goldilocks Principle. Assignments should be at the right

difficulty level 15.  Self-regulated Learning. Most students need to self-

regulate their learning 16.  Exam Expectations. Students benefit more from

repeated testing 17.  Spaced Learning.

Goldilocks, Self-regulated Learning, and Exam Expectations

GT: MCQ Principles 18.  Desirable difficulties. Challenges make learning and

retrieval effortful improve retention   Multi-part questions, on par with Step 1 question difficulty

20.  Cognitive Disequilibrium. Deep reasoning and learning is stimulated by problems that create cognitive disequilibrium, such as obstacles to goals, contradictions, conflict, and anomalies

  “2-jump” & “3-jump” questions

MCQs in Review Schedule

MCQs in Review Schedule

Spaced Learning in Gunner Training Personalized Review Schedule Generation

Spaced Learning   In 1913, Ebbinghaus discovered that the best time to

review something was just before it was forgotten

  Minimizes the amount of time spent learning   Minimizes the number of times the same material must be

relearned

Spaced Learning: Example   Let’s say it takes 2 minutes to memorize the 15 bones of

the wrist   Reviewing this material if it’s not forgotten takes 30

seconds   Re-learning this material if it’s forgotten takes another 2

minutes   Without review, this information is forgotten in 4 days   There is a test at the end of Day 6

Spaced Learning: Example   Option 1: Review it every day

  Day 0: 2 minutes learning   Day 1: 30 second review   Day 2: 30 second review   Day 3: 30 second review   Day 4: 30 second review   Day 5: 30 second review   Day 6: 30 second review Take the test

  Total time: 5 minutes   Pros: In this approach, the knowledge is never forgotten   Cons: Large burden of review few people will adhere to this

review schedule

Spaced Learning: Example   Option 2: Cram for the exam

  Day 0: 2 minutes learning   Day 1: no time spent   Day 2: no time spent   Day 3: no time spent   Day 4: no time spent (knowledge forgotten)   Day 5: no time spent   Day 6: 2 minutes re-learning Take the test

  Total time: 4 minutes   Pros: It’s “easy” – learn it once and re-learn it just before the

test   Cons: The knowledge is forgotten

  Cramming produces high stress   Doesn’t work in clinical settings (no predetermined “test” date)   Less effective as the volume of information goes up

Spaced Learning: Example   Option 3: Spaced Learning

  Day 0: 2 minutes learning   Day 1: no time spent   Day 2: no time spent   Day 3: no time spent   Day 4: 30 second review   Day 5: no time spent   Day 6: Take the test

  Total time: 2 minutes, 30 seconds   Compared to daily review (5 mins): 50% less time spent   Compared to cramming (4 mins): 38.5% less time spent

  Pros: In this approach, information is never forgotten and time spent studying is minimized

Spaced Learning Works in Medicine   3 randomized, controlled trials at Harvard Medical School

demonstrate that Spaced Learning improves retention of medical knowledge   RCTs are the gold standard experimental methodology   Published in:

  American Journal of Surgery, January 2009   Journal of General Internal Medicine, January 2008   Medical Education, January 2007

Basic Research on Spaced Learning   The molecular mechanism behind Spaced Learning is an

active area of research   Cell, October 2009. Cold Spring Harbor & Mount Sinai School

of Medicine   The Phosphatase SHP2 Regulates the Spacing Effect for Long-Term Memory Induction

  Journal of Neuroscience, August 2009. McGill University   PKC Differentially Translocates during Spaced and Massed Training in Alypsia

  Neuron, December 2006. Baylor College of Medicine   Drosophila Mushroom Body Neurons Form a Branch-Specific, Long-Term Cellular

Memory Trace after Spaced Olfactory Conditioning

There is rock-solid science behind Spaced Learning.

Gunner Training Works!

Gunner Training’s Effectiveness

* All data where user completed 4 reviews within the maximum time (115 days) ** Heteroscedastic, one-tailed t-test. Same results with paired, 2-tailed t-test.

*

19.3% (p<0.000001)**

Gunner Training’s Effectiveness

Avg time to 4th review: 26.4 days

Max time to 4th review: 115 days

  Two major successes:   1) Knowledge isn’t lost during >100 days since initial review   2) 19.3% gain in retained knowledge

Summary   GT is the first Knowledge Management specifically

platform for medical students   The application of 19 Learning Principles yields a highly

efficient and effective program   Users who follow our algorithms gain an average of 19.3%

in ability to recall knowledge after 4 reviews   Users don’t forget what they have already learned

Priced for Students

User Comments   “I absolutely loved this site. I

didn't get far into the program because of time constraints, but I really wish I knew about it before starting 2nd year. It would've been amazing to use it while studying for the course. I used to study with RecallPlus and Anki during the course but found it so time consuming to make the cards, so you can say Gunner Training is just perfect.”

– H.A., 2nd year medical student

  “I'll definitely let the 2nd years in my school know about your program. I've already recommended the program to the Dean of Student Affairs at my school.”

– J.W., 2nd year medical student

Anonymous Comments   It's surprising how much I've learned

in such a short time span with what feels like not a lot of effort. I've maybe spent 30 minutes to an hour every other day (sometimes every 2nd or 3rd day) on it and it's gone quickly.

I've been tracking my progress with the analytics it provides. After 2 weeks of studying and a few repetitions my recall has gone up significantly (the amount of 5's has increased by almost 20% - this meaning that when I do my scheduled review questions, the number of answers that I get right without having to think about the question is increasing). ”

AggieSean, on Student Doctor Network

  “On my exam, there were questions that I actually thought, ‘I know that flashcard exactly.’ This is, however, after I had continuously gotten the nit picky factoid wrong on several occasions at Gunner Training. ”

Caboose, on Student Doctor Network

http://www.gunnertraining.com