M nolin pkerfoot spbt13_m_learning knowledge retention

11

STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS

Michelle Nolin, CPLPSr. Director, Product Strategy & Solutions

Informa Training Partners

Dr. B. Price KerfootAssociate Professor

Harvard Medical School

2STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS

LEARNING OBJECTIVES

Identify pedagogically sound strategies for mLearning design

Appreciate findings from the latest instructional design research for mLearning

Identify techniques for mLearning that ensure knowledge and skills are gained, retained, and transferred

Apply 10 Tips for Knowledge Retention with mLearning


What your learners really want on their tablets and phones…


What your learners don’t want…

TAP PINCH-ZOOM

And then… READ,SCROLL,

READ, SCROLL, READ, SCROLL, READ,

SCROLL, READ, SCROLL, READ SCROLL


Are you (and your learners) a…

Digital native OR digital immigrant?


What is mlearning?

M = Mobile

DEVICE (Hardware)Smartphones, iPads, other

tablets

TECHNOLOGY (Software)

OS: Apple and DroidDozens of authoring tools

CONNECTIVITYCellular networksInternet/Intranet


eLearnng OR mLearning?

eLearning

• Real-time or self-paced• Synchronous or

asynchronous• Tethered (via Web,

Internet)• Structured, formal

(GUI, architecture, etc.)

mLearning

• Self-paced• Un-tethered• Informal• Unstructured, just-in-

time, field-based• Anywhere, anytime


Polling Question

Are you using mLearning in your organization today?

YES

Not yet, but actively working on mLearning initiative

Maybe next year

No plans to use mLearning


Challenges of mLearning

Device variability

Connectivity

Screen size and readability

Text entry

Limited memory

It’s MOBILE (and we work in a regulated industry)

REF: Elias T. Universal Instructional Design Principles for Mobile Learning. IRRODL.12, No 2, 2011.


Poll

What are some of the challenges you face with mLearning in your organization?


Unique opportunities

Mobile devices are inexpensive and pervasive

Handle media well

Continuous, just-in-time • In-the-field training and support• “Situational dependent knowledge”

Rapid and urgent


Best uses of mlearning

Designed correctly, mLearning IS a knowledge retention tool.

• LCOs: think “topics” NOT “courses”• Job aids/support tools (quick “look ups”)• Just-in-time training• Rapid deployment of timely content• Continuous touch points with your learners• Evaluations, assessments, and remediation


Drive Retention with the mLearning NINE

Start (so you can end with) your desired

outcomeBring it close to task Consider Bloom’s

taxonomy

Set clear objectives—and meet them

Match the objective to the device Engage learners

Establish logical architecture and

navigation—and stick to it

Establish and follow good design standards

Meet the learner where they live


Start with the desired outcome.

Instructional technology is great, BUT… it’s not about the tool, or the platform. What outcome do you need to achieve?

Content

Calendar

Culture Cost

4Cs of Training


Bring it close to task.

Examples


Consider Bloom’s taxonomy.

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge

Lower order thinking skills

Higher order thinking skills


Set clear objectives (and meet them).

Outcome

PerformanceLearning


Match the objective to the platform.

Not all objectives can be achieved on a mobile device

Some learning/performance objectives are better suited for a tablet, others on a smartphone


Engage learners.

Draw the learner in… limit passive “push down” of content

Use media where appropriate• Audio• Video• Animations

Link mLearning with live interaction and collaboration

Rethink assessments: gamify it


Establish logical architecture and navigation (and stick to it).

Design an intuitive GUI with multiple navigation paths through the content• Let the learner drive and control access

Design for cross-platform, cross-device use• Use responsive design• Optimize for touch screen environments


Establish and follow good design standards.

Micro chunk the content

More images and media, less text

Use large, easy to read fonts

Avoid “porting” existing text/elearning content—design and develop from the ground up for mobile devices

Design according to the aspect ratio of the device (not your computer monitor)


Meet the learner where they live.

mLearning allows us to meet learners where they live – on their PHONE and TABLETS.

• Cell phones = global population • ½ the US population has smartphones• Survey: 4 out of 5 people 18-44 check their

phone within 15 minutes of waking• 84% time spent on email, texting, social

media, only 16% on voice calls

Sticky mLearning: How to Ensure Knowledge Retention

in mLearning Programs

B. Price Kerfoot, MD EdM

VA Boston Healthcare SystemHarvard Medical School

Conflict of Interest Disclosure

• Harvard submitted patent on spaced education methodology.

• Harvard has launched a start-up company Qstream which hosts this methodology outside of the Harvard firewalls.

• As the “inventor”, I am an equity owner and director of this company.

Multi-institutional randomized controlled trial of 351 medical students at 4 northeastern medical schools

Acad Med. 2006;81(3):224-30.

Multi-institutional randomized controlled trial of 351 medical students at 4 northeastern medical schools

Acad Med. 2006;81(3):224-30.

Retention of BLS and ACLS Training Among Nurses

Resusc 2008; 78: 59-65

n=133

Lammers (Michigan State Univ) Acad Emerg Med 2008; 15: 1181-89

Hermann Ebbinghaus (1850-1909)

In 1885, he published "On Memory”

Forgetting curves

Learning curves

Spacing effect

Spaced Education: the basics

• Spacing Effect– spaced– repeated– repeated at increasing intervals

- Increased efficiency of learning- Reduction in the slope of the forgetting curve

Sisti et al, Learning & Memory 2007.

Pagani et al, Cell 2009.

Xue et al, J Cognitive Neuroscience 2010.

Psychol Sci, 2006; 17: 249

Science 2011; 331: 772-5.

“Testing effect”Retrieval practice

Psychol Sci, 2006; 17: 249

Spaced Education Delivery System:harnesses both the spacing & testing effects

Spaced Education Delivery System

VA PSA Screening Spaced Education Trial



It Works!

• Increases knowledge & retention- Medical Education 2007: 41:23-31 --- UGME- Journal of General Internal Medicine 2008; 23(7):973-8 --- UGME - Academic Medicine 2011 (in press) --- UGME - Journal of Urology 2007; 177, 1481-1487 --- GME– Journal of Urology 2009; 181, 2671-2673 --- GME– Journal of the American College of Surgeons 2010; 211: 331-72673 --- GME– Annals of Surgery 2009; 249: 744–749 --- CME

• Improves self-assessment of knowledge– American Journal of Surgery 2009; 197(1):89-95

• Changes behavior– American Journal of Surgery 2009: 197(2), 252-257– American Journal of Preventive Medicine 2010; 39: 472-8

• Is well-accepted by learners- Demonstrated in all trials to date

Interactive Spaced Education Versus Web-based Modules for Teaching Histopathology Diagnostic Skills:

a Randomized Controlled Trial

B. Price Kerfoot MD EdM, Yineng Fu MD, Michael Ritchey MD, Donna Connelly MS & Elizabeth Genega MD

VA Boston Healthcare SystemHarvard Medical School

Beth Israel Deaconess Medical CenterAmerican Urological Association

J Am Coll Surg 2010;211:331-7

ISE Delivery System: (2) the question

Spaced Education on GU Pathology

Weeks

0 10 20 30 40 50

Pe

rce

nta

ge

Sco

res

50

60

70

80

90

100

Web-based Teaching Modules

Module 1

Module 2

Module 3

OC-1

OC-2OC-3 OC-4

J Am Coll Surg 2010; 211: 331-7

Spaced Education on GU Pathology *

Weeks0 10 20 30 40 50

Pe

rcen

tage

Sco

res

50

55

60

65

70

75

80

Interactive Spaced Education

Cycle 1

Cycle 2

Cycle 3

OC-1

OC-4OC-3

OC-2

78% preferred SE over WBT 580/583 requested more SE

J Am Coll Surg 2010; 211: 331-7

Spaced Education Progress Testing (SEPT) of Medical Students:a Randomized Controlled Trial

B. Price Kerfoot, MD EdM, Kitt Shaffer MD PhD, Graham T. McMahon MD MMSc, Harley Baker EdD,

Edward Krupat PhD, Jamil Kirdar, MBChB MRCP, & Elizabeth G. Armstrong EdD

Harvard Medical School

Acad Med 2011, 86(3): 300-306

SEPT: Study Structure

• University of Virginia School of Medicine• University of Pittsburgh School of Medicine• Harvard Medical School• University of Alabama School of Medicine

Acad Med 2011, 86(3): 300-306

SEPT: 120-item assessment tool

Year of Training0 1 2 3 4 5

Perc

enta

ge o

f It

em

s C

orr

ect

35

40

45

50

55

60

65

All 120 Items

Acad Med 2011, 86(3): 300-306

SEPT: 40-item assessment of retention

Acad Med 2011, 86(3): 300-306

Weeks0 10 20 30 40

Per

cent

age

Sco

re

30

40

50

60

70

80

90

LPTASEPT

p<0.001effect size=0.95

2-weekcycled-review

6-weekcycled-review

Items 1-40initial presentation

Final cycle to assess

learning retention

SEPT: 40-item assessment of retention

J Cont Educ Health Prof, 31(2):103–108, 2011

74% of participants (181/246) completed the SE program. Of these, 97% (176/181) submitted the behavior change survey.

This randomized controlled trial was conducted from March 2009 to April 2010, immediately following the PriMed live CME conference in Houston, Texas.

J Cont Educ Health Prof, 2011; 31(2):103–8

86% agreed or strongly agreed that the SE program enhanced the impact of the live CME conference.

97% requested to participate in future SE supplements to live CME courses.

J Cont Educ Health Prof, 2011; 31(2):103–8

Interactive Spaced Education to Improve Prostate-Specific

Antigen Screening:a Randomized Controlled Trial

B. Price Kerfoot MD EdM, Galina Sokolovskaya MS

Elizabeth Lawler MPH ScD, David Gagnon MD PhD,

& Paul R. Conlin, MD

VA Boston Healthcare System

Am J Prev Med 2010; 39: 472-8


Am J Prev Med 2010; 39: 472-8


93% requested to participate in future spaced education programs

Spacing

Weeks0 10 20 30 40

Per

cent

age

Sco

re

30

40

50

60

70

80

90

LPTASEPT

p<0.001effect size=0.95

2-weekcycled-review

6-weekcycled-review

Items 1-40initial presentation

Final cycle to assess

learning retention

Weeks0 10 20 30 40 50

Pe

rce

nta

ge

Sco

res

50

55

60

65

70

75

80

Interactive Spaced Education

Cycle 1

Cycle 2

Cycle 3

OC-1

OC-4OC-3

OC-2

Reinforcement

Effective after a face-to-face eventDurable behavior change


Spaced Education Delivery System

Lessons from Harvard: Using Gamification to Juice

Your Sales Training


CHOPPED ACTIVITY

Break into small groups

Ingredients you need are in your “inbox”

Cook up an mLearning intervention

Present Your DISH to the larger group


10 Tips for Knowledge Retention with mLearning

1. Select the right content at the right time for the right reason in ALL mLearning.

2. Don’t let technology be the tail that wags the dog… desired outcome should drive your mLearning initiatives.

3. Couple engagement through mobile devices with the principles of spaced education to boost knowledge retention.



4. Investigate DIY, easy-to-learn, and cost-effective mLearning tools to create continuous touch points with your learners.

5. Think beyond “just-in-time” training and performance support for mobile learning… designed well, mLearning boosts knowledge retention.

6. Use mLearning to meet learners “where they live,” providing them with opportunities to access content when they need it, confirm what they know (and don’t know), and retain their knowledge.



7. Integrate mLearning tools with virtual instructor-led training (VILT) to make distance learning intimate.

8. Integrate mLearning onsite at live meetings for learner engagement.

9. mLearning is not reading on your phone… deliver learning interactions on mobile devices to engage learners, foster learning transfer, and sustain desired outcomes.

10. Recognize how digital natives and digital immigrants, like to learn, and deliver training on mobile devices that balances the needs of these two learner types.


Questions?

SPEAKER EMAILS:

Michelle Nolin: [email protected]

Dr. Kerfoot: [email protected]

mailto:[email protected]

Business

M nolin pkerfoot spbt13_m_learning knowledge retention