Multimedia Learning 

Leaning and understanding fromLeaning and understanding from animation and simulation

Jean Michel BoucheixLEAD CNRS U i it f BLEAD, CNRS, University of Bourgogne

National Seminar, Gunadarma University, 2‐2‐2011 

S h d lSchedule

f• 1‐ Cognitive processing of Animation in multimedia documents

2 L i ith i l t• 2‐ Learning with simulators  

Part I‐ Animations in learning dynamic processes from multimedia presentation

• Explosion of the numbers of animations (and video) in electronic documents and e‐learning

• Animation are more concrete than static pictures (or than a lot of words )• Animation are more concrete than static pictures (or than a lot of words )

• More realistic: step of the process

• Elicit more engagement and motivation than static pictures

• Are animations better and more efficient for learning and understanding than static pictures , in school and in professional learning ?

Wh t iti h t l d l b ilt f i ti ?• What cognitive processes: how mental models  are built from animation? 

Does animation facilitate learning, doing  procedures and comprehension?

• N t l (T k M i & Bét t 2002 Bét t• Not always (Tversky, Morrison & Bétrancourt, 2002, Bétrancourt, 2005)– Experimental results not in favor of a benefit in learning from animation 

compared to static picturescompared to static pictures• Höffler & Leutner (2007)

– Meta‐analysis of the research (26 studies, 76 pair‐wise comparisons) about the comparison between static pictures and animation for the sameabout the comparison between static pictures and animation for the same topic

– Advantage (positive effect, medium size, d  = .37) of instructional animation over static pictures in the comprehension of a dynamic process p p y por phenomenon.

Why does animation not always help learning?Why does animation not always help learning?

Intuitively designed, without scientific grounding  

When to look, where to look, what to look and how?

• Perceptual salience / thematic relevance

• TransienceAttention difficulties

Transience• Speed (high)• Attention direction and share

Cognitive difficulties : cognitive load and limited

• Expressiveness of the animation

• Apprehendability

working memory resources

Conceptual difficulties: priorpp y• Information search• Visual search (text and picture)• Simultaneous movements

Conceptual difficulties: prior knowledge

• Simultaneous movements

Difficulties mainly related to presentation format and information delivery.Can we improve the animation efficiency for learning purpose: yes we can?

How to improve animation presentation and design?How to improve animation presentation and design? 

1. Presentation format and information organization

2. Interactivity with the computer and user-control of the animation

3. Attention guiding and signaling (cueing techniques)

4. Providing strategies

5. Segmentation and speed of the animation

Experimental methodExperimental method

• a Individual pre test for prior knowledge• a‐ Individual pre‐test for  prior knowledge

• b‐ Individual study of the animation presented on the computer (time constraints , compulsory tasks or free study)

• c‐ Post‐test : Recall, comprehension measures (or production test),  transfer tests.

Eye‐tracking during learning

ASL Mobile Eye (60 Hz)

ASL 5000 ou Eye link, high speed,Corneal reflect and pupil delation Tobii

ExampleExample

Format

Boucheix & Schneider Learning and Instruction 2008Boucheix & Schneider, Learning and Instruction, 2008

Interactivity and User Control

Boucheix, Learning with animation, 2008

User Control

40

50

60

70

Score 0,50,60,70,80,9

0

10

20

30 Temps

00,10,20,30,4

,

Dynamique Contrôle Direct Contrôle indirect

Présentation

Interactive No-interactive

Knots performance

Three days after

Direct  learners attention on relevant location of the animation 

Signaling  with cueing techniques

Schneider & Boucheix, 2008 Kriz & Hegarty, 2007 Lowe & Boucheix, 2007

Adding arrows did not enhance animation processing

Schneider & Boucheix, 2008, Applied cognitive Psychology

HammerString Hammer

Damper Damper

Balance

Hammer‐ButtHigh perceptual 

salienceBalance

Back‐checkJack

Spoon

KeyWhippen

Spoon

New cueing techniques: the upright piano mechanism example 

Low perceptual salience buthigh thematic relevance  

Upright piano mechanism

Whippen

Upright piano mecahnism

16

18

20

12

14

16

8

10

DV

_1

2

4

6

EX-DES1 EX-DES2

AOI 2 3 4 5 6 7 8 9-2

0

2

AOI 2 3 4 5 6 7 8 9High thematic 

lAOI: 2 3 4 5 6 7 8 9 AOI: 2 3 4 5 6 7 8 9 RelevanceComponents of the pianoExperts 

(piano repairer)Novices(students) (piano repairer) ( )

Eye fixation of the components of the pianoEye fixation of the components of the piano

Boucheix, Lowe & Soirat, 2006

New cueing techniquesConditions

Control, no‐cues

Standard arrows cues

Spreading colour cues (causal chain)

P t l

Boucheix & Lowe, 2010

Perceptual 

Salience +

Perceptual 

Salience‐

70Dynamic signal, Spreading color cue

(along causal chain)

40506070

40

50

(along causal chain)

10203040

Saliency +

Saliency ‐ 20

30

40

Score /60

010

No‐cueing Spreading  0

10

/

cue

% eye fixations 

No‐cue With cue

Comprehension score

3D better than 2D3D better than 2DPutri, Dian‐Kemala & Boucheix, J.M., 2010

Giving strategies

1‐ Exposure 30’’p

2‐Study 60’’

3‐Motion‐ProductionPost test 30’’Post test 30’’

4‐ Recognition Post‐test, <30’’Fish Locomotion project, Tübingen, 

Boucheix & Lowe, 2009

Pure eel to pure trout Cont. Head Body Tail pCont. Head Body Tail

1,4

1,0

1,2

ExperimentalGroups are

agree

0,8

ee

Groups areSignificantly  better thanControl group

% who

 a

0,4

0,6

% w

ho a

gre g p

in recognizing a trout:

Clips 6,7,8! 

0,0

0,2 (p<.023)

The best are 

-0 ,2

,Body and 

Head groups 

“Does this fish swim like a trout?” cl ip 1 cl ip 2 cl ip 3 cl ip 4 cl ip 5 cl ip 6 cl ip 7 cl ip 8

%T ROUT

-0,4

A crucial perceptual and cognitive problem: The  learner’s segmentation process of the events of the animation

Animation and pictures used for rehabilitation in old people with memory  disabilities (mental simulation)p p y ( )

Pozzo, Boucheix & F i 2006 2011Fourier, 2006‐2011

Animation in public information comprehensionThe railway disruptions messages, LEAD & INRETSy p g ,

70

80

90

30

40

50

60

Deaf 1

Deaf 2

0

10

20

30

Old

Static Simul. Static Seq. Static strip Animated

3,5

4

2

2,5

3

,

0

0,5

1

1,5

« Your attention please, contrary to the informationthat has been displayed the train number 3458will not start from platform B, but from platform D»

Static Simul. Static Seq. Static Strip Animation

p , p

Jean-Michel Boucheix, Laurence Paire-FicouLaurent Saby, Aline Alauzet, Fanny Conte, Jonathan Groff & Stéphane Argon

Medical area

url.htm

rao_schneider13.flv

rao_schneider32.flv

Dynamic images of a heartAnimation of the stages of a skin 

disease

Animation controlled with haptic modality

Animation Cognitive Processing ModelPhase 5:  Mental model consolidation

Elaborating system function across varied operational requirementsFlexible high quality mental model

Top downTop downinfluenceinfluence

Phase 4:  Functional differentiationCharacterization of relational structure in domain‐specific terms

g q y

Boucheix

Phase 3: Global characterization

Characterization of relational structure in domain specific termsFunctional episodes

Boucheix &  Lowe2008

Phase 3:  Global characterizationConnecting to bridge across ‘islands of comprehension’ (Fayol, 2004)

Domain‐general causal chains

Phase 2:  Regional structure formationRelational processing of local segments into broader structures 

D i i h k

Phase 1:  Localized perceptual exploration

Dynamic micro‐chunks

Bottom upBottom up Parsing the continuous flux of dynamic information Individual event units

Bottom upBottom upinfluenceinfluence

II From multimedia to simulatorsII ‐ From multimedia to simulators

• Crane drivers

• Physicians in medical emergency  

Snapshots of the crane driver simulator screens

20003000400050006000

ht (k

g) to

be

ansp

orte

d

01000

17 21 25 29 33 37 41 45

Distance (m) on the jip of the crane

Wei

gh tra

Boucheix, 2003Boucheix, 2004, IEE

SALLE 1

C é

SALLE 1

C éSALLE 2SALLE 2

Learning complex medical emergency 

proceduresRoom 2

Room  1TV CameraCaméra

Moniteur

Caméra

MoniteurExpérimentateursExpérimentateurs

procedures with a full scale patient 

simulator

Camera

ScreenExperimenterNInfirmier

Ambulancier

Infirmier

Ambulancier

ppNurse

Nurse

Médecin

Si l

Médecin

Si l

Ordinateur RetourTV

Ordinateur RetourTV

l

PhysicianComputer Feed‐back TV 

SimulateurSimulateurSimulator

E Bonnetain (LEAD)E. Bonnetain (LEAD)C. Avena (CHU)M. Freysz (CHU)

LaerdalUP2FUP2F

Managingg ga cardiac arrest

procedure

Experimental design

Room 1 Room 2

250

300

350

400

)

100

150

200

250

Tim

e (s

ec)

0

50

H. Fibrill Pulse A. Perf A. Adré Adré shot

Experts NovicesExperts Novices

Messant, Bonnetain, Avena,Boucheix & Freysz, Ann.Franç.Anesth. 2008

Longitudinal study: effect of the number and complexity of the learning sessions

Bonnetain, Boucheix, Hamet,  & Freysz, 2010

50

60

70

30

40

50

Session 1

Session 2

Session 3

Session 4

70

80

50

60Session 1

Session 4

30

40

Simple Diff Same Diff DifferentSimple Diff.Same Diff.Different

Screen simulator for medical emergency learning

Bonnetain, Boucheix,  Hamet& Freysz, 2010

Learning Transfer from a screen simulator to a full scale simulator

E.G.l

Micro‐SimLearning

Sim‐ManTest

Class: resuscitation course 

Learning TestParticipantsWhole group28 volunteers

Randomization Two groups

C.G. Micro‐SimLearning

Sim‐ManTest

Medical students

Two groups

ClassPractice  LearningTestPractice

15

20

5

10 G.EXP

G.CONT

0

Perf.SimMan

Learning ecological car driving with attentional guidance  (visual, auditory or haptic signals)(Renault)

Cauchard, Boucheix, & Argon, 2010ANR: OKTAL, Renault, Le2i,  

Eye trackerFace Lab

PeripheraldLEAD. detection

task

Virtual reality for Learning

Boucheix & Bour, 2008 (AFPA & LEAD)

Wave simulator: for Learning and trainingin making a complex gesture: welding

Professional Learning schoolsgWelding with augmented reality

AFPA

Thanks you very much for your attentionattention

Jean‐Michel.Boucheix@u‐bourgogne.fr