39
Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRS Bayesian Action-Perception model Bayesian Action-Perception loop modeling: Application to trajectory generation and recognition using internal motor simulation E. Gilet (1) , J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1) (1) Laboratoire d’Informatique de Grenoble – CNRS, France (2) Laboratoire de Psychologie et NeuroCognition – CNRS, France July, 5, 2010 http://diard.wordpress.com/ [email protected]

E. Gilet (1) , J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

  • Upload
    jethro

  • View
    18

  • Download
    2

Embed Size (px)

DESCRIPTION

E. Gilet (1) , J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1) (1) Laboratoire d’Informatique de Grenoble – CNRS, France (2) Laboratoire de Psychologie et NeuroCognition – CNRS, France July, 5, 2010 http://diard.wordpress.com/ [email protected]. - PowerPoint PPT Presentation

Citation preview

Page 1: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Bayesian Action-Perception loop modeling: Application to trajectory generation and recognition using internal motor simulationE. Gilet(1), J. Diard(2), R. Palluel-Germain(2), P. Bessière(1)

(1) Laboratoire d’Informatique de Grenoble – CNRS, France(2) Laboratoire de Psychologie et NeuroCognition – CNRS, France

July, 5, 2010http://diard.wordpress.com/ [email protected]

Page 2: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception of actions

2

(Calvo-Merino et al., 2004)

Page 3: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Reading and writing letters

3

(Longcamp, 2003)

Writing

Reading pseudo letters

Reading letters

Page 4: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Interpretation• Motor simulation of actions during

perception

• Articulation between perception and action processes

4

Page 5: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Modeling both reading and writingModeling internal simulation of

movements

5

Page 6: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Bayesian Action-Perception (BAP) model

6

Page 7: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

7

Page 8: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

BAP model structure

8

internal letter representation

perception model

action modelsimulated perception model

coherence variables

Page 9: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Cartesian and effector spaces

• Common space for perceptive and motor internal representations– Cartesian space

9

Page 10: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Letter representation: sequences of via-points

10

Page 11: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

11

Letter representation

« Laplace succession laws »

Page 12: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Parameter indentification

12

Page 13: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

13

Page 14: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

14

Page 15: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

15

Page 16: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

16

Page 17: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception model

17

• Deterministic via-point extraction

Page 18: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

18

Action model

Page 19: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

19

Trajectory generation model

• Minimum-acceleration model:– Cost function– Boundary conditions

• Polynomial solution

Page 20: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

20

Simulated perception model

• Identical to the perception model

Page 21: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

21

Coherence variables

• Allow to activate or deactivate submodels– « Bayesian switch »

Page 22: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Coherence variable for controlling submodel activation

• Model– λ binary variable– Joint–

• Inference– P(A) = P(A): value of B does not influence A–

22

A B

λ

A B

A B

Page 23: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

23

Page 24: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception: reading letters

24

Correct recognition: 93.36%

Page 25: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception: writer recognition

25

Correct recognition: 79.5%

Page 26: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Action: writing letters

26

Variability between writers Variability between trials

Page 27: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Motor equivalence

27

Page 28: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Motor equivalence• Writer “style”

– (Wright, 1990)• Common

activated motor areas– (Wing, 2000)

28

(Serratrice. 1993)

Page 29: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Action: Motor equivalence

29

Page 30: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

30

Action: Motor equivalence

Page 31: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception and Action: Copy

31

Trajectory copy Letter copy

Page 32: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception and Action: Reading letters with motor simulation

32

Recall: reading letters without simulation

Page 33: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

33

Perception and Action: Reading letters with motor simulation

Page 34: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

34

Perception and Action: Reading letters with motor simulation

• Complete trajectories– Correct recognition score with simulation 93.36%– Correct recognition score without simulation 90.2%

• Incomplete trajectories

Page 35: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

35

Page 36: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Experimental prediction

36

Page 37: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Preliminary data

37

60

70

80

90

Control Group (motor simulation unaffected)

Motor interference Group (motor simulation affected)

Complete letters

Truncated letters

Recognition Performance (%)

F(1,23) = 3.06, p = 0.093

Page 38: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– Bayesian model of perception and action

– Includes an internal simulation loop

• Cognitive tasks– Reading without and with

motor simulation– Writer recognition– Writing with different

effectors– Copying letters and

trajectories• Basis for experimental

predictions38

Page 39: E. Gilet (1) ,  J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Thank you for your attention !

Questions ?