Modeling the development of mirror neuronsGiorgio Metta, Paul Fitzpatrick • MIT AI Lab, Humanoid Robotics Group • {pasa,paulfitz}@ai.mit.edu

Acknowledgments: This work is funded by DARPA as part of the “Natural Tasking of Robots Based on Human Interaction Cues” project under contract number DABT 63-00-C-10102.

4MimicryThe robot mimics the observed action trying to fulfill the goal rather than an actual movement

Question: How do mirror neurons develop?Answer: Two stage hypothesis: learn about objects and then imitate the goal of an action if directed towards the same object.

Problem

FactsF5 contains two classes of neurons: canonical

and mirror. Both classes respond when object-directed actions are executed.

Canonical neurons respond to the presentation of an object according to the potential grasp types the object affords.

Mirror neurons respond to an observed action in accordance to the grasp type.

1

3

side tap

back slap

pull in

push away

A toy car: it rolls in the direction of its principal axis

A bottle: it rolls orthogonal to the direction of its principal axis

A toy cube: it doesn’t roll, it doesn’t have a principal axis

A ball: it rolls, it doesn’t have a principal axis

Motor vocabulary

identifyand

localizeobject

search rotation

Previously-pokedprototypes

Learn the “geometry” of pokingE.g. poking from the left causes the object to move to the right

-200 -150 -100 -50 0 50 100 150 2000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

direction of movement [degrees]

P(x

)

back slap

-200 -150 -100 -50 0 50 100 150 2000

0.05

0.1

0.15

0.2

0.25

direction of movement [degrees]

P(x

)

side tap

side tap

back slap

Exploiting affordances during pokingWhen a known object is presented to the robot, it chooses the appropriate action to

make the object roll

Clustering

Interpreting observationsA foreign manipulator (human) pokes an object

The direction of movement is compared with the object afforadance

2

Final position

Initial position

Example 1

Final positionInitial position

Example 2

Final positionInitial position

Object, goal connects robot and human action

Our solutionWe used poking and prodding as a precursor to full-blown

manipulation

1. Learn about object affordances (canonical neurons)2. Interpret observations on the basis of affordances (mirror)

3. Poke/push objects according to their affordances4. Mimicry

12

34

Solution

Learn about object affordancesThe affordance considered is the direction of movement with respect to the object’s

principal axis

0 10 20 30 40 50 60 70 80 900

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

difference between angle of motion and principal axis of object [degrees]

estim

ate

d p

roba

bilit

y of

occ

urre

nce

Rolls at right angles toprincipal axis

Bottle, “pointiness”=0.13

MotivationOur inspiration comes from neuroscience: in particular, mirror neurons. This is a class of neurons found in the monkey's frontal cortex (area F5). A particular mirror neuron is activated both when the monkey executes an action and when it observes the same action performed by somebody else. Mirror neurons can be related to gesture recognition, language, and learning by imitation.