Learning by imitation:Computational Modeling And Robotics
Aude Billard
Computer Science DepartmentProgram of Neuroscience
University of Southern CaliforniaLos Angeles
Robot Learning By Imitation
Teaching a robot complex motor skills by demonstration
What Does It Take To Imitate?
• What should we imitate?
• Which features of the action are relevant?
• What should we pay attention to?
Finding the goal of the action
Grasping an object
Relevant Features: Hand-Object relationship
To what extend is biological inspiration useful?
Is a model of human imitation useful for robotics?
The imitator robot
Should the robot have the same body configuration?
EPFL, ASL Pygmalion RobotLausanne, Switzerland
Kawato Erato ProjectATR, Kyoto, JapanHumans and robots have different body dynamics
Learning by imitation: Motivations
Robotics: A means of transmitting motor skills - Coordinated behavior, implicit attentional mechanism - Natural means of interaction - No need of explicit programming
Biology: Computational Neuroscience - Abstract model of primate ability to imitate
- Neural mechanisms behind learning by imitation
- Cut down the debate concerning imitation
RoboticsMotion Studies
Computational Modeling
Imitation Learning
Modeling Implementing
From Human To Robot
Motion studies on human imitation
Recording and analysis of kinematics of full body motion
Collaborations:
James Gordon, Department of Biokinesiology & Stefan Schaal, CS dept, USC Steve Boker, Univ. of Notre Dame, Indiana
Visual and motor representation of movements:
Sensitivity to body cues Orientation of body, direction of limb motion Eccentric versus intrinsic space Tracking hand path versus joint angles
Motion studies on human imitation In collaboration with James Gordon, Department of Biokinesiology, USC
Steve Boker, Univ. of Notre Dame, Indiana
Hypotheses:
Imitation is based on a hierarchy of goals.It can be goal-directed, exact, partial.
The metric is task-dependent.
Bekkering, Wholschlager & Prinz, Psycholoquia 2000
Question I : What are the metrics behind imitation?
Question II: What mechanisms are behind the immediate body to body mapping?
Hypothesis
Biological clues: body symmetries, limb orientation
Results:
Bias for mirror imitation. Two transformations of frames of reference only.
Hypothesis:We have a model of the human body kinematics and dynamics
Question III: How do we recognize biological motions from non biological ones?
Results:
Hand path only is too ambiguous an information to reconstruct completely the motion.
Question IV: Which representation of movement?
HypothesesReconstruction is based on a model of natural motion
Basic, primitive patterns of motions: Coupled oscillation of limbs
Results:
No significant effect of display type on performance.
Poor performance on in-phase/anti-phase patterns: bimanual coordination
Non leading arm tends to produce the closest preferred pattern
Motion studies on human imitation
Experiments: 1. Goal-directed: grasping, kicking an object2. Functional: Tying shoes, stacking boxes3. Abstract: dance, highly skilled motion
Imitation Learning
Learning new motions requires both eccentric and intrinsic information, as well as information on amplitude, speed, acceleration.
Task-dependent method of analysis1. Eccentric: End-point trajectories
Principal Component Analysis
2. Egocentric: joint trajectories Cross-correlation, phase shift
RoboticsMotion Studies
Computational Modeling
Imitation Learning
Modeling Implementing
From Human to Robot
Neural mechanisms behind learning by imitation Hypotheses: 1. Common parametrisation to visual and motor systems
- Body-centered reference frame
- Coding of mvt in orientation, amplitude and speed
- Mirror Neurons: visuo-motor mapping
2. Dynamic learning of motor commands
- Coarse coding of information, movement sequence
- Adaptation and combination of basic motor patterns
Frontal Lobe:Decision CenterInhibition of motion
Pre-Motor Cortex / Broca’s area: Visuo-motor transformation / Mirror Neurons
Temporal Lobe (STS):Eccentric – Intrinsic visualRepresentation of movement
Parietal Lobe:Eccentric visual coding
Cerebellum:Timing, Sequencing
SMA:Sequence learning
Spinal Cord + Brain Stem:Basic motor patterns, CPGLocomotion, Reflexes
Motor CortexSomatotopic control
High-Level representation of the brain mechanisms underlying imitation
Functional and abstract model of the brain areas and their connection
Cerebellum:Timing, Sequencing
Frontal Lobe:Decision CenterInhibition of motion
SMA:Sequence learning
Pre-Motor Cortex: Visuo-motor transform
Spinal Cord Basic motor patterns, CPG
Motor CortexSomatotopic control
Temporal Lobe (STS):Eccentric – Intrinsic visual
Parietal Lobe:Eccentric visual coding
High-Level representation of the brain mechanisms underlying imitation
Functional and abstract model of the brain areas and their connection
Brain Stem
Schematic of the model
Segmentation
Joint Angle
Filtering of Small Motions
Neural Output
Visual Processing
Motor Control
Leaky-integrator neurons
Spring and Damper Muscle Model(Lacquaniti & Soechting 1986)
Flexor-extensor pair per degree of freedom (DOF)
41 DOFs simulator, 30 DOFs humanoid robot
Visuo-Motor Transformation
Visual Module
Learning Module
Motor Module
Fixed transformation
First order approximation of inverse dynamics
Visuo-Motor Module
Example: Imitating Human Arm Motions
Imitation of Gestures
Gesture 1 Gesture 2 Gesture 3
•Recurrent NN•Sequence learning•Generalization across movements
DRAMA: Dynamical Recurrent Associative Memory Architecture
- Fully recurrent NN with self connections on each unit
- Time-delay neural network: Learning of complex time series and of spatio-temporal invariance - Hebbian Learning: on-line and on-board robot learning
DRAMA: Unit Activation function
Decay of activityInput
Thresholds
Thresholds
CLMC-LAB
Learning of a dance movement sequence
Human DemonstrationReplay of Recordings
2nd pattern1st pattern
Learning actions: one by one
Joint by joint segmentation
3rd action pattern
CLMC-LAB
Learning of a dance movement sequence
1st pattern
Learned actions lead to the following postures
Learned Posture 1
2nd pattern
Learned Posture 2
Learning sequences of actions
ActionE
ActionC
ActionD
ActionB
ActionA
Learned Motor Programs
ActionA
ActionC
ActionE
ActionB
ActionD
Learning sequences of actions
ActionE
ActionC
ActionD
ActionB
ActionA
ActionA
ActionC
ActionE
ActionB
ActionD
Fully Recurrent NNConnectivity is built on-line
Time-delay neural networkLearn sequencing and timing of the action sequence
Imitate sequences of actions
ActionE
ActionC
ActionD
ActionB
ActionA
ActionA
ActionC
ActionE
ActionB
ActionD
Improvise using the learned sequences of actions
ActionE
ActionC
ActionD
ActionB
ActionA
ActionA
ActionC
ActionE
ActionB
ActionD
Randomly activate or shut down nodes to produce new action sequences
Modeling: Summary
• Data Segmentation: Finding the key features of motion
Change in speed and orientation, joint-based representation
• Common parametrization of movements: visual and motor systems Speed and direction of movement, joint-based representation
• Reconstruction of movements: robustness against perturbation
• Learning of actions: synchronous and sequential activations of limbs
• Recombination of basic movements: improvisation
RoboticsMotion Studies
Computational Modeling
Imitation Learning
Modeling Implementing
From Human to Robot
Robota
Robota Clever Toy and Educational Toy
Robota
First Prototype
Univ of Edinburgh, 1998
Second Prototype
LAMI - EPFL, 1999
In collaboration with Jean-Daniel Nicoud and Andre Guignard
Robota – The Product
DIDEL SA, Switzerland Jean-Daniel Nicoud, director of DIDEL SA
ROBOTA: TECHNICAL SPECIFICATIONS
Power Supply Connector
Left Leg Motor
Head Motor
Right Arm Motor
Left Arm Motor
Right Leg Motor
Head Potentiometer
Right and Left Arm Potentiometers
Fixation points for the stand
ROBOTA: TECHNICAL SPECIFICATIONS
MOTOR BOARD: RS232 Connector to PC
MOTOR BOARD: RS232 connector to Kameleon Board + Jumper Selector
SENSOR BOARD: SPI connector across Motor and Sensor Board
SENSOR BOARD: Spare SPI Connector
SENSOR BOARD: Buzzer
SENSOR BOARD: 3x 2-pin connector for head, arms and legs switches
SENSOR BOARD: Connector for 2x 2-pin analog input (e.g. IR RX)
SENSOR BOARD: Connector for 2x 3-pin analog output (e.g. IR TX)
ROBOTA: TECHNICAL SPECIFICATIONS
Fixation points on Motor Board
Fixation points on Battery board
Fixation points on Kameleon Board
Robota battery set attaches to the back of the Motor Board7.2V,7x1.2NiCd
Robota at the Museum
La Cite des Sciences et de l’Industrie, French National Science Museum
November 2001 – July 2003
Robota: Applications I
French National Science Museum - La Cite des Sciences et de L’industrie
LANGUAGE GAME
Robota: Applications III
Aurora Project Dr. Kerstin Dautenhahn,
Univ. of Hertfordshire
www.aurora.com
Center for Fundamental Infant Development
Drs Demuth, Pena, Bradley, Turman
USC Dept of Biokinesiology and Physical Therapy
USC Premature Infant Follow-Up Pediatric Clinic
Mechatronics: Programming Humanoids Robots
USC – CS499
Undergraduate Computer Science Degree (4th Year)
30 Students. Equipment (Robots + PCs) supported by a «Innovative Teaching » grant from Intel Corp.
ROBOTA Class SyllabusAssignment 2
“Conversational Robot"
ROBOTA Class SyllabusAssignment 4
“Imitator Robot"
Robota drives a car
Carbota
Robota leg motors are wired to the remote control car motors:
Kameleon 376 Processor Board (K-Team) is attached to the front of Robota
Carbota
So pretty! Robota sitting in the remote controlled car.
Final projects
Robota Counting Game Robota Learns to Dress up
Robota Does Cross-Country skiing
ROBOTA UserGuide
Webots-Robota : The SimulatorWebots is a product of Cyberbotics (www.cyberbotics.com)
RoboticsMotion Studies
Computational Modeling
Imitation Learning
Modeling Implementing
From Human to Robot
Summary
Biology Robotics
Take inspiration from Biology and Psychology to design robots
which can learn from interacting with humans and with other robots
Keywords: Learning, human-robot and multi-robot interactions
To use robotic tools (robots and realistic simulations) to build biologically plausible, computational models of animal ability.
Keywords: Computational neurobiology, biologically inspired robotics
Biology Robotics