Upload
baldwin-holland
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
A Path to Robot Consciousness via Social Cognition: Agency and
Intention Peter Ford Dominey, Robot Cognition LaboratoryStem Cell and Brain Research Institute INSERM U846, France, [email protected]
ComprendreAMORCES
Outline
More questions than answers…. Motivation – Can robotics clarify our ideas about
consciousness? Context – relation of consciousness and meaning Sytem architectures Progress in Meaning and cooperation in robots
Action and language Shared plans “Meaning” Simulation as meaning
Discussion
Motivation – cooperative robots
When a robot can do things like …. Learn the name of an object, Learn a new action with that
object Tell you what it knows Ask questions when it doesn’t
understand Anticipate what you are about
to do … can it be useful in the
scientific study of consciousness? Dominey, Metta, Nori, Natale (2008)
IEEE Int. Conf. On Humanoid Robotics
Context – consciousness meaning and robotics
Jeffrey Gray: Conscious experiences are imbuded with meaning; computers cannot
(without human interpretation) compute meaning; therefore, computers cannont be conscious
Robots differ from computers in that they are endowed with just such behavioral dispositions [to compute meaning]… [but] we should remain doubtful whether they are likely to experience conscious percepts… that will depend on how the trick of consciousness is done.
Daniel Dennett: … set out to make a robot that is theoretically interesting independent
of the philosophical conundrum about whether it is conscious. Such a robot would have to perform a lot of the feats that we have typically associated with consciousness in the past, but we would not need to dwell on that issue from the outset.
Towards a System Architecture of Consciousness
Brain interacts unconsiously with
world Constructs a simulation
Simulation is consiously perceived Public cognitve space Private cognitive space Private bodily space
Real Unperceived External World
The unconsciousBrain
Experienced external world, including body
from outsidePublic cognitive
space
Inner cognitiveexperiences
(thoughts,Images…)
Private cognitive space
Inner bodily sensations, feelings
Private body space
Cybernetic interactionsConstrains
Simulates
Conscious experience
The different spaces of conscious experience, from Gray 2004
System architecture for a Cooperative Robot
Lallée, Lemaignan, Lenz, Melhuish, Natale, Skachek, van Der Zant, Warneken, Dominey (submitted)
Real Unperceived External World
The unconsciousBrain
Experienced external world, including body
from outsidePublic cognitive
space
Inner cognitiveexperiences
(thoughts,Images…)
Private cognitive space
Inner bodily sensations, feelings
Private body space
Cybernetic interactionsConstrains
Simulates
Conscious experience
External world
Cybernetic interaction
Simulates
Linking Grammar Learning with Vision for Event Description and Interrogation
Humannarrator
Vision, "Meaning"Extraction
Spoken Language Interface
(CSLU RAD)
Grammatical Construction Model: Sentence to Meaning
The moon gave the cylinder to the block.The block was gave the cylinder by the moon.The cylinder was gave to the block by the moon.
Event(Agent, Object, Recipient)
E(A,O,R) Sentence
CCD Camera
Gave(moon, cylinder, block)
Dominey & Boucher (2005) Artificial Intelligence
Meaning in Cooperation: Language-Based interaction with the Robot Apprentice
Cooperative Table Assembly Scenario Robot Helps Users to Assemble a Table
Functional Requirements - The robot should: Respond to human spoken
commands with simple behaviors
Open left hand, turn right,.. Grasp(X): X in <visible>
Learn complex behaviors constructed from the primitives
Give me the orange leg Hold the table
Kawada Industries HRP-2 PlatformCNRS-AIST Joint Robotics Laboratory
LAAS, Toulouse, France
Spoken Language Programming
Part of Joint Robotics Laboratory project, AIST/CNRSDominey, Mallet, Yoshida (2007) IEEE ICRA, IEEE Humanoids, (2009) IJHR
Spoken Language Programming
Method Hand coded « primitives » (postures) and grasp(x) procedure Sequenced together via spoken
language Macro Programming
Humanoids 2007 Procedure with Arguments
ICRA 2007 Generalizes to different tasks
Assembly, disassembly
Part of Joint Robotics Laboratory project, AIST/CNRSDominey, Mallet, Yoshida (2007) IEEE ICRA, IEEE Humanoids, (2009) IJHR
User guides action by spoken language
Pseudo-code: At each command: If current subsequence is in
InteractionHistory L1 – anticipate speech L2 – propose next action L3 – take initiative Increment L
Else get next command Execute Update Interaction History
Automatic Learning, and Anticipation
Dominey, Metta, Nori, Natale (2008) IEEE Int. Conf. On Humanoid Robotics
Progressive effects of Learning
First experieceWith leg 1
Speech anticipationWith leg 2
Action propositionWith leg 3
Robot initiativeWith leg 4
Mea
n E
xecu
tion
time
for
a si
ngle
act
ion
(sec
)
Tomasello M, Carpenter M, Call J, Behne T, Moll HY (2005) Understanding and sharing intentions: The origins of cultural cognition, Beh. Brain Sc;. 28; 675-735.Dominey PF (2005) Toward a construction-based account of shared intentions in social cognition. Behavioral and Brain Sciences 28:696-+.
But,…. Cooperation Requires Shared Plans
Lallee, Warneken, Dominey (2009) EpiRob, Humanoids Workshop
Perceive action Attribute agency Form shared plan
Ordered list of (agent, action) pairs Use it in cooperation
Role reversal Limitations: robot doesn’t know “why?”
Larry (left) Robert (right)
ToyBox
Learning Shared Plans from Observation
Approaching Meaning: Linking actions to states
Lallee et al. Submitted Frontiers NeuroRobotics
Learn to recognize action via Dynamic perceptual primitive patterns
Visible, Contact, Moving
Enrich this with knowledge of Enabling state (initial) Resulting State (final/goal) « Derived predicates »
Derived Predicates On, Under Has
Reasoning: Forward chaining from current state to
goal Backward chaining from goal to current
state
Submitted to IROS 2010
Meaning: Linking actions to states
Learn the name ofa new action
Learn the relation between “cover” and “on”
Demonstrate transferto a new enactment
Learn to recognize action via Dynamic perceptual primitive
patterns Visble, Contact, Moving
Enrich this with knowledge of Enabling state (initial) Resulting State (final/goal) « Derived predicates »
Derived Predicates On, Under Has
Reasoning: Forward chaining from current
state to goal Backward chaining from goal to
current state
« Cover Arg1 with Arg2 »
Meaning: Linking actions to states
Language and meaning
Language can augment meaning derived from vision Explaining derived statesExplaining causal relations
MeaningInitial State –Action – Final State-
Vision
Language Action
Towards a System Architecture of Consciousness
Brain interacts unconsiously with
world Constructs a simulation
Simulation is consiously perceived Public cognitve space Private cognitive space Private bodily space
Real Unperceived External World
The unconsciousBrain
Experienced external world, including body
from outsidePublic cognitive
space
Inner cognitiveexperiences
(thoughts,Images…)
Private cognitive space
Inner bodily sensations, feelings
Private body space
Cybernetic interactionsConstrains
Simulates
Conscious experience
The different spaces of conscious experience, from Gray 2004
Hybrid Embodied-Propositional System
Propositional system manipulates compact, « symbolic » representations of actions, plans Embodied system employs « situated simulations », unpacking the compact representations Language allows the speaker to « direct the film » that unfolds in the listener’s mind
Madden, Hoen, Dominey (2009) A Cognitive Neuroscience Perspective on Embodied Language forHuman-Robot Cooperation, Brain and Language
Towards Embodiment: Learning to predict the perceptual consequences of a motor action
Grasping requires vision of the handThe hand has “infinite” posturesHow to reduce the visual recognition
space?
VisionProprioception(joint angles)
…Hand Posture
3Hand Posture 2Hand Posture
1
Perceptual-Motor Learning
Area 5 MMCM
Multi Modal Convergence Maps (MMCM) Topographical Organisation (Kohonen SOM like)
Associates Distinct Patterns of Joint Angles with the Corresponding Image of the hand
Training: Vision-Proprioception pairs every
100ms for 8 min 6 joints moved in cyclic pattern ~16 cycles
Me
an
Re
cog
ntin
tim
e (
+ S
D,
SE
)0,005
0,010
0,015
0,020
0,025
0,030
0,035
0,040
0,045
0,050
LEFT OFF RIGHT OFF LEFT ON RIGHT ON
Vis-Motor LearningOFF ON
Experimental Effects on Performance
Using Vis-Motor Learning has a significant effect on visual recognition time F(1,39) = 418, p < 0.0001
iCub (Robotcub project)
S. LALLEE1, G. METTA2, L. NATALE2, U. PATTACINI2, *P. F.DOMINEY1; (2009) Proprioception of the hand contributes to visual recognition speed andaccuracy: Evidence from the Multi-Modal Convergence Map model of ParietalCortex Area 5, Society for Neuroscience Abstract
Return to the neurophysiology: of language, action and cooperation
Ventral stream (green) phonological and lexical processing (STS, MTG, PFCv)
Dorsal stream (Blue) grammatical integration/unification and sensorimotor interface, simulation (TPJ, PFCdl, PPC)
Complex Event Recognition (Orange) social cognition, cooperation (STS), Agency, Simulation, Intention,Teleological reasoning, Perspectivie taking
Discussion
A Path to Robot Consciousness via Social Cognition: Agency and Intention Manipulates representations of self, other Perspective taking Recognition of agency Designation of intention based on action recogntion Use of language to express beliefs
Can robot studies be used to address any aspects of consciousness? What is the roadmap?
Action and language Shared plans “Meaning” Simulation as meaning Self – body scheme
How would we define robot consciousness?
Acknowledgements
Collaborators Jocelyne Ventre-Dominey Michel Hoen Carol Madden Felix Warneken Toshio Inui Frank Ramus Anthony Mallet Eiichi Yoshida Giorgio Metta Giulio Sandini Francesco Nori Lorenzo Natale Ugo Pattacini
Research Organizations CNRS INSERM
Funding CHRIS (EU FP7) Organic (EU FP7) French ANR
Amorces (PsiRob) Comprendre (Blanc)
RobotCub (EU FP6) iCub Open Call
Students/ PostDocs Jean-David Boucher Stephane Lallee Mehdi Khamassi Xavier Hinaut Anne-Lise Jouen
The Cube Game
Fagel, Bailly (GIPSA-Lab, Grenoble)Boucher,Ventre-Dominey, Dominey(INSERM-RCL, Lyon)
The different spaces of conscious experience
Real Unperceived External World
The unconsciousBrain
Experienced external world, including body
from outside
Public cognitive space
Inner cognitiveexperiences
(thoughts,Images…)
Private cognitive space
Inner bodily sensations, feelings
Private body space
Cybernetic interactionsConstrains
Simulates
Conscious experience
(From Gray, 2004, Fig 1.1)
Integration: Hybrid Telelogical/Embodied Cognitive System
Madden, Hoen, Dominey (2009) Brain and LanguageiCub project – Lyon, June 2009
Teleological reasoningEmbodied representation/simulation
Learning Actions and their Consequences
Learn to recognize action via Dynamic perceptual primitive
patterns Visble, Contact, Moving
Enrich this with knowledge of Enabling state (initial) Resulting State (final/goal) « Derived predicates »
Derived Predicates On, Under Has
Reasoning: Forward chaining from current
state to goal Backward chaining from goal to
current state
« Cover Arg1 with Arg2 »
Meaning: Linking actions to states
RobotCub – A Great Achievement
Distinctions and Definitions
Public vs Private Public – « the red
book on the shelf » Private – thoughts,
feelings
Inner vs External
Replace Explicit Programming
Use On-line, automatic learning of behavior via continuous comparison with the Interaction History
Learning from Experience:Automatic Learning, and Anticipation
Dominey, Metta, Nori, Natale (2008) IEEE Int. Conf. On Humanoid Robotics
Real Unperceived External World
The unconsciousBrain
Experienced external world, including body
from outside
Public cognitive space
Inner cognitiveexperiences
(thoughts,Images…)
Private cognitive space
Inner cognitiveexperiences
(thoughts,Images…)
Private body space
Cybernetic interactionsConstrains
Simulates
Conscious experience
Perceptual-Motor Learning
Multi Modal Convergence Maps (MMCM) Topographical Organisation (Kohonen SOM like)
Associates Distinct Patterns of Joint Angles with the Corresponding Image of the hand
VisionProprioception(joint angles)
…Hand Posture
3Hand Posture 2Hand Posture
1
Area 5 MMCM
We want to cooperate with this guy… But How?
Plan Define a Context for Cooperation Build some basic tools: Spoken Language Programming Learning
Automatically from one’s own experience Shared plans from Observation The meaning of actions
Towards Embodyment A Hybrid Propositional & Embodied Cognitive System