Embed Size (px)
Citation preview
Cognitive Interaction Technology Center of Excellence
gentsSociable
Imitation Mechanisms of Social Resonance for Embodied Agents
Amir SadeghipourSociable Agents Group
1
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
2
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
3
Cognitive Interaction Technology Center of Excellence
gentsSociable
Motivation
4
Communication& Coordination Verbal
&Non-verbal:
Gestures
Perception - Action Link
(Motor Resonance)
Cognitive Interaction Technology Center of Excellence
gentsSociable
• Focus: Human nonverbal social interaction• Capabilities:
• Perception• Representation• Generation
• Cognitive facilities:• Fast & real-time• Concurrent processing• Different levels of abstraction: from form to meaning
• Generation, perception & recognition
• Perception-action integration• Imitation learning
• Behavior coordination5
MotivationRequirements
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
6
Cognitive Interaction Technology Center of Excellence
gentsSociable
Cognitive Model
7
Motor Knowledge
Observation
Inverse Models
Forward Models
Recognition Learning
Cognitive Interaction Technology Center of Excellence
gentsSociable
8
Observation
Inverse Models
Forward Models
Recognition Learning
Cognitive ModelMotor Knowledge
Motor Knowledge
Cognitive Interaction Technology Center of Excellence
gentsSociable
9
Motor Knowledge
Observation
Inverse Models
Forward Models
Recognition Learning
Motor Commands
Motor Programs
Motor Schemas
Cognitive ModelMotor Knowledge
Motor Programs
Motor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
Motor Knowledge
Motor Commands(Right hand)
Motor Programs(Right hand)
Motor Schemas
MC1 MC2 MC3 MC4
MP1 MP2
Waving
MC1M
C4
MC2
MC3
MP1 MP2
Waving
Cognitive ModelMotor Knowledge
Cognitive Interaction Technology Center of Excellence
gentsSociable
11
Motor Knowledge
Observation
Inverse Models
Forward Models
Recognition Learning
Motor Commands
Motor Programs
Motor Schemas
Cognitive ModelGeneration
Motor Programs
Motor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
12
Motor Knowledge
Observation
Inverse Models
Forward Models
Recognition Learning
Motor Commands
Motor Programs
Motor Schemas
Cognitive ModelPerception
Motor Programs
Motor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
13
Observation
Inverse Models
Forward Models
Recognition Learning
Cognitive ModelRecognition process: Forward Models
Motor Knowledge
Motor Commands
Motor Programs
Motor Schemas
Motor Programs
Motor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
14
Observation
Inverse Models
Forward Models
Recognition Learning
Cognitive ModelForward Models
MS-FM
MP-FM
MC-FM
Motor Knowledge
Motor Commands
Motor Programs
Motor Schemas
Motor Programs
Motor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
• Perception challenges:• Bottom-up perception process• Top-down perception process
• Bayes′ theorem: P(A|B) = α P(B|A) P(A)
15
Cognitive ModelForward Models
Forward Models
MS-FM
MP-FM
MC-FM
Observation
likelihood a priori
• Perception challenges:• Bottom-up perception process• Top-down perception process
Cognitive Interaction Technology Center of Excellence
gentsSociable
16
Cognitive ModelForward Models
Forward Models
MS-FM
MP-FM
MC-FM
Observation
= Pt-1( hmc | o )
Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)
Cognitive Interaction Technology Center of Excellence
gentsSociable
17
Cognitive ModelForward Models
Forward Models
MS-FM
MP-FM
MC-FM
Observation
• Perception challenges:• Bottom-up perception process• Top-down perception process
Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)
Pt( hmp | o, Hmc ) = α Pt(o, Hmc | hmc) Pt(hmc)
Pt( hms | o, Hmc, Hmp ) = α Pt(o, Hmc, Hmp | hmc) Pt(hms)
Cognitive Interaction Technology Center of Excellence
gentsSociable
• Perception challenges:• Bottom-up perception process• Top-down perception process
Pt( hmp | Hms ) = α Pt(Hms | hmp) Pt(hmp)
Pt( hmc | Hmp ) = α Pt(Hmp | hmc) Pt(hmc)
18
Cognitive ModelForward Models
Forward Models
MS-FM
MP-FM
MC-FM
Observation
Cognitive Interaction Technology Center of Excellence
gentsSociable
• Top-DownPt( hmp | Hms ) = α Pt(Hms | hmp) Pt(hmp)Pt( hmc | Hmp ) = α Pt(Hmp | hmc) Pt(hmc)Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)
• Bottom-UpPt( hmp | o, Hmc ) = α Pt(o, Hmc | hmc) Pt(hmc)Pt( hms | o, Hmc, Hmp ) = α Pt(o, Hmc, Hmp | hmc) Pt(hms)
• Prior probability at t = Posterior probability at t-1
19
Cognitive ModelBehavior Coordination
Forward Models
MS-FM
MP-FM
MC-FM
Observation
Cognitive Interaction Technology Center of Excellence
gentsSociable
• t = 0 ⇒ Initialization from „motor activation“
• Consequently, the agent(a) tends to perform MC, MP & MS that have been perceived last (P➟A)(b) and its sensitivity is primed by self performed MC, MP & MS (A➟P)
20
Cognitive ModelBehavior Coordination
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
21
• 3D time-of-flight camera: SwissRanger SR4000• Marker-free tracking software: IISU 2.0• Virtual Agent: Vince
Cognitive Interaction Technology Center of Excellence
gentsSociable
ResultsFramework
22
Cognitive Interaction Technology Center of Excellence
gentsSociable
Results
23
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
24
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outlook
• Main challenges:• Beyond motor schema towards meaning and social intention• Considering finger configuration• Creating motor schema automatically
25
Cognitive Interaction Technology Center of Excellence
gentsSociable
OutlookBeyond Motor Schema
• Main challenges:• Beyond motor schema towards meaning and social intention• Considering finger configuration• Creating motor schema automatically
26
Motor Knowledge
Observation
IM
FM
Motor Commands
Motor Programs
Motor Schemas
Motor Programs
Motor Commands
Social Intention (Semantics)
Language(labels)
Cognitive Interaction Technology Center of Excellence
gentsSociable
OutlookFinger Configuration
• Main challenges:• Beyond motor schema towards meaning and social intention• Considering finger configuration• Creating motor schema automatically
27
Motor Knowledge
Motor Commands
Motor Programs
Motor Schemas
Motor Programs
Motor Commands
Motor ProgramsMotor Programs
Motor CommandsMotor Commands
Cognitive Interaction Technology Center of Excellence
gentsSociable
OutlookAutomatic Motor Schema Generation
• Main challenges:• Beyond motor schema towards meaning and social intention• Considering finger configuration• Creating motor schema automatically
28
considering variant and invariant features:• Different performances• Left and right hand• Velocity• Position• Repetition• Size• Rotation
Cognitive Interaction Technology Center of Excellence
gentsSociable
Outline
• Motivation
• Cognitive Model
• Results
• Outlook
• Conclusion
29
Cognitive Interaction Technology Center of Excellence
gentsSociable
• Focus: Human nonverbal social interaction• Capabilities:
• Perception• Representation• Generation
• Cognitive facilities:• Fast & real-time• Concurrent processing• Different levels of abstraction: from form to meaning
• Generation, perception & recognition
• Perception-action integration• Imitation learning
• Behavior coordination30
Conclusion
Motor Knowledge
Observation
IM
FM
Motor Knowledge
Cognitive Interaction Technology Center of Excellence
gentsSociable
Conclusion
31
Thanks!
Web: www.techfak.uni-bielefeld.de/ags/soa/research/gesture-imitation/
Cognitive Interaction Technology Center of Excellence
gentsSociable
DepotForward Models: Bottom-Up Process
32
• MC:
• MP:
• MS:
