Upload
osborn-baldwin
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
Bob MarinierAdvisor: John Laird
Functional Contributions of Emotion to Artificial
Intelligence
Introduction
2
Folk psychology considers emotions to be a distraction from logical thought
People tend to think that emotion is unknowable, indefinable
Psychological work in the last several decades has demonstrated that emotion plays a critical role in effective functioning and learning
Introduction
3
Research GoalsBring the functionality of emotion to AICreate a precise computational definition of emotion
ApproachIntegrate emotion with a complete agent frameworkComputationally distinguish emotion, mood and
feelingWeight feeling’s importance by computing its
intensityUse feeling as intrinsic reward signal to drive
reinforcement learning
Appraisal Theories of Emotion
4
A situation is evaluated along a number of appraisal dimensions,many of which relate the situation to current goalsNovelty, goal relevance, goal conduciveness, expectedness, causal
agency, etc.Result of appraisals determines emotionThe emotion is combined with mood, which is an “average” over
recent emotions, to form a feeling, which is actually perceived with some intensity
The feeling can then be coped with (via internal or external actions)
SituationGoals
Appraisal
Emotion, Mood, Feeling
Coping
Appraisals to Emotions(Scherer 2001)
5
Joy Fear Anger
Suddenness High/medium High High
Unpredictability High High High
Intrinsic pleasantness Low
Goal/need relevance High High High
Cause: agent Other/nature Other
Cause: motive Chance/intentional Intentional
Outcome probability Very high High Very high
Discrepancy from expectation
High High
Conduciveness Very high Low Low
Control High
Power Very low HighWhy these dimensions?What is the functional purpose?
Functions of Emotion
6
Situation summary: Appraisals and emotion provide abstract interpretation
Decouples stimulus/response: Can react to interpretation instead of stimulus
Attention: Some appraisals help prioritize processingHistorical context: Mood provides a context for current
interpretationsLearning: Feeling may provide an intrinsic reward signalMemoryDecision makingAction preparationCommunication
Outline
7
Integrate emotion with a complete agent framework
Computationally distinguish emotion, mood and feeling
Weight feeling’s importance by computing its intensity
Use feeling as intrinsic reward signal to drive reinforcement learning
Discussion & Conclusion
SituationGoals
Appraisal
Emotion, Mood, Feeling
Coping
Newell’s Abstract Functional Operations(Newell 1990)
8
Allen Newell defined a set of computational Abstract Functional Operations that are necessary and sufficient for immediate behavior in humans and complete agents
Perceive Obtain raw perception
Encode Create domain-independent representation
Attend Choose stimulus to process
Comprehend
Generate structures that relate stimulus to tasks and can be used to inform behavior
Task Perform task maintenance
Intend Choose an action, create prediction
Decode Decompose action into motor commands
Motor Execute motor commands
Newell’s Abstract Functional Operations(Newell 1990)
9
…but how these actually work was not clear.
Perceive What information is generated?
Encode What information is generated?
Attend What information is required?
Comprehend
What information is required and generated?
Task What information is required?
Intend What information is required?
NAFO and Appraisal(Marinier & Laird 2006)
10
Generated By Required By
Suddenness Perceive
AttendUnpredictability
EncodeIntrinsic pleasantness
Goal relevance
Causal agent
ComprehendComprehend,Task,Intend
Causal motive
Outcome probability
Discrepancy from expectationGoal/need conducivenessControl
Power
Integrate emotion with a complete agent framework
Computationally distinguish emotion, mood and feeling
Weight feeling’s importance by computing its intensity
Use feeling as intrinsic reward signal to drive reinforcement learning
Discussion & Conclusion
Outline
11
SituationGoals
Appraisal
Emotion, Mood, Feeling
Coping
Body
Symbolic Long-Term Memories
Procedural
Short-Term Memory
Situation, Goals
Decision Procedure
Chunking
Reinforcement
Learning
Semantic
SemanticLearning
Episodic
EpisodicLearning
Perception ActionVisual
Imagery
Extending Soar with Emotion(Marinier & Laird 2007)
12
Soar is a cognitive architecture
A cognitive architecture is a set of task-independent mechanisms that interact to give rise to behavior
Cognitive architectures are general agent frameworks
Feelin
g
Genera
tion
Reinforcement
Learning
Emotion.5,.7,0,-.4,.3,
…
Extending Soar with Emotion(Marinier & Laird 2007)
13
Body
Decision Procedure
Perception Action
Appraisals
Feelings Short-Term Memory
Situation, Goals
Mood.7,-.2,.8,.3,.6,
…
Feelings
Knowledge
Architecture
Symbolic Long-Term Memories
Procedural
Chunking
Semantic
SemanticLearning
Episodic
EpisodicLearning
+/- In
tensity
Feeling.9,.6,.5,-.1,.8,
…
VisualImagery
Computing Feeling from Emotion and Mood(Marinier & Laird 2007)
14
Assumption: Appraisal dimensions are independentLimited Range: Inputs and outputs are in [0,1] or [-1,1]Distinguishability: Very different inputs should lead to
very different outputsNon-linear: Linearity would violate limited range and
distinguishability
Example
16
Maze Task
Start
Goal
17
Feeling Dynamics Results
18
very easy
Computing Feeling Intensity(Marinier & Laird 2007)
19
Motivation: Intensity gives a summary of how important (i.e., how good or bad) the situation is
Limited range: Should map onto [0,1]No dominant appraisal: No single value should drown out
all the othersCan’t just multiply values, because if any are 0, then
intensity is 0Realization principle: Expected events should be less
intense than unexpected events
Example
21
Outline
22
Integrate emotion with a complete agent framework
Computationally distinguish emotion, mood and feeling
Weight feeling’s importance by computing its intensity
Use feeling as intrinsic reward signal to drive reinforcement learning
Discussion & Conclusion
Intrinsically MotivatedReinforcement Learning(Sutton & Barto 1998; Singh et al. 2004)
23
Environment
Critic
Agent
Actions
StatesRewar
ds
External Environment
Internal Environment
Agent
Critic
Actions
StatesRewar
dsDecisions
Sensations
Appraisal
Process+/-
Feeling Intensity
“Organism”
Learning Task
Start
Goal
24
Learning Results
25
Discussion & Conclusion
26
DiscussionAgent learns fast
Gets frequent reward signalsMood accelerates learning
Provides reward during those steps in which the agent has no emotion
ConclusionDeveloped an initial computational model of
emotionIntegrated model with complete agent frameworkDemonstrated some functional advantages of
integration