André Pereira

Outline

• Brain– What it does?– Brain structures and functions– Neural networks– On intelligence

• Emotions• Conclusions

What it does?

• Enables us to: See, hear, smell, taste, feel Speak, gesticulate, act ... Remember, think, predict, learn, dream ...

• The brain self analyzes the brain Right now we are thinking about our brain with our

brain The brain ultimate challenge is to fully understand

itself

Brain structures and functions

Cerebellum

• Means “little brain”• Coordinates voluntary movement and

balance• Enables nonverbal learning and memory• Helps us

– judge time– modulate our emotions– distinguish sounds and textures

Limbic system

• Border system• Important links to emotions• Basic motives (e.g. Sex and food)• Hippocampus

– Processes memory• Amygdala• Hypothalamus

Amygdala

• Influences fear and agression• Perception of rage and fear• Processes emotional memories (LeDoux,

2000)• Related also to:

– Positive emotions– Attention

Hypothalamus

• Performs body maintenance duties• Hunger, thirst and body temperature• Monitors blood chemistry• Takes orders from other parts of the brain• Contains reward centers

Cerebral Cortex

• 85 % of the brain’s weight• Neural networks enable perceiving,

thinking and speaking• Body’s control center and information

processing center• Mostly filled with axon connections• Contains 20-23 billion nerve cells• 300 trillion synaptic connections

Cerebral Cortex

• Two hemispheres• Four lobes in each hemisphere

– Frontal, parietal, occipital and temporal– Each have several functions– Many functions use more than one

Cortex functions

Cortex emotional functions

• Orbitofrontal cortex– Reward / Punishment

• Pre-frontal cortex– Approach / avoid related affect

• No single location, region or circuit dedicated to emotion generation– “Older” subcortical and cortex are

both involved

Association Areas

• Three fourths of the cortex is uncommited to sensory or muscular activity– Neurons in these parts integrate information– Link sensory inputs with sensed memories– Found in all four lobes

• Brains are made of neurons; therefore, the brain is a neural network

Computational Neural Networks

• What is it?– A method of computing, based on the interaction

of multiple connected processing elements– A Neural Network generally simply maps a set of

inputs to a set of outputs• What can it do?

– Compute a known function– Approximate an unknown function– Pattern Recognition– Signal Processing– Learn to do any of the above

Neural Network

Input 0 Input 1 Input n...

Output 0 Output 1 Output m...

Jeff Hawkins – On Intelligence

Hierarchical Temporal Memory

• Machine learning model• Models structural and algorithmic properties

of the neocortex• Based on memory prediction theory

presented on his book (On intelligence)• Similar to neural networks but tries to

emulate the human cortex functions– Can also be considered a bayesian network

where the network consists of a collection of nodes arranged in a tree-shaped hierarchy

Outline

• Brain• Emotions

– Functions of emotions– Definitions– Scherers appraisal model– Databases– Emotion Recognition

• Conclusions

Functions of emotions

• Physical– Preparing action and reaction

• Cognitive:– focusing attention– prioritising goals– influencing decision-making– shaping memory / learning

Functions of emotions

• Social– Emotional expression for communication

/signaling and social co-ordination– Elicits adaptive social responses from others

• Anger elicits fear responses (Dimberg&Ohman96)• Distress elicits sympathy (Eisenberg et al89)

Functions of emotions

• Creativity and Emotions highly related– Many famous artists suffered from mood

disorders– Positive moods can enhance creative problem

solving

• Memory and Emotions highly related– Positive mood can remember happy things

more easily

Emotions in learning

III

IVIII

Constructive learning

Un-learning

Positive AffectNegative Affect

AweSatisfaction

Curiosity

HopefulnessFresh Research

DisappointmentPuzzlementConfusion

FrustrationDiscardMisconceptions

Emotional health problems

• Understanding the mind and the brain also help in studying emotional health problems such as:– Anxiety neurosis– Depression– Stress– ...

• Some authors have turned to connectionist modeling of the presumed neural bases to study them

Phineas gage and Elliot

• Phineas Gage had an accident with a metal bar in 1848 and physically survived

• Damasio’s patients with similar frontal-lobe disorders– Dangerous choices no bad feelings

– Make a bad investment– Don’t learn: will invest again

• Indecisive– Search the entire problem space

• Computers are similar• Emotions are important to rational thinking

Definitions

• Because of its Interdisciplinary no common definition still exists– Phylosophy– Psychology– Neuroscience– Computational neuroscience– Machine learning– Robotics– ...

Definitions

• Discrete emotion theories– Ekman

• Dimensional emotion theories– Russel

• Componential models of emotion (Appraisal theories)– Scherer

Discrete emotion theories

• Followed Darwin’s work

• Focus on facial expressions

• Innate & universal• Emotion = family of

related states• In limbic parts of

brain

Ekman’s six facial expressions

Dimensional emotion theoriesRussel’s model of affect

Appraisal

• Appraisal– Is an evaluation of the personal significance

of events as central antecedents of emotional experience

• Appraisal theories– specify a set of criteria or dimensions that are

presumed to underlie the emotion-constituent appraisal process

Appraisal theories

WorldEmotions

…

N

C

A

GC

GR

V

N

Appraisal Dimensions

•Mental structures required:•Situations, Expectations, Beliefs, Values, Goals, Plans, Causal structures, Agent history

•Typically implemented using symbolic representations

•Rules, Semantic nets …•Large amounts of domain-specific knowledge required•Complex reasoning required

This is thedifficult part!

•“Could” be easy•Vector maps directly onto n-dim space of appraisal dimensions•Emotion identified via some measure of proximity (Euclidian dist.)

Scherer’s appraisal model

Scherer’s appraisal model

Scherer’s appraisal model

• Investigation of the temporal dynamics of neural networks – constitute evidence for the sequential temporal

unfolding of appraisals• Investigation of dissociable neural networks

– constitute evidence for the existence of different levels of processing in appraisal

• Amygdala plays a central role in the appraisal process

Emotion Databases

• Generally acted data• Should involve people in natural situations

– Conveying valid pictures of emotion in action and interaction

• Should annotate: – emotional content (smile, negative/positive …) – recording context (Intrusiveness, formality…)– agent characteristics(age, gender…) – …

• Great value to those wishing to use neural/learning techniques for emotion recognition and expression

(Cowie et al.,2005)

Recognizing emotions

• A machine learning approach to emotion recognition in real-life situations using speech (Devillers et al., 2005)– Uses real data from two call centres

• A variety of machine learning approaches were used (SVMs, decision trees ...)– No significant diferences were found between

them• What improved the system was the fusion

of inputs (prosodic, lexical ...)

Recognizing emotions

• A neural network architecture was constructed to be able to handle the fusion of different modalities (Taylor and Fragopanagos 2005)– Facial features, prosody and lexical content in

speech• Database created using a wizard of oz

version of the ELIZA concept introduced by Weizenbaum (1966)

Outline

• Brain• Emotions• Conclusions

Conclusion

• The brain is still a “?” for us!• Emotions are an important part of our

brain, it improves our cognition and social skills

• Such improvements need to be studied and their study may also benefit computational paradigms

Conclusions

• Improved HCI in many areas– Call centres– Machine tutoring – Really natural language processing

• With emotion exchange – Games ...

• Machines will be more effective if they are based on the understanding of our emotions and brain

The end

• Questions?