IJCNN 2013 Panel in Higher Emotion 1

Emotion in Brain-Inspired Developmental Networks

Juyang (John) WengComputer Sci., Neurosci., Cognitive Sci.

Michigan State UniversityEast Lansing, MI 49924 USA

weng@cse.msu.edu

IJCNN 2013 Panel in Higher Emotion 2

Patterns of Communication(a) Point to Point(b) Release hormones into blood

(c) Autonomic nervous system;(d) Diffuse modulatory systems

Beart, Connors & Paradiso 2007

IJCNN 2013 Panel in Higher Emotion 3

Major Neurotransmitters

Beart, Connors & Paradiso 2007

IJCNN 2013 Panel in Higher Emotion 4

BMI 831Cognitive Science

for Brain-Mind ResearchLecturer:

Juyang Weng

IJCNN 2013 Panel in Higher Emotion 5

BMI 871Introduction to Computational

Brain-Mind Lecturer:

Juyang Weng

IJCNN 2013 Panel in Higher Emotion 6

Motivation/emotion in the Brain Motivational system in the brain:

biological causality for the development of emotion Lower (physiology) emotion:

Arousal: noradrenaline, oxytocin, cortisol Habituation sensitization, familiarization

(acetylcholine and norepinephrine systems) Pain avoidance (serotonin system) Pleasure seeking (dopamine system)

Basic emotion: angry, happy, sad, scared, disgust, etc. Higher emotion: moods, dispositions (e.g., jealousy)

IJCNN 2013 Panel in Higher Emotion 7

Neuromodulatory Systems

Jeff Krichmar, AB, 2008

IJCNN 2013 Panel in Higher Emotion 8

Quiz: Why Modulations

Quiz: The main purpose of modulatory systems is:A. Animal likeB. Establish bounds with

humanC. Look intelligentD. Speed-up learningE. Being quick to fight or

flee

IJCNN 2013 Panel in Higher Emotion 9

Quiz: Why Modulations

Quiz: The main purpose of modulatory systems is:A. Animal likeB. Establish bounds with

humanC. Look intelligentD. Speed-up learningE. Being quick to fight or

flee

IJCNN 2013 Panel in Higher Emotion 10

Why Motivation? Evolution: Pressure of survival Understanding of the environment should not be

the primary goal of a life Fruit flies, Rats, Humans

Sexual selection: survival is not sufficient Males: Fight with males, court with females Females: Select the male winners

Motivation: quick ways to sense a variety of values Neuromodulators, diffused transmission

IJCNN 2013 Panel in Higher Emotion 11

In the blind, visual cortex is reassigned to audition and touch.

Therefore, we chose not to statically model brain areas! Brain areas should emerge

IJCNN 2013 Panel in Higher Emotion 12

Motivation: Architectural Concepts

IJCNN 2013 Panel in Higher Emotion 13

Symbolic Value Systems Sutton & Barto 1981:

reward as positive values Delayed rewards

Ogmen 1997: Punishments, rewards, novelty Sporns et al. 1999: Darwin robot Kakade & Dayan 2002: novelty and shaping Oudeyer et al. 2007: error max; progress max; similarity-

based progress maximization Huang & Weng 2007: punishment, reward, novelty Cox & Krichmar 2009: Neuromodulation as a robot controller Singh et al. 2010: reward and evolution

IJCNN 2013 Panel in Higher Emotion 14

Symbolic: Q-Learning

Symbolic reinforcement learning No need for state transition probability

Value driven Does not allow state inconsistency

Future value estimation for delayed rewards Time discount model: state-action value:

IJCNN 2013 Panel in Higher Emotion 15

Q-Learning Limitations

Symbolic: Exponential number of states, static Brittle

Fixed, greedy value function: Immediate rewards are preferred

Does not allow perception Does not allow creativity for other non-

modeled concepts

IJCNN 2013 Panel in Higher Emotion 16

Theory: For Any AFA There Is a GDN

Weng IJCNN 2010

GDN: GenerativeDevelopmental Network

AFA: Agent Finite Automaton

Marvin Minsky criticized ANNs

IJCNN 2013 Panel in Higher Emotion 17

Emergent: 5-HT & DA Systems Blue: 5-HT system Red: DA system Mechanism 1 (extra cell):

Mechanism 2 (within cell):zip increases the threshold Tzis reduces the threshold TIf pre-action > T, fire

IJCNN 2013 Panel in Higher Emotion 18

5-HT, DA, Ach, NE

Serotonin (5-HT): pain, stress, threats and punishment

Dopamine: (DA)pleasure, wanting, anticipation, and reward

Acetylcholine (Ach): expected uncertainty?

Norepinephrine (NE): novelty (unexpected uncertainty)?

IJCNN 2013 Panel in Higher Emotion 19

Emergent Value Systems

Daly, Brown, Weng 2011, Paslaski, VanDam, & Weng 2011, Weng et al. Neural Networks 2013: Neuromorphic Motivated Systems based on DN: 5-HT and DA Wandering or foraging, face recognition

Wang, Wu, & Weng 2011, 2012 Neuromorphic Motivated Systems based on DN: Ach and NE Novelty and Uncertainty Synapse maintenance Segmentation of objects from backgrounds

Zheng, Qian, Weng, Zheng 2013 Effects on internal brain areas: change learning rates

IJCNN 2013 Panel in Higher Emotion 20

Effects of 5-HT and DA on Y?

Effect on Z area: Inhibit and excite actionsWeng et al. IJCNN 2011

Effect on Y area:on learning rateZheng, Qian, Weng & Zhang, IJCNN 2013

IJCNN 2013 Panel in Higher Emotion 21

Dopamine System

IJCNN 2013 Panel in Higher Emotion 22

Serotonin System

IJCNN 2013 Panel in Higher Emotion 23

Acetylcholine System

IJCNN 2013 Panel in Higher Emotion 24

Norepinephrine System

IJCNN 2013 Panel in Higher Emotion 25

Bottom-Up Weights and 5-HT

IJCNN 2013 Panel in Higher Emotion 26

Y Weights for Z Neurons

Paslaski et al. IJCNN 2011

IJCNN 2013 Panel in Higher Emotion 27

Error Rates for Recognition

Paslaski et al. IJCNN 2011

IJCNN 2013 Panel in Higher Emotion 28

Navigation, Wandering, Foraging

Three synthetic agents: Self (S) Attractor (A) Repulsor (R)

5-HT: distance-lonely and distance-fear

DA: d < distance-desire

IJCNN 2013 Panel in Higher Emotion 29

Setting

IJCNN 2013 Panel in Higher Emotion 30

Environmental Settings

No brainer (control): no pain, no pleasure Love or War (D-lonely, D-fear) = (50, 50) Little Danger (D-lonely, D-fear) = (25, 125) Much Danger (D-lonely, D-fear) = (125, 25) Looking over the Fence

(D-lonely, D-fear) = (125, 125)

IJCNN 2013 Panel in Higher Emotion 31

Average Distance

Daly et al. IJCNN 2011

IJCNN 2013 Panel in Higher Emotion 32

Ach and NE: Goal for Segmentation

Wang, Wu and Weng, IJCNN 2011

(a) Bottom-up input to a neuron. (b) True object contour. (c) Estimated synaptogenic factor

IJCNN 2013 Panel in Higher Emotion 33

Global Confidence: Face Recognition

IJCNN 2013 Panel in Higher Emotion 34

Ach & NE: Receiver Operating Curve

IJCNN 2013 Panel in Higher Emotion 35

ACh Estimates

IJCNN 2013 Panel in Higher Emotion 36

DN: Motivational System as Emotion

Emergent nervous system Modeled:

5-HT (serotonin) DA (dopamine) Ach (acetylcholine) NE (norepinephrine)

Basic and higher emotion: developed from experience