IJCNN 2013 Panel in Higher Emotion 1 Emotion in Brain-Inspired Developmental Networks Juyang (John) Weng Computer Sci., Neurosci., Cognitive Sci. Michigan

IJCNN 2013 Panel in Higher Emotion 1

Emotion in Brain-Inspired Developmental Networks

Juyang (John) WengComputer Sci., Neurosci., Cognitive Sci.

Michigan State UniversityEast Lansing, MI 49924 USA

[email protected]


Patterns of Communication(a) Point to Point(b) Release hormones into blood

(c) Autonomic nervous system;(d) Diffuse modulatory systems

Beart, Connors & Paradiso 2007


Major Neurotransmitters

Beart, Connors & Paradiso 2007


BMI 831Cognitive Science

for Brain-Mind ResearchLecturer:

Juyang Weng


BMI 871Introduction to Computational

Brain-Mind Lecturer:

Juyang Weng


Motivation/emotion in the Brain Motivational system in the brain:

biological causality for the development of emotion Lower (physiology) emotion:

Arousal: noradrenaline, oxytocin, cortisol Habituation sensitization, familiarization

(acetylcholine and norepinephrine systems) Pain avoidance (serotonin system) Pleasure seeking (dopamine system)

Basic emotion: angry, happy, sad, scared, disgust, etc. Higher emotion: moods, dispositions (e.g., jealousy)


Neuromodulatory Systems

Jeff Krichmar, AB, 2008


Quiz: Why Modulations

Quiz: The main purpose of modulatory systems is:A. Animal likeB. Establish bounds with

humanC. Look intelligentD. Speed-up learningE. Being quick to fight or

flee


Quiz: Why Modulations

Quiz: The main purpose of modulatory systems is:A. Animal likeB. Establish bounds with

humanC. Look intelligentD. Speed-up learningE. Being quick to fight or

flee


Why Motivation? Evolution: Pressure of survival Understanding of the environment should not be

the primary goal of a life Fruit flies, Rats, Humans

Sexual selection: survival is not sufficient Males: Fight with males, court with females Females: Select the male winners

Motivation: quick ways to sense a variety of values Neuromodulators, diffused transmission


In the blind, visual cortex is reassigned to audition and touch.

Therefore, we chose not to statically model brain areas! Brain areas should emerge


Motivation: Architectural Concepts


Symbolic Value Systems Sutton & Barto 1981:

reward as positive values Delayed rewards

Ogmen 1997: Punishments, rewards, novelty Sporns et al. 1999: Darwin robot Kakade & Dayan 2002: novelty and shaping Oudeyer et al. 2007: error max; progress max; similarity-

based progress maximization Huang & Weng 2007: punishment, reward, novelty Cox & Krichmar 2009: Neuromodulation as a robot controller Singh et al. 2010: reward and evolution


Symbolic: Q-Learning

Symbolic reinforcement learning No need for state transition probability

Value driven Does not allow state inconsistency

Future value estimation for delayed rewards Time discount model: state-action value:


Q-Learning Limitations

Symbolic: Exponential number of states, static Brittle

Fixed, greedy value function: Immediate rewards are preferred

Does not allow perception Does not allow creativity for other non-

modeled concepts


Theory: For Any AFA There Is a GDN

Weng IJCNN 2010

GDN: GenerativeDevelopmental Network

AFA: Agent Finite Automaton

Marvin Minsky criticized ANNs


Emergent: 5-HT & DA Systems Blue: 5-HT system Red: DA system Mechanism 1 (extra cell):

Mechanism 2 (within cell):zip increases the threshold Tzis reduces the threshold TIf pre-action > T, fire


5-HT, DA, Ach, NE

Serotonin (5-HT): pain, stress, threats and punishment

Dopamine: (DA)pleasure, wanting, anticipation, and reward

Acetylcholine (Ach): expected uncertainty?

Norepinephrine (NE): novelty (unexpected uncertainty)?


Emergent Value Systems

Daly, Brown, Weng 2011, Paslaski, VanDam, & Weng 2011, Weng et al. Neural Networks 2013: Neuromorphic Motivated Systems based on DN: 5-HT and DA Wandering or foraging, face recognition

Wang, Wu, & Weng 2011, 2012 Neuromorphic Motivated Systems based on DN: Ach and NE Novelty and Uncertainty Synapse maintenance Segmentation of objects from backgrounds

Zheng, Qian, Weng, Zheng 2013 Effects on internal brain areas: change learning rates


Effects of 5-HT and DA on Y?

Effect on Z area: Inhibit and excite actionsWeng et al. IJCNN 2011

Effect on Y area:on learning rateZheng, Qian, Weng & Zhang, IJCNN 2013


Dopamine System


Serotonin System


Acetylcholine System


Norepinephrine System


Bottom-Up Weights and 5-HT


Y Weights for Z Neurons

Paslaski et al. IJCNN 2011


Error Rates for Recognition

Paslaski et al. IJCNN 2011


Navigation, Wandering, Foraging

Three synthetic agents: Self (S) Attractor (A) Repulsor (R)

5-HT: distance-lonely and distance-fear

DA: d < distance-desire


Setting


Environmental Settings

No brainer (control): no pain, no pleasure Love or War (D-lonely, D-fear) = (50, 50) Little Danger (D-lonely, D-fear) = (25, 125) Much Danger (D-lonely, D-fear) = (125, 25) Looking over the Fence

(D-lonely, D-fear) = (125, 125)


Average Distance

Daly et al. IJCNN 2011


Ach and NE: Goal for Segmentation

Wang, Wu and Weng, IJCNN 2011

(a) Bottom-up input to a neuron. (b) True object contour. (c) Estimated synaptogenic factor


Global Confidence: Face Recognition


Ach & NE: Receiver Operating Curve


ACh Estimates


DN: Motivational System as Emotion

Emergent nervous system Modeled:

5-HT (serotonin) DA (dopamine) Ach (acetylcholine) NE (norepinephrine)

Basic and higher emotion: developed from experience

Documents

IJCNN 2013 Panel in Higher Emotion 1 Emotion in Brain-Inspired Developmental Networks Juyang (John) Weng Computer Sci., Neurosci., Cognitive Sci. Michigan