FROM NEURONS TO BRAINS TO NEURAL NETWORK MODELS

1

+ + +- --- --

+ + +

I1 I2 I3

3

2

x1 x2 x3

(c) CELEST 20072

THE UNIFYING THEME OF CELEST CURRICULUM: METACOGNITION

LEARNING ABOUT LEARNING

THINKING ABOUT THINKING

A focus on neuroscience is a novel and compelling approach to learning because it explicitly focuses on human perception and learning

Teaches students various study strategies while instructing students in a variety of critical math and science skills

(c) CELEST 20073

FROM NEURONS…

Anatomy, morphology, physiology, specialization…

Neurons and the Synapse

How Neurons transmit an action potential and how the synapse

works

(c) CELEST 20075

The Beginning

Any thought, experience, or action that you do can be known as a stimulus. Those stimuli generate nerve impulses.

For example, when you see something light is reflected off a surface and enters your eye. Then it stimulates the retina’s photoreceptors which begins stimulating the nerves to create an electric impulse. That goes to the neurons.

http

://reu.u

wo

sh

.ed

u/va

ug

ha

n.p

hp

REFLECTED LIGHT

Photoreceptors on the retina

(c) CELEST 20076

ANATOMY OF A NEURONNerve impulse travels along Nerve cells otherwise known as neurons. These neurons have many parts which are involved in the transmission of the cell.

Here are the parts to a neuron that are present in the transmission of this impulse

Dendrites- act to conduct the electrical stimulation received from other neural cells to the cell body

Cell Body- also known as the “soma” can vary in size depending upon the type of neuron. It also contains the nucleus

Nucleus- Is responsible for producing most of the RNA in the Neurons and most proteins used by neurons are created by mRNA, which can create structures such as ion channels.

(c) CELEST 20077

ANATOMY OF A NEURON 2

Axon- Conducts the electric impulse from the cell body to the axon terminals

Myelin sheath- Is an insulating material which prevents the electric impulse from leaking allowing the impulse to travel rapidly. Some types of neurons don’t have this.

Schwann Cell- also aids in the insulation allowing the electric impulse to travel rapidly down the axon

Nodes of Ranvier- is the place where the electric impulse jumps to in each cell to carry the impulse down the axon acting like an electric amplifier.

Axon Terminal- Is the end of the axon which is part of the chemical synapse.

(c) CELEST 20078

Types of NeuronsThere are many types of Neurons here are four examples

of 4 more common ones.

Bipolar neurons are usually part of sensory path such as smell, sight, taste, hearing and vestibular functions. Unipolar neurons are also sensory neurons.Multipolar neurons are the majority of the brains neuronsPyramidal neuron are found in the hippocampus and cerebral cortex.

(c) CELEST 20079

ANATOMY OF AN ACTION POTENTIAL

The electrical impulse if large enough becomes known as an action potentials which is used to communicate with other neurons

An action potential occurs when an electrical charge travels down the axon from the cell body to the axon terminals through the Nodes of Ranvier

Axon

Axon Terminals

DendritesCell #1

Cell #2

(c) CELEST 200710

Action Potentials The impulse causes sodium channels to

open which allows sodium ions to start flowing into the neuron changing the charge gradient causing cell depolarization ( which means the potential difference is rising).

The deplolarization eventually reaches a threshold for starting

Sodium ion flow

the action potential, which means that the neuron will fire and more sodium channels open. The sodium ions continue to flow in and the depolariztion continues. As it reaches its maximum potential the sodium channels begin closing,

(c) CELEST 200711

Action PotentialsPotassium ion flow And the potassium channels

begin opening, which allows potassium ions to flow into the cell.

This flow begins repolarization and starts returning the potential to the rest potential.

However, channels stay open too long and the cell becomes hyperpolarized. At cell cannot fire until the cells restpotential is restored. This restoration is when the sodium potassium pump along with the outflow of potassium restores the ion concentrations to the beginning and the cell is ready to fire again

(c) CELEST 200712

Sodium/Potassium PumpThe pump acts to restore the

original Sodium ion and Potassium ion concentration because now The concentration of the Sodium ions inside the cell is to high and the Potassium ion concentration is to low. So, the pump works by pumping 3 sodium ions in while 2 potassium ions are pumped out.

The pump works by having ATP and 3 sodium ions bind to the pump. Then the ATP is hydrolyzed, which releases ADP that causes a comformational change in the pump and the sodium ions are exposed outside of the cell.

(c) CELEST 200713

Sodium/Potassium Pump Two potassium ions then bind

with the pump

ATP binds to the pump again causing it to reorient and the potassium ions are released into the cell.

The process then starts all over again. So the whole process continues to cycle until the rest potential is restored.

(c) CELEST 200714

Action Potential 2

The graph shows the action potential process in terms of the electric impuse. It is important to know that the cell can depolarize in the positive or negative direction.

Getting a visual of this is very important, so check out these animations

(c) CELEST 200715

HOW NEURONS COMMUNICATEWhen the electric impulse

reaches the axon terminals the electrical signal is converted to a chemical signal

These chemical signal are called neurotransmitters, which can be either excitatory or inhibitory

Neurotransmitters are released from the axon terminal through the synapse to the dendrite terminals of one or many other cells

Axon Terminal

Synapse

Neurotransmitter

Dendrite Terminal

(c) CELEST 200716

Types of Neurotransmitters The many different types of neurotransmitters are

contained within the vesicles. Each vesicle contains a specific type of neurotransmitter. On the next slide is a sample list of some of the more common neurotransmitters and their functions. Some of them can be excitatory which means that when they hit the receptors it causes a depolarization on the post synaptic neuron. (causing the peak on the graph to go up)

The other neurotransmitters are inhibitory which means they hyperpolarize postsynaptic neuron. (causing the peak on the graph to go down)

(c) CELEST 200717

Neurotranmitters 2Acetylcholine - voluntary movement of the muscles mostly excitatory Norepinephine- wakefulness or arousal, excitatory Dopamine - voluntary movement and motivation,

"wanting" , excitatory or inhibitory Serotonin - memory, emotions, wakefulness, sleep and

temperature regulation, excitatoryGABA (gamma aminobutyric acid) - inhibition of motor

neurons, inhibitorGlycine- spinal reflexes and motor behavior,mostly inhibitory

(c) CELEST 200718

SYNAPSE

Most synapses are unidirectional: one neuron sends a neurotransmitter to the other at that synapse across the synaptic cleft, but not the other way

around. The neuron who sends the neurotransmitter is called the presynaptic neuron.The neuron who receives the chemical messenger is called the postsynaptic neuron.

More on Synapse

Synaptic cleft

(c) CELEST 200719

Neurotransmitters When the action potential reaches the synapse the

depolarization causes calcium ion channels to allow calcium ions in which allows the vesicles to fuse with the membrane, and the vesicles to release the neurotransmitters from the presynaptic neuron axon terminal. The transmitter then can bind with the postsynaptic dendrite arm receptors. This binding then can begin the whole transmission process. The receptors will then either release the neurotransmitter to be recycled in a process called uptake or it will be broken down by enzymes. Lets watch several clips.

(c) CELEST 200720

Synapse Clip 1

(c) CELEST 200721

Synapse Clip 2

(c) CELEST 200722

Synapse Clip 3

(c) CELEST 200723

EXCITING A POST-SYNAPTIC NEURON

The level of excitation the postsynaptic neuron can receive is a function of how many synaptic connections a neuron’s dendrite has, as well as how many receptor sites there are per synapse

(c) CELEST 200724

SIGNAL PROPAGATION

The whole circuit can be broken down into a number of neurons and synapses.

Each neuron is in a certain state of activation.

This state of activation can be transferred to another cell via synapses.

synapse

(c) CELEST 200725

TO BRAINS…

Anatomy, morphology, physiology, specialization…

THE BASIC PARTS OF THE BRAIN

(c) CELEST 200727

MAP OF THE CORTEXES

(c) CELEST 200728

INTERNAL STRUCTURES OF THE BRAIN

(c) CELEST 200729

YOUR “3-BRAINS IN ONE”

The Triune Brain

(c) CELEST 200730

“BRAIN 3”

(c) CELEST 200731

(c) CELEST 200732

“BRAIN 2”

• THE “OLD MAMMALIAN” BRAIN”

(c) CELEST 200733

(c) CELEST 200734

“BRAIN 1”

REFERENCES

BOOKS, WEBSITES

(c) CELEST 200736

References• Diamond, Marian. MAGIC TREES OF THE MIND.• Jensen, Eric. TEACHING WITH THE BRAIN IN MIND• TEACHING WITH THE ARTS IN MIND • LeDoux, Joseph. THE SYNAPTIC SELF.• Ratey, John. A USER’S GUIDE TO THE BRAIN.• Wolfe, Patricia MIND MATTERS • Wolfe, Patricia BUILDING THE READING BRAIN• Brain website #1• Brain website #2• Brain website #3• Brain website#4• Brain website #5• Brain website #6

(c) CELEST 200737

TO NEURAL NETWORK MODELS…

Goal: represent, explain and predict reality (neuron, neural mass, electronic properties, chemical reactions, brain function, animal and human behavior)

Methods: directed graph, mathematical equation

Analysis technique: equilibrium analysis, simulations with systematic parameter variation

Key concepts: constant and phased input, bottom up activation, top-down priming, matching…

(c) CELEST 200738

A mature science of learning requires that we understand how

BRAIN MECHANISMS

give rise to BEHAVIORAL FUNCTIONS

MODELING HOW THE BRAIN LEARNS

(c) CELEST 200739

Mind-Body Problem

Many groups study BRAIN OR BEHAVIOR

BRAIN provides MECHANISMS BEHAVIOR provides FUNCTIONS

Without a link between them

BRAIN MECHANISMS have no FUNCTIONBEHAVIORAL FUNCTIONS have no MECHANISM

WHY IS IT IMPORTANT TOLINK BRAIN TO BEHAVIOR?

CELEST provides this link!

(c) CELEST 200740

What level of brain organizationcontrols behavior?

What is the functional unit of behavior?

BRAIN evolution needs to achieveBEHAVIORAL success

What level of BRAIN processing governs BEHAVIORAL success?

HOW DOES THE BRAIN CONTROL BEHAVIOR?

40 years of modeling show:The NETWORK and SYSTEM levels!

(c) CELEST 200741

BEHAVIOR IS AN EMERGENT PROPERTY OF NEURAL NETWORKS

Need to simultaneously describe 3 levels (at least): BEHAVIOR

NETWORK NEURON

and a MODELING language to link them

How are individual NEURONS designed and connected so that the NETWORKS they comprise generate emergent properties that govern successful BEHAVIORS?

Does this mean that individual neurons are unimportant?Not at all!

CELEST studies all of these levels simultaneously

(c) CELEST 200742

HOW MODELS LINK BRAIN TO BEHAVIOR

A successful MODELING APPROACH has unified these levels during 40 years of modeling led by CELEST scientists. In this approach, you analyse:

REAL-TIME AUTONOMOUS LEARNING SYSTEMS!

This theme makes possible a MODELING CYCLE that can link brain to behavior

How an individual adapts on its own in real time to a complex and changing environment

(c) CELEST 200743

Design Principles

Mathematicaland Computer

Analysis

Technological Applications

MODELING CYCLE

BehavioralData

Predictions

MIND

Neural Data

Predictions

BRAIN

(c) CELEST 200744

TWO KEY CONCLUSIONS

1. Advanced brains look like they do to enable

Lesson: The Architecture is the Algorithm

Lesson: You cannot fully understand adult neural information processing without studying how the brain LEARNS

2. Recent models show how the brain’s ability to DEVELOP and LEARN greatly constrain the laws of

REAL-TIME AUTONOMOUS LEARNING

ADULT INFORMATION PROCESSING

(c) CELEST 200745

BEHAVIORAL AND BRAIN MODELING

of normal and abnormal LEARNING during

Perception

Cognition

Emotion

Action

Discovers MECHANISMS that control learning

MODELS SERVE AS CELEST UNIFYING THEMES

(c) CELEST 200746

THINKING OUTSIDE THE BOX

Models are not only tied to data

How can we begin to know how the brain works? Think about it!

Thought experiments are often used to consider what must be true for particular situations to exist

(c) CELEST 200747

EXAMPLE 1: STABILITY-PLASTICITY DILEMMA

(c) CELEST 200748

EXAMPLE 2: MASS ACTION

(c) CELEST 200749

EXAMPLE 3: THE GATED DIPOLE

(c) CELEST 200750

DESIGN OF LEARNING ENVIRONMENTS:HOW PEOPLE LEARN

Learner-centered Focus on student’s previous knowledge and misconceptions

Knowledge-centered Structured towards progressive formalization of knowledge

Promote deep understanding and subsequent transfer

Assessment-centered Constant and interactive feedback

Community-centered Universally relevant and applicable topic, easily applied to everyday experience and problem solving

(c) CELEST 200751

LEARNER-CENTERED

CELEST curriculum focuses on the learner creating a deep understanding about how their brain works.

Example: BrightnessLab corrects misconceptions about how vision works

(c) CELEST 200752

KNOWLEDGE-CENTERED

CELEST curriculum is knowledge centered because it provides for progressive formalization using a system of models that range from those similar to everyday experience to increasingly abstract conceptual designs and mathematical formalizations and analysis.

(c) CELEST 200753

KNOWLEDGE-CENTERED MODELING

Petrosino, 2003

EXAMPLE: BrightnessLab Models

(c) CELEST 200754

ASSESSMENT-CENTERED

CELEST curriculum promotes interaction between students and their peers, students and their teacher, and students and the computer

Student activities are designed to provide formative feedback

Summative feedback activities are designed to test students’ content-knowledge and provide an arena to help students develop strategies to expand and transfer their knowledge to solve a wider variety of problems

(c) CELEST 200755

COMMUNITY-CENTERED

CELEST curriculum is universally relevant and applicable to all people and readily transferred to everyday experience

Everybody has a brain!

(c) CELEST 200756

WHAT WEB-BASED CURRICULUM EXISTS?

http://cns.bu.edu/CELEST/http://cns.bu.edu/CELEST/http://cns.bu.edu/CELEST/privatehttp://cns.bu.edu/CELEST/private

BrightnessLab: Seeing is Believing / Brightness Contrast

Sequence Learning: Make Your Memory Stronger!

Associative Learning: Learning in the blink of an eye

Obstacle Avoidance Navigation: Watch Where You’re Going!

Recognition: How do we know what we know?

(c) CELEST 200757

WHY WE BEGIN WITH THESE MODULES

Perception: the basis for knowledge about the world. Half of the brain is dedicated to visual processing. BrightnessLab begins the systematic study of visual processing

Action: reflex and planned movements. Given a goal, how do we decide to move as a reaction to sensory (visual) input? Obstacle Avoidance Navigation explores the question of reactive movement as opposed to memory guided movement

Cognition: how we know that we know. We begin the exploration of memory and learning by studying Sequence Learning of numerical lists and continue through an examination of Recognition and metacognition

Emotion: spontaneous physical and mental states. Since perception, action & cognition are all mediated by emotions, we begin to address them by studying adaptive timing, a basic form of Associative Learning

(c) CELEST 200758

COMMON PRINCIPLES

Laminar or layered organization

Parallel and interaction processing streams

Activation (excitatory and inhibitory) has a limit

Activation will naturally (passively) decay

(c) CELEST 200759

BREAKTHROUGHS IN BRAIN COMPUTINGModels that link detailed brain CIRCUITS to the ADAPTIVE BEHAVIORS that they control

INDEPENDENT MODULES Computer Metaphor

COMPLEMENTARY COMPUTING Brain as part of the physical world

Describe NEW PARADIGMS for brain computing

LAMINAR COMPUTINGWhy are all neocortical circuits laminar?

How do laminar circuits give rise to biological intelligence?

Mind/Body Problem

(c) CELEST 200760

Principles ofUNCERTAINTY and COMPLEMENTARITY

Multiple Parallel Processing Streams Exist

UNCERTAINTY PRINCIPLES operate at individual levelsHierarchical interactions resolve uncertainty

Each stream computes COMPLEMENTARY propertiesParallel interactions overcome complementary weaknesses

HIERARCHICAL INTRASTREAM INTERACTIONS

PARALLEL INTERSTREAM INTERACTIONS

ADAPTIVE BEHAVIOR = EMERGENT PROPERTIES

(c) CELEST 200761

VISUAL BOUNDARY AND SURFACE COMPUTATIONS ARE COMPLEMENTARY

orientedinwardinsensitive to direction-of-contrast

unorientedoutwardsensitive to direction-of-contrast

BOUNDARYCOMPLETION

SURFACE FILLING-IN

Neon color spreading

(c) CELEST 200762

Object plans and working memory

Spatial plans and working memory

Spatially invariant object recognition and attention

Spatialattention and tracking

3-D filling-in of binocular surfaces and figure-ground perception

Predictive target tracking and background suppression

Optic flow navigation and image stabilization

Depth-selective capture and filling-in of monocular surfaces

3-D boundarycompletion and separation of occludingand occluded boundaries

Enhancement ofmotion directionand featuretracking signals

Monoculardouble-opponentprocessing

Stereopsis Motion detection

Photodetection and discount illuminant

IT PPC

V4 MST

V2V2

MT

V1

Boundary-surface consistency

Formotionbinding

Retina and LGN

PFC PFC

WHAT STREAM WHERE STREAM

Object plans and working memory

Spatial plans and working memory

Spatially invariant object recognition and attention

Spatialattention and tracking

3-D filling-in of binocular surfaces and figure-ground perception

Predictive target tracking and background suppression

Optic flow navigation and image stabilization

Depth-selective capture and filling-in of monocular surfaces

3-D boundarycompletion and separation of occludingand occluded boundaries

Enhancement ofmotion directionand featuretracking signals

Monoculardouble-opponentprocessing

Stereopsis Motion detection

Photodetection and discount illuminant

IT PPC

V4 MST

V2V2

MT

V1

Boundary-surface consistency

Formotionbinding

Retina and LGN

PFC PFC

WHAT STREAM WHERE STREAM

CELEST PROJECTS TO DEVELOP UNIFIED MODEL OF HOW VISUAL CORTEX SEES

BOTTOM-UPTOP-DOWNHORIZONTALinteractions everywhere toovercomeCOMPLEMENTARYWEAKNESSES

Not independentmodules

(c) CELEST 200763

BIOLOGICAL TAKE HOME LESSONS

1. Need to model PAIRS OF COMPLEMENTARY CORTICAL STREAMS

to computeCOMPLETE INFORMATION

about a changing world

2. Need INTERACTING TEAMS OF SCIENTISTS

A CENTER! to characterize the large

FUNCTIONAL BRAIN SYSTEMS that control adaptive behavior

(c) CELEST 200764

COMPLEMENTARY STREAMS COOPERATE TO COMPUTE COMPLETE INFORMATION

Perception-cognition-emotion-action systems use several types of

MULTI-DIMENSIONAL LEARNED INFORMATION FUSION

Multiple sources of partial information are combined during learning

Complementary types of learning work together to solve environmental problemse.g., What-Where learned information fusion

CELEST thrusts are designed to model how this works

(c) CELEST 200765

CELEST MODELS COMPLETE BRAIN SYSTEMS

Perception-cognition-emotion-action systems use several types of

Multiple sources of partial information are combined during learning

Complementary types of learning work together to solve environmental problems

e.g., What-Where learned information fusion

Not a future wish; a present coordinated research program

MULTI-DIMENSIONAL LEARNED INFORMATION FUSION

(c) CELEST 200766

WHY THESE PARTICULAR THRUSTS?ORDINARY BEHAVIORS USE LARGE FUNCTIONAL

BRAIN SYSTEMS

Child’s task: Visually find and pick up a stationary cup of milk to drink

Spatially orient to the cup Where stream 3See cup What stream 1Recognize cup What stream 1 Want to pick cup up What stream 3Plan to pick cup up What-Where stream 3,5Pick cup up What-Where stream 1,3,5

THRUST

This perception-cognition-emotion-action cycleuses What-Where learned information fusion

Need visual, temporal, parietal, prefrontal cortices...

(c) CELEST 200767

WHY THESE PARTICULAR THRUSTS?ORDINARY BEHAVIORS USE LARGE FUNCTIONAL

BRAIN SYSTEMS

Child’s task: Orient to mother’s voice and say: “Mommy, give me milk”

Hear mother’s voice What stream 2Recognize voice What stream 2Spatially orient to voice Where stream 3Want to talk to mother What stream 3Plan to talk to mother What-Where stream 3,5 Talk to mother What-Where stream 2,3,5

THRUST

This perception-cognition-emotion-action cyclealso uses What-Where learned information fusion

Need auditory, temporal, parietal, prefrontal cortices...

(c) CELEST 200768

CELEST THRUSTS ENABLE MODELING OF COMPLETE BRAIN SYSTEMS

Perception-cognition-emotion-action systems enable the brain to learn adaptive behaviors in real time within a changing world

Just as important for

developing new engineering systems that intelligently process huge amounts of data in unpredictably changing environments

providing insights into how to improve

learning in the classroom

(c) CELEST 200769

WHY IS THIS POSSIBLE NOW?Recent models and modeling PARADIGMSdeveloped by CELEST scientists:

COMPLEMENTARY COMPUTINGand

LAMINAR COMPUTINGhave begun to clarify how these large functional brain systemscompute the sort of

complete informationthat controls successful

adaptive behaviors

CELEST brings together personnel and resources needed to take the next steps

(c) CELEST 200770

A NEURON-INSPIRED MODEL

xi zij xj

vi eij vj

Source: http://webspace.ship.edu/cgboer/neuron.gif© Copyright 2003 C. George Boeree

xi Short-term memory traces

vi Cell populations

eij Axons

zij Long-term memory traces

xj Short-term memory traces

for the next neuron

vj Cell populations

Source: S. Grossberg (1988). Nonlinear neural networks: Principles, mechanisms, and architectures. Neural Networks, 1, 17-61.

Key:

(c) CELEST 200771

GRAPHING CONVENTIONS

Modulators Learned weights

Excitation

Inhibition

--

++

(c) CELEST 200772

TYPES OF CONNECTIONS

Convergent Divergent

“In-star” “Out-star”

(c) CELEST 200773

TYPES OF CONNECTIONS

Feedforward Feedback

VARIETIES OF LEARNING MUST BE MODELED

RecognitionReinforcement TimingSpatialMotor Control

IdentifyEvaluateSynchronizeLocateAct

WhatWhy WhenWhereHow

(c) CELEST 200775

A model that clarifies how animals learn to attend to external events that predict satisfaction of internal drives in real time

Autonomous Adaptive Mobile Robots

e.g., MAVIN RobotWaxman et al., MIT Lincoln Lab

CogEM MODEL

(c) CELEST 200776

+

SCS1

SCS2

CS1 CS2

D

SENSORY

MOTOR

DRIVE

Competitionfor STM

ConditionedReinforcerLearning Incentive

MotivationalLearning

Internal Drive Input

MotorLearning

+ +

CogEM MODEL:3 Types of Representations and Learning

Grossberg, 1971+

(c) CELEST 200777

DRIVE REPRESENTATIONS

Sites where reinforcement and homeostatic inputs interact to generate emotional and motivational output signals

Emotion nodesBower et al., 1981

Adaptive Critic ElementsBarto, Sutton, and Anderson, 1983

Facilitator Neuron (Aplysia)Walters and Byrne, 1983Hawkins, Abrams, Carew, and Kandel, 1983

(c) CELEST 200778

NEURAL DRIVE REPRESENTATIONS

Facilitator Neuron (Aplysia)

Walters and Byrne, 1983

Hawkins, Abrams, Carew, and Kandel, 1983

Amygdala

Aggleston et al., 1995

LeDoux et al., 1988

(c) CELEST 200779

INTERPRETATION OF CogEM ANATOMY

SENSORYCORTEX

PREFRONTALCORTEX

AMYGDALA

DRIVE

(c) CELEST 200780

AMYGDALA AND NEARBY AREAS

(c) CELEST 200781

Adapted from Barbas (1995)

Visual Cortex

AuditoryCortex

GustatoryCortex

OlfactoryCortex

Lateral PrefrontalCortex

OrbitalPrefrontalCortex

AMYGDALA

SomatosensoryCortex

(c) CELEST 200782

Buonomano, Baxter, & Byrne, Neural Networks, 1990

Grossberg, Behavioral and Brain Sciences, 1983

FACILITATOR NEURON ~ DRIVE REPRESENTATION

+

+

+

+

-

-

APLYSIA

Why similar circuit in MAMMALS and INVERTEBRATES?

Both solve similar environmental/behavioral problems!

SYNCHRONIZATION PROBLEM

Variable CS-US Delays

PERSISTENCE PROBLEMS

Multiple emotional meanings

CS1 CS2

CR1 CR2

Food Sex

Grossberg (1975)

(c) CELEST 200784

Why is ONSET of a shockNEGATIVELY REWARDING?

Why is OFFSET of a shock POSITIVELY REWARDING?

OPPONENT EMOTIONS IN DRIVE REPRESENTATIONS

FEAR vs RELIEF

FEAR

RELIEF

(c) CELEST 200785

REINFORCEMENT

VISUAL PERCEPTION

MacKay Illusion

ON

OFF

Shock on Fear (Estes & Skinner, 1941)Shock off Relief (Denny, 1971)

Picture off Negative Aftereffect

Picture on Percept

OPPONENT REBOUND IS UBIQUITOUS

(c) CELEST 200786

CS

US

Fear

Relief

OPPONENT PROCESSINGCognitive-Drive Associations

Primary excitatory associations Primary inhibitory associations

CS

US

Fear

CS

Fear

CS

Fear Relief

ON OFF

CS

Relief

Rebound

(c) CELEST 200787

BEHAVIORAL CONTRAST: REBOUNDS!

1. A sudden DECREASE in frequency or amount of FOOD can act as a NEGATIVE reinforcer: Frustration

2. A sudden DECREASE in frequency or amount of SHOCK can act as POSITIVE reinforcer: Relief

ShockLevel

Trials

(c) CELEST 200788

Responses per minute (VI schedule)

Daily sessions

TRIAL SHOCK LEVEL

1-56-10

11-1516-2021-25

0 Moderate 0 Intense 0

BEHAVIORAL CONTRAST

Reynolds (1968)

(c) CELEST 200789

MULTIPLE FUNCTIONAL ROLES OF SHOCK

1. Reinforcement sign reversalAn ISOLATED shock is a negative reinforcerIn certain CONTEXTS, a shock can be a positive reinforcer

2. STM-LTM interactionPrior shock levels need to be remembered (LTM) and used to calibrate the effect of the present shock (STM)

3. DISCRIMINATIVE AND SITUATIONAL CUESThe present shock level is UNEXPECTED (NOVEL) with respect to the shock levels that have previously been contingent upon experimental cues

1. Shock as a reinforcer

3. Shock as an expectancy2. Shock as a sensory cue

(c) CELEST 200790

How are ON and OFF reactions generated at the drive representations?

Through a

GATED DIPOLE

OPPONENT PROCESS

Grossberg (1972)

OPPONENT PROCESSING

(c) CELEST 200791

UNBIASED TRANSDUCER

S = inputT = outputT = SB

Suppose T is due to release of chemical transmitter y at a synapse:

S T y

RELEASE RATE: T = S y (mass action)

ACCUMULATION: y = B~

B is the gain

Grossberg (1968)

(c) CELEST 200792

TRANSMITTER ACCUMULATION AND RELEASE

T = S yy B

Differential Equation:

Transmitter y tries to recover to ensure unbiased transduction

y = A (B – y) – S ydt

d

Accumulate Release

What if it falls behind?

Transmitter y cannot be restored at an infinite rate:

Evolution has exploited the good properties that happen then

(c) CELEST 200793

HABITUATIVE TRANSMITTER GATE

T = S y

y = A (B – y) – S ydt

d

Recent experiments support this prediction:

Visual Cortex: Abbott et al. (1997): depressing synapses

Somatosensory Cortex: Markram et al. (1998)

(c) CELEST 200794

MINOR MATHEMATICAL MIRACLE

At equilibrium:

0 dy

dtA(B y) Sy

y AB

A S

y decreases when input S increases:

However, output Sy increases with S!

Sy ABS

A S(gate, mass action)

Transmitter

(c) CELEST 200795

HABITUATIVE TRANSMITTER GATE

ABS1

A S0

Weber Law

ABS0

A S 1

(c) CELEST 200796

NONRECURRENT GATED DIPOLE

(c) CELEST 200797

y1

ON-RESPONSE TO PHASIC ON-INPUT

S1=f(I+J) S2=f(I)

11

SA

BA y

22

SA

BA y

1

1111

SA

ABSyST

2

2222

SA

ABSyST

+ +- -

y2

s2s1

IJ

OFFON

T1T2

J))f(If(I))(A(A

f(I))-J)B(f(IAT-TON

2

21

Note Weber Law

When f has a threshold, small I requires larger J to fire due to numerator, but makes suprathreshold ON bigger due to denominator

When I is large, quadratic in denominator and upper bound of f make ON small

(c) CELEST 200798

OFF-REBOUND DUE TO PHASIC INPUT OFFSET

Shut off J (Not I!). Then:

A J)f(I

AB

1y f(I)A

ABy

2

y1 and y2 are SLOW

T1 = S1y1 T2 = S2y2

T1 < T2

J))f(If(I))(A(A

f(I))J)ABf(I)(f(ITTOFF

12

=

Why is the rebound transient? Note equal f(l) inputs

A

f(I)

ON

OFF=Arousal sets sensitivity of rebound:

<

S1 = f(l) and S2 = f(l)

Note Weber Law due to remembered previous input

(c) CELEST 200799

NOVELTY RESET: REBOUND TO AROUSAL ONSET

Equilibrate to I and J: S1=f(I+J) S2=f(I)

11 SA

BA y

2 S2A

BA y

Keep phasic input J fixed; increase arousal I to I* = I + ∆ I:

J))f(If(I))(A(A

J)*f(I)f(I-J)B(f(I*)f(I-J))*f(I-AB(f(I*)

How to interpret this complicated equation?

OFF = T2 - T1 = f(I*+J) y2 - f(I*) y1

OFF reaction if T1 < T2

(c) CELEST 2007100

NOVELTY RESET: REBOUND TO AROUSAL ONSET

f(w) f(w)= Cw: Linear signal

A)-IABJ(OFF

J)II)(A(A

OFF > 0 only if there is enough novelty: ∆I > A

∆I = I*- I

OFF response increases with J: If a given cell has a greater effect on a mismatchedexpectation, then it is reset more vigorously

Selective reset of dipole field by unexpected event

(c) CELEST 2007101

GOLDEN MEAN

Behavior

Arousal

Underaroused Depression Overaroused Depression

“UP” brings excitability “DOWN”

INVERTED U AS A FUNCTION OF AROUSAL

Elevated thresholdHyperexcitable above threshold

Low thresholdHypoexcitable above threshold

J))f(If(I))(A(Af(I))-J)B(f(IA2

ON

(c) CELEST 2007102

Consider the simplest type of

COGNITIVE-EMOTIONAL LEARNING

(c) CELEST 2007103

CLASSICAL CONDITIONING(Nonstationary prediction)Bell (CS) Bell (CS)

Fear (UR)

(CR)

ASSOCIATIVE LEARNING

A BCS US

ABCS US

ABCS US

A BCS US

CR

(c) CELEST 2007104

INTERSTIMULUS INTERVAL (ISI) EFFECT

ISI

CS

US

CR

0

0 ISILarge ISI obvious: No CS-US correlation

Why poor learning at 0 ISI, with good correlation?

(c) CELEST 2007105

INTERSTIMULUS INTERVAL (ISI) EFFECT

(c) CELEST 2007106

SECONDARY CONDITIONING(Advertising!)

CS1 becomes a CONDITIONED REINFORCER

CS2 becomes a CONDITIONED REINFORCER

CS1

CS2

CS1

FEAR

FEAR

US

(c) CELEST 2007107

How are

CLASSICAL CONDITIONING

and

ATTENTION

related?

(c) CELEST 2007108

PARALLEL PROCESSING OF EQUALLY SALIENT CUES

CS2

Light

FEAR

US

CS1

Bell t

t

t

t

vs. OVERSHADOWING (Pavlov)

CS1

CS2

FEAR

FEAR

(c) CELEST 2007109

BLOCKINGMINIMAL ADAPTIVE PREDICTION

CS1

CS2

FEAR

FEAR

CS2 IS IRRELEVANT

Phase I

Phase II

US

CS1

CS2

US

CS1

(c) CELEST 2007110

BLOCKING = ISI + SECONDARY CONDITIONING

Blocking Zero ISI

1)

2)

CS1

Fear

US

CS2

Fear

CS1

CS

Fear

US

No CS2 conditioning No CS conditioning

(c) CELEST 2007111

CS1 becomes a conditioned reinforcer by learning to activate a strong reinforcer-motivational (emotional) feedback pathway

Sensory Representation

Incentive Motivation

Drive Representation

Conditioned Reinforcer

CS1

US

+

CONDITIONED REINFORCER

(c) CELEST 2007112

CogEM EXPLANATION OF ATTENTIONAL BLOCKING

1. Sensory representations compete for LIMITED CAPACITY STM2. Previously reinforced cues amplify their STM via

POSITIVE FEEDBACK3. Other cues lose STM via COMPETITION

Competitionfor STM

SENSORY

MOTORDRIVE

SCS1

SCS2

CS1 CS2

+

+

D

ConditionedReinforcerLearning Incentive

MotivationalLearning

Internal Drive Input

MotorLearning

+

(c) CELEST 2007113

CS

SensoryInput

STMactivitywithoutmotivationalfeedback

STMactivitywith motivationalfeedback

+

time

(c) CELEST 2007114

BLOCKING

STM suppressedBy competition

STM amplifiedBy (+) feedback

X2

X1

t

t

X2

X1

CS1

CS2

+

(c) CELEST 2007115

SCS SUS

Sampling interval

POSITIVE ISI

CS input

US input

SCS activity

SUS activity

(c) CELEST 2007116

ISI EFFECT

Grossberg and Levine, 1987

(c) CELEST 2007117

CS1 D LTM Trace

EMOTIONAL CONDITIONING

CS1

+

D

Anticipatory CRa c

b d

CS1’S STM trace

D’s STM Trace

US’s STM Trace

(c) CELEST 2007118

CS2 D LTM Trace

CS1’S STM trace

CS2’s STM Trace

a c

b d

CS1 D LTM Trace

CS1 - US CS1 + CS2 - CS2

US

CS1

+

CS2

BLOCKING

(c) CELEST 2007119

UNIFIED EXPLANATION OF BLOCKINGISI EFFECT

SECONDARY CONDITIONING ANTICIPATORY CR

COOPERATION

between

COGNITIVE and EMOTIONAL

representations

COMPETITION

between

COGNITIVE

representations

(c) CELEST 2007120

MINIMAL ADAPTIVE PREDICTION

BLOCKING UNBLOCKING

CS1 US

CS1Fear

CS1 US1

CS1 Fear

CS1 + CS2 US

CS2 Fear

CS2

Fear if US2 > US1

CS2 is irrelevant CS2 predicts US change

Learn if CS2 predicts a different (novel) outcome than CS1

CS2 not redundant (“wallpaper”)

CS1 + CS2US2

xRelief if US2 < US1

1) 1)

2) 2)

(c) CELEST 2007121

MINIMAL ADAPTIVE PREDICTOR

CS2 US2t

(US1 US2) ><

HOW ART WAS DISCOVERED IN 1973!

1. Pay attention to (code, learn) RELEVANT cues

CS1 predicts US1

2. Unexpected CONSEQUENCES redefine the set of relevant cues

Changing US1 to US2 makes CS2 relevant

3. Unexpected consequence (NOVELTY) feeds back in time via a NONSPECIFIC event to redefine relevant cues

4. Distinguish NOVELTY from EMOTIONAL SIGN