Classical Conditioning: Mechanisms and Theory

Eyeblink Class Study

• 60 conditioning trials (blocks of 20)

• 7 blocks of 4 probe trials

• C1, P1, C2, P2, C3, P3, P4, P5, P6, P7

• Acquisition, extinction

Results: Individual

100

75

50

25

1 32 54 6 7

% C

R in

Blo

ck

Blocks (of 4 probe trials)

Results: Averaged

100

75

50

25

1 32 54 6 7

% C

R in

Blo

ck

Blocks (of 4 probe trials)

Unconditional/Conditional

• US: elicits response without training

• Cs: elicits response due to training (association)

• Not quite so clear-cut

Consider

• Aversive conditioning: tone (CS), mild shock (US)

• Pavlov: mild shock(CS), food (US)

• Sign tracking: light (CS), saccharin (US)

• Taste aversion: flavour of saccharin (CS), illness (US)

Novelty

• Prior associations

• Familiar vs. unfamiliar stimuli

• Not “unlearning” of familiar stimuli, per se

• Basically, need to learn something different

Latent Inhibition/CS Preexposure

• Highly familiar stimuli more difficult to associate with US than novel stimuli

• Preexposure group

Phase 1 Phase 2 Phase 3

Exp. gr. “CS” alone CS-US test

Cont. gr. nothing CS-US test

Exp.Cont.

CR

mag

nitu

de

Latent Inhibition

• Habituation function

• Typically we think of habituating to a US; ambiguity in CS/US designation

• Attentional processes

• CS- could also explain, but doesn’t suppress responding to other CS+

US Preexposure

• Subjects exposed to US before CS-US pairings slower to produce CR

• Associative interference (Hall 2008)– Association of contextual CS with US

during US preexposure– In essence, need to extinguish context CS

to associate novel CS with US

• Could this be habituation of US, too?• Test methodology?

Ayres, Moore & Vigorito (1984)

• Stimulus salience

• Stimulus novelty

• Conditioned suppression

Method

• Stimuli– CS: tone, light– US: shock

• Stage 1: pair CS with US; suppression ratio

• Stage 2: pair second CS (novel or familiar) with US; suppression ratio

• Stage 3: extinction of second CS

Results

T-T L-TL-L T-L

Stage 1: 1st stim. & shock

0.5

0.4

0.3

0.2

0.1

Sup

pres

sion

Rat

io

2 4 6 8 10 Day

1 2 3 1 2

Stage 2: 2nd stim. & shock

Stage 3: 2nd stim. extinction

Tone

Light

Familiar Novel Familiars (T-T & L-L) show less suppression than novels (L-T & T-L): preexposure

Salience and Intensity• Salience: significance, noticeability,

detectability• Salience and intensity often used

synonymously– Low to moderate levels, probably interchangable– Consider high level stimulus– Physiological damage– Not salient, but definitely intense

• Better to treat intensity as a component of salience

Salience

• Increase via:

• Intensity

• Relevance– Physiological needs– Similarity of

environmental stimuli (e.g., naturalistic CS”)

Belongingness: Stimuli Relevance

• Equipotentiality principle• Pavlov• Any stimulus should, relatively, be equally

conditionable with any other stimulus– E.g., CS1 easily associated with US1, should also

be easily associated with US2– Easy-to-easy, hard-to-hard

• But doesn’t always work this way• Garcia & Koelling’s work on taste aversion

Stimuli Relevance

• Biological predispostions; evolved• Pigeons

– Visual CS associated more easily than auditory CS with food US

– But auditory CS easier than visual CS when shock is US

• Fear conditioning in primates (rhesus monkeys, human children)– CS of snake vs. flower

Wilcoxon et al. (1971)

• Biological preparedness in conditioning

• Rats nocturnal, quails diurnal

• Taste aversion

• Blue water, sour water

• Quails: colour --> stronger CS

• Rats: taste --> stronger CS

Higher Order

• Few pairings, higher-order

• Extensive training, CS-

• Solution: periodic reconditioning of first-order

CS1 US

CR

CS2 CS1

CR

First-order Second-order

CS3 CS2

CR

Third-order

Rizley & Rescorla (1972)

• Extinction of CS1 does not affect CS2

• CS1 = tone, US = shock, CS2 = light

• Experimental group: 1. CS1-US, 2. CS2-CS1, 3. extinguish CS1, 4. test CS1 & CS2

• Control group: 1. CS1-US, 2. CS2-CS1, 3. nothing, 4. test CS1 & CS2

Results

0.5

0.4

0.3

0.2

0.1

Sup

pres

sion

Rat

io

1- 2 3-4 5-6 7-8 1-2 3-4 1-4 5-8 9-12

Acquisition CS2 (light) CS2 (light)

test

CS1 (tone)

test

Exp. gr.

Cont. gr.

CS2 still shows suppression

for both exp. & cont. groups…

Even though CS1 shows no

suppression in exp. group.

Holland & Rescorla (1975)

• CS1 = light, CS2 = tone, US = food

• CS1-US then CS2-CS1

• Then satiate (devalue) US• Test:

– CS1’s CR weakened

– CS2’s CR unaffected

• Manipulating CS1-US relationship doesn’t seem to affect CS2’s representation

Sensory Preconditioning

• Pair two stimuli (e.g., light and tone)

• Pair one with US… becomes a CS

• Now second stimulus also makes CR

Blocking

• Pair CS1 and US repeatedly

• Make compound CS1-CS2 and keep pairing with US

• CS1 gives strong CR

• CS2 gives weak CR

Value of Classical Conditioning

• Preparedness

• Evolution, survival mechanisms

• Foresight, anticipation

Zamble et al. (1985)

• Male rats

• Give male repeated access to receptive females; pair with explicit CS

• With CS, initiates copulation sooner, ejaculates quicker

• Competitive advantage over other males

Hollis (1984)

• Blue gourami

• Males hold territory

• Attack intruders

• Condition light with intruder– Resident attacks intruder sooner– Resident won conflict more often

Learning and Homeostasis

• Preparation• Homeostatic systems• Feedback lag in control system• Classical conditioning associations can

influence homeostatic systems• Prepare for events that will perterb the

system• Minimize lag

Effects

• Generally, very adaptive• However, sometimes difficulties• Conditioned compensatory responses• Drug tolerance• S. Siegel’s work on drug tolerance

– Contextual CS prepare opposing CR to maintain homeostasis

– Difficulties if contextual CS absent

Stimulus Substitution Theory

• Pavlov’s theory

• Through repeated pairings of the CS and the US the CS becomes a substitute for the US so that all responses initially elicited only by the US are now also produced by the CS

Jenkins & Moore (1973)

• Pigeons• Food or water as US• CRfood = pecked

response key as if eating; rapid pecks with open beak

• CRwater = pecked response key as if drinking; slower pecking with beak closed, often with swallowing water food

Problems with Stimulus Substitution Theory

• CS not a complete substitute for US– E.g., eyeblink differences– Magnitudes

• CSs produce different responses– Omissions and additions– E.g., conditioned suppression in rats

• US = shock, UR = flinch, CS = tone, CR = freeze

• Conditioned compensatory responses

Nervous System

What is Learned in Classical Conditioning?

• US centre, Response centre, CS centre

CS Centre

US Centre Response Centre

US

CS

Response

S-S

S-R

S-S or S-R Connections?

• Stimulus-Stimulus (S-S) Theory– Two associations

• Learned CS centre to US centre

• Innate US centre to response centre

• Stimulus-Response (S-R) Theory– One association

• CS centre to response centre

CS Centre

US Centre Response Centre

CS Centre

US Centre Response Centre

Rescorla’s (1973) Experiment• Habituation to weaken US-response link

• Conditioned suppression procedure– Loud noise

• Experimental protocol

Group Phase 1 Phase 2 Test

Habituation

Control

Noise(habituation)

No stimuli

Light: low CR

Light: high CR

Light Noise

Light Noise

ResultsControl: high CRHabituation: low CRSupports S-S theory

CS/US Influence on CR

• Timberlake & Grant (1975)– Second rat as CS in sign tracking– CR --> social, not consumatory behaviour

• Akins (2000)– Male quails’ behaviour sequence– General or focal search for female– CS = visual stimulus, US = female quail– CS-US interval short (1 min.) or long (20 min.)

Akins (2000)%

Tim

e ne

ar C

S

Focal Search

CS-US Interval (min.)

1 20C

ross

ings

(P

acin

g)

General Search

CS-US Interval (min.)

1 20

Paired CS &US

Unpaired CS & US

Re: S-S Theory

• Requires flexibility in CR-UR relationship

• CSs not associated with all aspects of US

• CS and US can interact

• CR depends on sensory properties of CS and presentation context

Rescorla-Wagner Model

• Learning is a discrepancy between– Expectation– Occurrence

• Level of surprise --> degree of conditioning– More surprising, more learning– Early vs. later trials

Mathematical Model

Vn = k( - Vn)

• V = CS-US associative strengthV = change in associative strength per

trial

• k = salience of stimuli = asymptotic maximum of V (due to

US) - V = “suprisingness”

R-W and Blocking

• CS1 paired with US repeatedly

• Vcs1 approaches l

• By the time CS2 added, very little associative strength left to be acquired

• CS1 very predictive of US; little “surprise” left, so not much need for CS2

Overexpectation Effect

• Predicted by Rescorla-Wagner model before being empirically demonstrated

Group Phase 1 Phase 2 Test Results

Overexpectation

Control

L ... 1 foodT ... 1 foodL ... 1 foodT ... 1 food

L+T ... 1 food

no stimuli

LTLT

moderate CRmoderate CR

strong CRstrong CR

Conditioned Inhibition• Excitatory CS paired with inhibitory CS• Opponent process system• Example: tone = CS+, light = CS-• Start values:

Vtone = 100.0, Vlight = 0.0, k = 0.2, = 0.0

• We defineVsum = Vtone + Vlight

• And Vn = k(Vmax - Vsum)

Vsum

Vtone

Vlight

100

50

0

-50

Ass

ocia

tive

Str

engt

h

Trials

Trial Vn Vtone Vlight Vsum

1 -20.0 100 0.0 1002 -12 80 -20 603 -7.2 68 -32 364 -4.3 60.8 -39.2 225 -2.6 56.5 -43.5 136 -1.6 53.9 -46.1 87 -0.9 52.3 -47.7 58 -0.6 51.4 -48.6 39 -0.5 50.8 -49.2 210 50.4 -49.6 0

80 -20 = 60

60.8 -39.2 = 22

CS+ CS-

V1=0.2(0-100)=-20

V2=0.2(0-60)=-12

V3=0.2(0-36)=-7.2

CS Preexposure Effect

• Preexposure group: give CS alone

• Control group: pair CS with US

• Test: how long to get CR in both groups

• “Habituation” in preexposure group– Learn to pay less attention to CS– CS irrelevant, nonpredictive– Must “unlearn” during test phase

CS Preexposure Effect

• Not explained by Rescorla-Wagner model• First preexposure trial

– No prior conditioning

Vn = 0, = 0

• But, something is learned• Salience variable, not constant

– Salience of CS decreases during preexposure– Pay less attention to CS

Other Issues

• Extinction– Not return to zero

• CS-– Extinguished not by being presented by

itself but by extinguishing CS+

• Stimuli must be either CS+ or CS-, not both depending on context

Attentional Models

• Numerous stimuli in environment

• Can’t attend to them all

• Selectively process (remove signal from noise)

• E.g., cocktail party effect– Attend to one conversation at a time– Your name

Mackintosh’s Theory

• Treats salience as a variable, not a constant

• Consider two stimuli, L and T• If L is a better predictor of the US, then

the salienceL will increase and salienceT will decrease

• Attend to the more informative stimulus

Mackintosh: Blocking

• Trained on CSL; salienceL high because CSL is predictive of US

• In compound CS phase this means salienceT will drop towards zero

• CST will receive little attention, hence the weak CR

Pearce & Hall’s Theory

• CSs become ineffective whenever the US is already well predicted

• If situation changes so that US is surprising then more learning about the CS

• Attention to CS depends on surprisingness of US on previous trial

• Assume surprisingness of US will alter attention paid to CSs on subsequent trials– Contrast to R-W (surprisingness of US on trial

determines what is learned on that same trial)

Attentional Theories and Blocking

• On first compound CS trial, CS2 should be quite surprising (previously only CS1 paired with US)

• Should result in heightened attention to (and learning about) CS2 on subsequent trials

• But, CS1 blocks learning about CS2 on first compound CS trial

Temporal Coding Hypothesis

• Contiguity

• ISI: short delay vs. long delay and trace

• ITI: generally, stronger CR with it is spaced further apart

• CS duration: can also influence learning

• Learn not only that CS is paired with US but also when the US will occur

Holland (2000)

• US = food CS = noise CR = time at food cup

• Two CS durations (trial duration = T) of 10 or 20 seconds

• Six it is (15 to 960 seconds)

• Results explainable by I/T ratio I/T Ratio

1.5 3 6 12 24 48

Tim

e at

Foo

d C

up (

CR

)

T = 10 sec.

T = 20 sec.

Rate Estimation Theory

• Extension of relative-waiting-time hypothesis– CS only informative about US if you spend less time

waiting for US when CS is present

• Nonassociative theory• CRs reflect subjects estimates of rate of US

presentations during CS and absence of CS• Doesn’t fit will with neurophysiological data on

associative learning• Heavy computational burden

– May work in controlled (restricted) laboratory environment, but in real world environment?

Comparator Hypothesis

• Traditional interpretation is that blocking from failure to learn about CS2

• Comparator assumes that subject learns about CS2, but ability to respond is blocked

• Revaluation effects

• Extinguishing CS1 can result in CS2 now producing a CR

Comparator Hypothesis

• Theory of performance, not learning

• Condioned responding depends on: – Associations between CS and US– Associations between US and other stimuli

(comparator cues; may include experimental context)

Comparator Hypothesis

• Only allows formation of excitatory associations with US

• Excitation or inhibition determined by relative strengths of excitatory conditioning to target CS as opposed to other comparator stimuli

• If excitatory value of CS greater than that of comparator stimuli, then CS+; if lower, then CS-

• In essence, another opponent process model