PSY402
Theories of Learning
Chapter 8, Theories of Appetitive and Aversive
Conditioning
Operant Conditioning
The nature of reinforcement:
Premack’s probability differential theory
Response deprivation theory
Behavioral economics:
Behavioral allocation – blisspoint
Choice behavior – Herrnstein’s matching law.
Momentary maximization theory
Delay-reduction theory
Probability-Differential Theory
Premack – a reinforcer can be any activity that is more likely to occur than the reinforced behavior.
Manipulators vs eaters
High probability behaviors can be used as reinforcers of low probability behaviors.
Frequency of the reinforcer decreases when it is made contingent on another response.
Activities can be Reinforcers
Playing with toys reinforces
working math problems
correctly
Response Deprivation Theory
Timberlake & Allison – deprivation occurs
when an activity is used as a reinforcer and is
not freely emitted.
The activity is reinforcing because it satisfies the
deprivation created.
The animal tries to return to its pre-deprivation
level of responding.
Activities can be reinforcing even if their
initial baselines were not higher.
Behavioral Allocation
Blisspoint (paired basepoint) – the free operant level of two responses.
Unrestricted responding with two choices of behaviors.
Blisspoint is used to figure out how much behavior an animal will engage in to obtain a reward.
Animals try to get as close to the blisspoint as possible.
Finding the Blisspoint
Contingency Lines for Rewards
Problems with Contingencies
Blisspoint is established by looking at
behavior before a contingency is established.
The established contingency must take
blisspoint into account or it may not increase
desired behavior.
Choice Behavior
Herrnstein’s matching law – describes how
animals act when they have two or more
choices.
Different responses have different schedules of
reinforcement.
Responding to each choice is proportionate to the
reinforcement for each choice – after learning.
This can be expressed mathematically.
Mathematical Expression
The formula for the matching law is:
where R1 and R2 are the rates of response for two
alternative responses
And r1 and r2 are rates of reinforcement for those
responses
Law Predicts Pecking Behavior
Delayed Gratification
Why does anyone choose a smaller reward part of the time?
Animals and people typically choose a small immediate reward over a larger delayed reward.
Large rewards are selected when:
The choice is made in advance of reward.
Reinforcers are not visible or reward is already present (pleasurable activity).
Complexities of the Matching Law
Maximizing law – sometimes the aim is to
obtain as many rewards as possible.
Explains FR-10 vs FR-40 schedules.
Doesn’t work for VI vs VR schedules.
Momentary maximization theory – choose
best alternative at the time.
Delay reduction theory – choose what will get
the reward the fastest.
Aversive Theories: Explaining
Avoidance
The existence of avoidance behavior implies a
cognitive process:
Behaving in order to prevent an aversive event.
Behaviorists like Hull needed to explain this
without cognition.
Mowrer’s two-factor theory was developed to
explain this – but it has problems needing
explanation.
Mowrer’s Two-Factor Theory
Mowrer proposed a drive-based two-factor theory to avoid explaining avoidance using cognitive (mentalistic) concepts.
Avoidance involves two stages:
Fear is classically conditioned to the environmental conditions preceding an aversive event.
Cues evoke fear -- an instrumental response occurs to terminate the fear.
Mowrer’s View (Cont.)
We are not actually avoiding an event but
escaping from a feared object (environmental
cue).
Miller’s white/black chamber – rats escaped
the feared white chamber, not avoided an
anticipated shock.
Fear reduction rewards the escape behavior.
Criticisms of Two-Factory Theory
Avoidance behavior is extremely resistant to
extinction.
Should extinguish with exposure to CS without
UCS, but does not.
Levis & Boyd found that animals do not get
sufficient exposure duration because their
behavior prevents it.
Avoidance persists if long latency cues exist
closer to the aversive event.
Is Fear Really Present?
When avoidance behavior is well-learned the animals don’t seem to be afraid.
An avoidance CS does not suppress operant responding (no fear).
However, this could mean that the animal’s hunger is stronger than the fear.
Strong fear (drive strength) is not needed if habit strength is large.
Avoidance without a CS
Sidman avoidance task – an avoidance
response delays an aversive event for a period
of time.
There is no external cue to when the aversive
event will occur – just duration. Temporal
conditioning.
How do animals learn to avoid shock without
any external cues for the classical
conditioning of fear?
Kamin’s Findings
Avoidance of the UCS, not just termination of the CS (and the fear) matters in avoidance learning.
Four conditions:
Response ends CS and prevents UCS.
Reponse ends CS but doesn’t stop UCS.
Response prevents UCS but CS stays.
CS and UCS, response does nothing (control condition).
Both Factors are Important
Termination and
Avoidance both
show greater
learning
D’Amato’s Acquired Motive View
D’Amato proposed that both pain and relief motivate avoidance.
Anticipatory pain & relief responses.
Shock elicits unconditioned pain response RP and stimulus SP motivates escape.
Classically conditioned cues sP elicit anticipatory pain response rP that motivates escape from the CS.
Anticipatory Relief Response
Termination of the UCS produces an unconditioned relief response RR with stimulus consequences SR.
Conditioned cues elicit an anticipatory relief response rR with stimulus consequences sR.
Example: dog bite elicits pain response, sight of dog elicits anticipatory pain, house elicits relief
A Discriminative Cue is Needed
During trace conditioning no cue is present when UCS occurs and no avoidance learning occurs.
A second cue presented during avoidance behavior slowly acquires rR-sR conditioning.
Similarly, in a Sidman task, cues predict relief -- associated with avoidance behavior, not the UCS.
A Second Cue Helps Trace Learning
Group TS saw a
second cue
associated with
termination of
shock
Thorndike’s Negative Law of Effect
Thorndike suggested that punishment
weakens an S-R bond.
Skinner’s finding that suppression of behavior is
temporary contradicts this.
The effect of punishment must be something
different than weakening of the S-R bond.
Guthrie’s View of Punishment
When punishment occurs, the response to it is
conditioned to the environment during the
event.
Freezing, jumping, flinching.
The effect on behavior depends on the UCR
elicited by the shock.
Shock to forepaws inhibits running but a shock to
hindpaws facilitates it.
Monkeys struggle more when shocked.
Guthrie’s Competing Response Theory
Guthrie suggested that punishment works
only if the response elicited by the
punishment is incompatible with the punished
behavior.
Gerbils punished for standing upright do it more,
not less.
Problems with Guthrie’s Theory
Response competition alone is insufficient to
make punishment effective.
When punishment is contingent instead of just
co-occurring, it is more effective.
Contingent means the punishment happens only
when the behavior occurs, not independent of it,
randomly
Este’s Motivational View
When a behavior is rewarded, the
motivational system becomes associated with
the behavior.
The response occurs the next time the
motivational system is activated.
Punishment works by changing the motives.
Stimuli associated with punishment inhibit the
motivational state.
Support for Estes
Thirsty rats were trained to lever press for
water and “dry lick” for air on alternate days.
Punishment of both behaviors had a greater effect
on dry licking (a thirst-related behavior) than
lever pressing.
If the behavior rather than the motive were being
suppressed no such difference should occur.
Results differed with hungry rats.