Chapter 5: Instrumental Conditioning: Nonreward, Punishment, & Avoidance

PSY 445: Learning & Memory

Nonreward◦Extinction◦Omission

Punishment Avoidance learning

◦Negative Reinforcement

The non-reinforcement of a previously reinforced responseThe result is the decrease in the strength of the initially reinforced response

Extinction

Strength of R

Hi

Lo

Acquisition R No S*

A selected response prevents a positive reinforcer from occurring

Consists of the removal of a stimulus (usually one that is considered to be pleasant) following a response that leads to a decrease in the future strength of that response

R ------------------------------- S (tease sister) (no computer games)

Unpleasant Emotional Effects Frustration; AggressionExtinction BurstTemporary increase in the nonreinforced behaviorAlessandri, Sullivan, & Lewis (1990)

Spontaneous RecoveryAfter a delay the response recovers

Infant mobile

See next slide

Unpleasant Emotional Effects Frustration; AggressionExtinction BurstTemporary increase in the nonreinforced behaviorAlessandri, Sullivan, & Lewis (1990)

Spontaneous RecoveryAfter a delay the response recovers

Infant mobile

See next slide

Spontaneous Recovery

Strength of R

Hi

Lo

Acquistion Extinction S+ occurs

Time passes

When old behavior doesn’t work, new behaviors may replace itNeuringer, Kornell, & Olafs (2001)ProcedureRats trained to press three specific levers (of five present) to get food pellet rewardExtinction phase food pellets withheldResultsRats tried new sequence of lever pushing to see if they could get reward InterpretationReal-world adaptation: If food is not obtained with one behavior, survival depends on us to be flexible enough to try novel and creative behaviorThis runs counter to Thorndike’s old interpretation where cats continued with stereotyped behavior

Extinction is slower if a partial reinforcement schedule (PRS), rather than a continuous reinforcement schedule (CRS) was in effect before extinction◦ Acquisition: CRS rats ran faster to look for reward◦ Extinction: PRS rats ran faster to look for reward

Explanations for resistance Discrimination HypothesisFor CRS rats, the onset of extinction is readily discriminated from the previous continuous schedule of rewards; something has changedFor PRS rats, the first extinction trial is not considered permanent Frustration HypothesisFor PRS rats, the frustrating aftereffects of nonreward become associated with the subsequent occurrence of rewardFrustration experiences on one trial becomes SD for reward

Sequential HypothesisAt the start of a new trial, PRS rats remember the outcome of the previous trial and associates it with the outcome of the current trialLess trials needed here than with frustration idea (memory quicker to form than is frustration)Trials can be spaced further apart as memory persists longer than frustration

An aversive stimulus is presented immediately after a behavior in order to decrease the future probability of that behavior (or strength of this response)

For example: when your dog chases a car

R --------------------------------------- S (run s into traffic)

(scolded)

Punishing stimuliShock used in most animal studiesResponse-contingent punishmentBe sure specific response is being punishedIntensityStrong better than weak; don’t use gradual increasesDelay of punishmentMore effective when applied immediatelySolomon, Turner, & Lessac (1968)Dogs punished immediately, after 5 seconds,; after 15 secondsImmediate worked best; fifteen second delay was least effective

Schedule of punishmentContinuous schedule is generally better than partial schedule; however, spontaneous recovery is more likely

Incompatible responsesBe careful to punish in a way that will lead to the desired outcomeUnfortunately, at times the punishing stimulus can elicit behaviors that are incompatible with the desired outcome

Concurrent reinforcementThe effects of punishment can be neutralized if positive reinforcement of the inappropriate behavior occurs along with punishment

Providing a verbal rationaleLess severe punishment is enhanced by including an explanation Cheyne, Goyeche, & Walters (1969)ProcedureDifferent types of punishment with or without verbal rationale were given to children attempting to reach for a toyResultsNo rationale most severe punishment is most effectWith rationale lesser punishment worked significantly better

See next slide

InterpretationStrong emotional responses that accompany punishment may interfere with the ability to attend to and/or learn from the situation (narrowing of attention)

Cheyne, Goyeche, & Walters (1969)

Results

Individual differencesSome people or animals may be less reactive to punishment than others

Conditioned fear avoidance◦ The place where punishment takes place or the person

administering the punishment may become CS leading to CR Aggression

◦ Punishment-aggression hypothesis Miller (1948): rats exhibited shock-aggression behavior Aggression is reinforced Generalized aggression

Paradoxical rewarding effects◦ The pairing of a punishing stimulus with a positive reinforcer

can convert the punisher into a secondary reinforcer◦ The punishing stimulus can inadvertently enforce behavior

rather than suppress in it Masserman (1943)

Masserman (1943)ProcedureTrained cats to lever press for unpleasant blasts of airPhase 1: Lever press for reward (food)Phase 2: Blasts only occasionally and mildPhase 3: Gradually increased air blasts and decreased food rewardsResults Cats would bar press for punishmentInterpretationKnowing the history of an organism can explain irrational behavior

Frustration produced by nonreward can elicit aggression (similar to punishment)

Does punishment work in everyday life? Are punished children better behaved?

Methodological issues Different types of punishment Who decides if the child is misbehaving

Conflicting research◦ Some say more problems; others say neither benefits

nor deficits Correlational issues

◦ Not necessarily cause and effect relationship

Resistance to extinction is the extent to which responding continues after an extinction procedure has begun

Persistent responding = high resistance to extinction

Responding stops quickly = low resistance to extinction

Schedule of reinforcement◦Partial during acquisition

Magnitude of the reinforcer◦Smaller during acquisition

Delay of the reinforcer◦Delayed during acquisition

Generalized PersistenceIn certain cases, responding is resilientCan carry over from one situation to anotherEisenberger et al. (1979)Procedure Depressed patients asked to perform choresPhase 1: Gratitude was either on continuous or partial reinforcement schedulePhase 2: New person requests favor and shows gratitude on partial reinforcement scheduleResultsPatients spent more time on a chore (punching computer cards) if initially on partial reinforcement schedule InterpretationPersistence generalized beyond initial task to another task and from another person

An instrumental response escapes or prevents an aversive stimulusOften referred to as negative reinforcement because the instrumental response increases in frequency

Two types of Negative Reinforcement◦Escape Conditioning When a behavior has terminated an aversive

stimulus◦Avoidance Conditioning When a behavior has prevented an aversive

stimulus

Typical procedureLearning task begins with warning signal which is followed by shockOn succeeding trials, selected instrumental response prevents aversive stimulus and terminates both the warning signal and trial

Shuttle Box

Active Avoidance◦Active performance of a certain response

prevent the aversive stimulus Passive Avoidance

◦Withholding a response prevents the aversive outcome

Conditioning Theory◦Watson-Mowrer Two-Process Theory

Cognitive Theory Functional Approach

Watson-Mowrer Two-Process Theory (Mowrer , 1947) Avoidance learning involved two processes - (1) classical conditioning and (2) instrumental conditioning

Process 1:Dangerous, painful, aversive stimuli (US) causes an innate fear response (UR) Other stimuli present at the time get associated with fear through classical conditioning.When these other stimuli (CSs) are encountered again, they evoke a fear response (CR)Process 2:The presence of fear and all of its visceral effects is aversiveAny response that removes these fear-evoking stimuli will be negatively reinforced

Solomon & Wynne (1953)Illustrates Two-Process Theory

Shuttle box consists of:◦ 2 chambers separated by a barrier several inches

high◦ Could freely move between chambers◦ Separate lights◦ Metal floor that could be electrified to deliver shock

ProcedureExperimental Session – 10 trials eachDog placed in one compartment of a shuttle boxAfter a time, light in the compartment that dog was in went off, while other light stayed on10 seconds later, the dog received a shock through the floor until jumped over barrierMeasured response latency ResultsEscape first: First few responses were usually longer than 10 seconds. So dog was getting shocked and then escaping.Avoidance response: By roughly the 5th trial, dogs response latency was less than 10 seconds. So dog never received shock in these cases.InterpretationBoth processes are involved

Solomon & Wynne (1953)

LimitationsApparently endless persistence in avoidance behavior

◦Fear to warnings never seem to go away ◦Examples: people with phobias

Seems to fit better with classical conditioning ideas

Eliminating avoidance behavior requires modifying two expectations:Response - expect to avoid aversive outcomeNo response – expect to get shockedSubject must learn that the warning stimulus no longer signals danger

Species-Specific Defense Responses (SSDRs)Studying the natural behaviors of the organism’s reactions to threats in the wildInnate responses that are primed in a fearful or threatening situationThese behaviors are witnessed in lab setting (for example, mice in fearful situations will “freeze”)

When a behavior has two opposing outcomes reward and punishment what is the result?Relative intensityProximity to the consequence (physically or temporally)

Approach CopingConfronting the stressor; problem-focused copingAvoidance CopingShutting down and trying to deny or suppress the unpleasant thoughts and feelings; repressive copingDistraction can be an adaptive form of avoidance coping

Averill & Rosenn (1972)Shock-avoidance experimentProcedureAudiotape with two tracksTrack 1: MusicTrack 2: Tone of the impending shockResultsSome students listened to music exclusivelyInterpretationSometimes the anticipation of unpleasantness is worse than the actual aversive stimulus

Which style is best? It depends…Is stressor controllable or not?Habituation and counterconditioning process processes are involvedAre thoughts intrusive? Ironic processes may be involved (Wegner, 1987)

Learning that there is an explicit lack of contingency between responses and an aversive outcome

Seligman & Maier (1967)ProcedurePhase 1: Dogs receive either: escapable shock (press button with nose), inescapable shock, no shockPhase 2: All placed in shuttle box (two components)Light warned of shock on one sideEscape shock by moving to other sideResultsDogs that had been exposed to escapable shock and no shock quickly learned to escapeInescapable shock dogs “quietly whined"InterpretationInescapable shock dogs developed a perceived loss of control

Hiroto (1974)ProcedureCollege students exposed to loud tone pulses that were “unpleasant but not harmful”Phase 1: Escapable group and inescapable groupPhase 2: Both groups now could escape toneResultsInescapable group was slow to respond in Phase 2InterpretationInescapable college students developed a perceived loss of control

DepressionSymptoms of both helpless and depressed mimic each other – reductions in activity; loss of motivation; disruptions in eating, sleeping, and sexual behaviorAssertiveness training can be beneficial to these individuals

Causal Attribution TheoryThis is the idea that helplessness is ultimately determined by the belief that our behavior is ineffectual in determining what happens to usInsofar as their causal attributions are concerned, helpless individuals have an external locus of control

Classroom EducationLocus of control is involved here as wellPhysical HealthCorrelational findings associate negative attitudes with physical health problems

Amygdala is involved in classical conditioning of fearStrong evidence from studies on both animals and humans

Pet Containment SystemsInvisible Fencing technique applies Watson-Mowrer theory of conditioningPotential to provoke aggression is a concernTreatment of Obsessive-Compulsive DisorderStimulus extinction and response prevention have been used to treat these individualsUsually combined with other methods (cognitive therapies; pharmacological treatments)

Some slides prepared with the help of the following websites:www.radford.edu/~pjackson/ExtinctIC.pptwww.public.coe.edu/~mbaker/baker/learning/lectureshttp://pjackson.asp.radford.edu/punishment3.pdfhttp://web.mnstate.edu/malonech/Psy342/Terry%20Notes/Instrumental%20Cond.%20T4T5.htm