MIT9 00SCF11 Lec09 Learning

8/16/2019 MIT9 00SCF11 Lec09 Learning

1/54

9.00 Learning9.00 Learning

Professor John Gabrieli


2/54

Recent Research on Effective Study

• Test First, Study Later

• Testing Yourself on Material More

Useful Than Continued Study

(ok to have wrong answers)


3/54

LEARNINGLEARNING

• how behavior changes within the

lifespan of an individual• everything we know that is not

genetically given

• how to predict the future on the basisof past experience

• to imbue the world with meaning• learning about learning through

scientific psychology


4/54

LEARNINGLEARNING

1. Classical Conditioning

2. Operant Conditioning

3. Limits to Conditioning


5/54

Ivan PavlovIvan Pavlov (1849(1849--1936)1936)

• Nobel Prize for reflexes of digestion - food inmouth provokes specific salivation to preparefor digestion - salivation reflexes

"But Professor, there's a revolution going on withshooting in the streets."

"What difference does it make when you've workto do in the laboratory? Next time there's arevolution, get up earlier!“

• cut esophagus so food could not go to stomach• placed food in dog's mouth, stomach secreted

plenty of gastric juice

• sight of food or sight of feeder - psychicsecretions or conditioned reflex


6/54

PAVLOVIAN CONDITIONING

A modification of Pavlov’s method Image by MIT OpenCourseWare.


7/54

• Unconditioned Stimulus

(UCSUCS) – food• Unconditioned Response

(UCRUCR) - salivating (food)• Conditioned Stimulus

(CSCS) – bell

• Conditioned Response(CRCR) – salivating

• new association !!bell (CS) & salivating (CR)

• law of association by contiguity

(Aristotle)


8/54

Source: Stangor, C. Introduction to Psychology . Flatworld Knowledge, 2010. Courtesy of Flatworld Knowledge.

Classically Conditioning a

Salivation Response

http://www.flatworldknowledge.com/pub/introduction-psychology/67025http://www.flatworldknowledge.com/pub/introduction-psychology/67025


9/54

Image by MIT OpenCourseWare.



10/54

Ivan Pavlov & His Dogs

• http://www.youtube.com/watch?v=hhqumf

pxuzI

http://www.youtube.com/watch?v=hhqumfpxuzIhttp://www.youtube.com/watch?v=hhqumfpxuzIhttp://www.youtube.com/watch?v=hhqumfpxuzIhttp://www.youtube.com/watch?v=hhqumfpxuzI


11/54

• how to predict the future on the basis ofpast experience

bell---food---salivation• to imbue the world with meaning

bell---means that food is near (any UCS worked)


12/54

Water Demo


13/54


(UCSUCS) - water in face• Unconditioned Response

(UCRUCR) - flinching to water

UCS-UCR association is built-in reflex

• Conditioned Stimulus

(CSCS) - hearing “ CAN”

• Conditioned Response

(CRCR) - flinching to “ CAN”• new association !!

CAN (CS) & flinching (CR)


14/54

Balloon Demo


15/54


(UCSUCS) - balloon noise• Unconditioned Response

(UCRUCR) - flinching

UCS-UCR association is built-in reflex

• Conditioned Stimulus

(CSCS) - needle touching balloon

• Conditioned Response

(CRCR) - flinching to needle• new association !!

needle (CS) & flinching (CR)


16/54

PROPERTIES OF CLASSICALPROPERTIES OF CLASSICAL

CONDITIONINGCONDITIONING• extinction

• generalization gradient• discrimination training

black (CS+) & gray (CS-) squares

• second-order conditioningbell (CS)

bell (US) & black square (CS)

• is temporal contiguity the basis of classicalconditioning?


17/54

EXTINCTION GENERALIZATION


Acquisition Extinction andAcquisition Extinction and


18/54

Acquisition, Extinction, and Acquisition, Extinction, and

Spontaneous Recovery inSpontaneous Recovery in

Classical ConditioningClassical Conditioning

Source: Stangor, C. Introduction to Psychology . Flatworld Knowledge, 2010. Courtesy of Flatworld Knowledge.

http://www.flatworldknowledge.com/pub/introduction-psychology/67025http://www.flatworldknowledge.com/pub/introduction-psychology/67025


19/54

PROPERTIES OF CLASSICALPROPERTIES OF CLASSICAL

CONDITIONINGCONDITIONING• extinction

• generalization gradient• discrimination training

black (CS+) & gray (CS-) squares

• second-order conditioningbell (CS)

bell (US) & black square (CS)

• is temporal contiguity the basis of classicalconditioning?

S d O d C diti iSecond Order Conditioning


20/54

Second Order ConditioningSecond Order Conditioning

© source unknown. All rights reserved. This

content is excluded from our Creative

Commons license. For more information, see

http://ocw.mit.edu/fairuse.

http://ocw.mit.edu/fairusehttp://ocw.mit.edu/fairuse


21/54

CONTINGENCY

© Harper Collins College Publishers. All rights reserved. This content is excluded from our

Creative Commons license. For more information, see http://ocw.mit.edu/fairuse.



22/54

• cognitive conditioning: Blocking

contiguity vs. contingency

CS1 CS1 + CS2 CS2tone (food) tone + light (food) light (food)

CS1+ CS2 CS2tone + light (food) light

(food) more conditioning to CS2

less conditioning to CS2


23/54

Demo


24/54

• how to predict the future on the basis of pastexperience

bell---food---salivation

• to imbue the world with meaning

bell---means that food is near

(any UCS worked)

Why do we work hard? (where is the UCS?)


25/54

LEARNINGLEARNING





26/54


27/54

© source unknown. All rights reserved. This content is excluded from our Creative

Commons license. For more information, see http://ocw.mit.edu/fairuse.



28/54

consequence of a response determines

whether it is strengthened or weakened© Worth Publishers, Inc. All rights reserved. This content is excluded from ourCreative Commons license. For more information, see http://ocw.mit.edu/fairuse.



29/54

John B. WatsonJohn B. Watson (1878(1878--1958)1958)

• University of Chicago/Johns Hopkins• "mind" unobservable

• behaviorism

• study behavior = observable actions, not the mind• identify environmental conditions

• no fundamental difference between animals and

humans• describe lawful relations between environment-

behavior reflexes

SSTIMULITIMULI--------------------RRESPONSESESPONSES

(environment) (behavior)


30/54

Fear Conditioning

Classical Conditioning of a Phobia: Little Alberthttp://www.youtube.com/watch?v=0FKZAYt77ZM

B F SkiB F Skinner (1904(1904 1990)1990)

http://www.youtube.com/watch?v=0FKZAYt77ZMhttp://www.youtube.com/watch?v=0FKZAYt77ZM


31/54

B. F. Skinner B. F. Skinner (1904(1904--1990)1990)

• consequences of responses

• operant response is an action that operateson environment to produce someconsequence

• Beyond Freedom and Dignity• operant behavior

classical conditioning CS elicits CRinstrumental conditioing CRs are emitted -

CRs = operants

create the CRs

OPERANT CONDITIONING


32/54

OPERANT CONDITIONINGOPERANT CONDITIONING

• novel response?

successive approximations & shaping

high lever

• click & pellet

• location, click & pellet• face lever, location, click & pellet

• stretching body upward, face lever, location, click

& pellet• touch lever with paws, stretching body upward,

face lever, location, click & pellet

• press high lever, touch lever with paws, stretchingbody upward, face lever, location, click & pellet

INSTRUMENTAL/OPERANTINSTRUMENTAL/OPERANT


33/54

INSTRUMENTAL/OPERANTINSTRUMENTAL/OPERANT

CONDITIONINGCONDITIONINGoperate as instruments to produce desired effect

• Law of Effectconsequence of a response determines whether

it is strengthened or weakened

Reward - Strengthened

No Reward - Weakened

Punishment - Very Weakened

http://www.learner.org/resources/series138.html?pop=yes&pid=1529

11:37

REINFORCEMENT

http://www.learner.org/resources/series138.html?pop=yes&pid=1529http://www.learner.org/resources/series138.html?pop=yes&pid=1529


34/54

REINFORCEMENT

• Primary Reinforcers

food, thirst, pain

• Secondary Reinforcers

money, attention, praise, admission,

promotion

• Positive (increase behavior)

• Negative (decrease behavior, escape)

• Punishment

• Partial Reinforcement

PartialPartial reinforcement effectreinforcement effect


35/54

PartialPartial --reinforcement effectreinforcement effect

More

resistant to

extinction

100% group

30% group

Trials

S p e e d ( r u

n n i n g t i m e

/ d i s t a n c e )

5 10 15 200

0.2

0.4

0.6

0.8

1.0

LEARNED HELPLESSNESSLEARNED HELPLESSNESS


36/54

LEARNED HELPLESSNESSLEARNED HELPLESSNESSSeligman & dogs

Phase 1

Yoked in hammock Group A Group B

with shocks: could stop could not stop

by pushingpanel near

nose

equal number & duration of

shocksPhase 2

avoidance learning in shuttle box

CS - tone

jump within 10 secs to avoid shocks

Group A - learns

Group B - does not learn

motivational deficits - slow to initiate known actionsemotional deficits - listless, frightened, distress

co nitive deficits - oor learnin in new situations


37/54

Avoidance of

Shock by Dogs

© source unknown. All rights reserved. This content isexcluded from our Creative Commons license. For more

information, see http://ocw.mit.edu/fairuse.

Learned HelplessnessLearned Helplessness



38/54

No. of trials

M e d i a n l a

t e n c y ( s e c )

Learned HelplessnessLearned Helplessness


39/54

Depression & People?

how we explain life to ourselvesinternal-external

global-specificstable-unstable

LEARNINGLEARNING


40/54

LEARNINGLEARNING





41/54

LIMITS TO CONDITIONING

• preparedness

• latent learning

• contingency

• reward value

• delayed gratification

• when reward harms

• language

Results of Garcia andResults of Garcia and KoellingKoelling’’ss


42/54

© source unknown. All rights

reserved. This content is excluded

from our Creative Commons license.

For more information, see

http://ocw.mit.edu/fairuse.

ExperimentExperiment -- Taste AversionTaste Aversion

Characteristics of the Conditioned StimulusCharacteristics of the Conditioned Stimulus

and the Unconditioned Stimulus Affect theand the Unconditioned Stimulus Affect the



43/54

and the Unconditioned Stimulus Affect theand the Unconditioned Stimulus Affect the

Acquisition of the Conditioned Response Acquisition of the Conditioned Response

Condit ioned stimuli:

Unconditioned stimuli

P e r c e n t a g

e o f n o r m a

l f l u i d i n t a k

e

After exposure to X-

rays, rats avoided

flavored water, not

light and noise

After being shocked,

rats avoid light and

noise, not flavored

water


44/54

Preparedness

Picture - CS

snakes/spiders or flowers/mushroomsShock - US

UR - GSR (sweat)

Better conditioning for snakes/spiders

P d


45/54

Preparedness

Little Albert Study

rat - workedwooden block, piece of cloth

did not work


46/54

LATENT LEARNING

3 groups of rats in goal maze

• food reward every day

• no rewards

• no rewards for 10 days; then reward

Latent LearningLatent Learning


48/54

• 2-month-old infant/crib/color mobile

• moving head, switch in pillow/smile &coo

• second group/no control/no smile, nocoo despite equal number of mobileturns

CONTINGENCY


49/54

Reward Value

• negative contrast

What happens when you switch

to a worse reward?


50/54

Courtesy of Elsevier, Inc., http://www.sciencedirect.com. Used with permission.

DELAYED GRATIFICATION

http://www.sciencedirect.com/http://www.sciencedirect.com/


51/54

• 4-5 year-old children – 653 childrenof faculty and graduate students

• two snacks

one or two marshmellows

• wait 15 minutes to get two – 30%

waited 15 minutes• correlated 10 years later with

behavioral problems, academic and

social success (210 SAT points for30 sec vs. 15 min delay)

WHEN REWARDS HARM


52/54

WHEN REWARDS HARM

rat & running wheel

run for fun

run for food - no longer will run for fun

preschoolers draw for fun

gold stars (conditioned reinforcer)no gold stars, no drawing

Is Language Learning AIs Language Learning A


53/54

Conditioned Skill?Conditioned Skill?• 1 month - switch inside rubber nipple - hooked to tape

recorder - when baby sucks, tape plays - ba ba ba vs.

pa pa pa - may not be in their own language(Kikuyu/Spanish) and which their parents may beunable to distinguish (Czech, Hindi, Inslekampx)

• at 4 days, a French baby prefers French to Russian,

Italian, backwards French• correction/reinforcement?

2-year-old: "Mamma isn't boy, he a girl."

Mother" "That's right."• generative - sentences are produced that are unique

• everybody learns it without training

• overgeneralizing - "My teacher holded the rabbit."

O C


54/54

MIT OpenCourseWare

http://ocw.mit.edu

9.00SC Introduction to PsychologyFall 2011

For information about citing these materials or our Terms of use, visit http://ocw.mit.edu/terms.
http://ocw.mit.edu/http://ocw.mit.edu/termshttp://ocw.mit.edu/termshttp://ocw.mit.edu/

Documents

MIT9 00SCF11 Lec09 Learning