The Cognitive Dog:Savant or Slacker

The Cognitive Dog:Savant or Slacker

Class 13: Observational Learning

Remember timestamp!

Agenda

• Final paper

• Carolyn on learning theory

• Bruce on observational learning theory

The brain trust Kate

The Cognitive Dog

Class 13

ABC’S OF LEARNING THEORY

How Dogs Learn – The Basics

How Dogs Learn

CLASSICAL CONDITIONINGAlso known as Pavlovian, Associative and Respondent

OPERANT CONDITIONING What we know as “training”

How Dogs Learn

CLASSICAL CONDITIONING

• Pavlov’s Dogs

• Relates to reflexes not to voluntary behaviors

• Happening all the time

• Clicker and food is a classical association

• Teaching a cue is a classical association

How Dogs LearnOPERANT CONDITIONING

Definition (from How Dog’s Learn Burch & Bailey, 1999): “The part of science of behavior that explains the functional relationshipbetween environmental events and behavior. It is a key component in explaining how all organisms (including dogs) learn.” In simpleterms what happens in the environmentwill affect what a dog will do in the future. If a dog receives a cookie for sitting he will be more likely to sit again. If a bee stings a dog when he sits he is less likely to sit.


REINFORCEMENT:

Strengthens behaviors

.

PUNISHMENT:

Weakens behaviors


REINFORCEMENT:• Strengthens behaviors• Is anything that will increase the likelihood of

a behavior to happen in the future• To be effective it should closely follow the

behavior so an association is made


PUNISHMENT:• Weakens behaviors• Punishment decreases the likelihood of a

behavior to be repeated in the future• To be effective it should closely follow the

behavior• Has potential fallout; fear, anxiety

How Dogs Learn

REINFORCEMENT in dog trainingStrengthens behaviors

• Positive Reinforcement Add good As perceived by the dog

• Negative Reinforcement Remove badAs perceived by the dog

How Dogs Learn

REINFORCEMENT in dog training

Positive Reinforcement

Add good - As perceived by the dog

Examples: Food, toys, exercise,

play & petting (not all dogs)

How Dogs Learn


Negative Reinforcement

Remove bad, As perceivedby the dog

Examples: Stop pulling on a leash, stop electronic collar; spray or shock, stop

ear pinch

How Dogs Learn


Positive Reinforcement Primary Reinforcer Secondary Reinforcer

Negative Reinforcement Primary ReinforcerSecondary Reinforcer

How Dogs LearnREINFORCEMENT in dog training

Positive & Negative Reinforcement Primary An Unconditioned Reinforcer. A primary reinforcer will increase the likelihood that a behavior will be repeated in the future when presented immediately following a behavior. Related to basic needs, food, drink, some touch.

Secondary A Conditioned Reinforcer. It is something that by itself does not mean anything to the dog. But when paired with a primary reinforcer the secondary will then strengthen behaviors. For dogs these can include, praise, clicker, petting, verbal reprimand or sound. This pairing is classical conditioning.

REINFORCEMENT INCREASES BEHAVIOR

How Dogs Learn

Punishment in dog training: Punishment decreases behavior

• Positive Punishmentadd something bad

• Negative Punishmenttake away something good

The good or bad must be the dogs perception

How Dogs Learn

Punishment in dog training:

Positive PunishmentAdd something thing bad

Examples: electric shock, leash correction, spray bottle or collar, throwing objects, hitting

How Dogs Learn


Negative Punishment

Remove something good

Examples: Time out in crate = removes you or the fun, Take dog away from fun event

such as agility run for broken stay

How Dogs Learn

Punishment in dog training: Positive Punishment

Primary Punisher

Secondary Punisher

Negative Punishment

Primary Punisher

Secondary Punisher

How Dogs Learn


Positive & Negative Punishment

Primary An Unconditioned Punisher. The dog does not need prior experience for it to be perceived as a punisher. Extreme heat or cold, shock, pinch, citronella spray, hitting are all examples of primary

Secondary A Conditioned Punisher. It is something that by itself does not mean anything to the dog. But when paired with a primary punisher the secondary will then weaken behaviors. Verbal or sound. This pairing is classical conditioning.

PUNISHMENT DECREASES BEHAVIORBut there is possible increased anxiety and fallout


Reinforcement Punishment

Positive Primary: Food Shock Secondary: Click Warning Sound

Negative Primary: Remove Shock Remove Food

Secondary: Remove Verbal Remove Verbal

How Dogs Learn

finally…The ABC's of learning

A= Antecedent or A Signal (cue)

B= Behavior (action)

C= Consequence (payoff)

Good Consequence = Reinforcement Bad Consequence = Punishment

How Dogs Learn

The ABC's of learning

A= Antecedent or A SignalCues

Verbal: words, soundsVisual: hand, body, motionEnvironmental: agility obstacles

Think about the dog you have when making choices for cues; visual or sound stimulation stronger?

How Dogs Learn


B= BehaviorAction or duration of non actionVoluntary action: sit, down, lift pawNon action: stay in position

Behaviors that are offered by the animal appear to be learned more

quickly vs. guiding or continuous luring

How Dogs Learn


C= Consequence

Payoff

Good Consequence = Reinforcement, R+Increases likelihood of behavior

Bad Consequence = Punishment

Decreases likelihood of behavior

How Dogs LearnOnce learned, Good Consequences =

Behaviors Happen:

Human Example: A = Antecedent (cue): Alarm goes off, ugh…

B = Behavior: Get up and go to workC = Consequence: Paycheck, yahoo!

Canine Example:A = Antecedent (cue): Verbal “Sit”

B = Behavior: Dog puts butt on the floorC = Consequence: Dog gets to go out to play!

How Dogs Learn

“Natural” Progression of A-B-C

Begging:

A - Owner eating at table

B - Dog Begs

C - Owner gives a piece of food

Think the begging will be repeated?!

How Dogs Learn

Training Steps: A new alphabet

B-C-A

Maximizes learning & the quality of the final behavior performance

B = Behavior: Get itC = Consequence: Give itA = Antecedent or A Signal: Add after behavior is strong, insert before behavior

How Dogs Learn

Training Steps

B = Behavior: Get the dog to perform the desired behavior

Capture: R+ dog doing behavior

Shape: R+ parts of the behavior working closer to the end behavior

Lure: Use of food or toy as a guide to where you want the dog then R+

How Dogs Learn

Training StepsC = Consequence:

R+

When the dog does the behavior- Capturing

When the dog does part of the behavior- Shaping

As the dog follows the food into position or partial position - Luring

How Dogs Learn

Training Steps

A = Antecedent or Cue: Add after the dog becomes good at the behavior, cue will pair with the end behavior performance (usually better than beginning)

Cue is presented just before the dog does the behavior = Classical Conditioning

R+ as usual

How Dogs Learn

EXAMPLE OF CAPTURING – SHAPING – LURING

Down – Capture: Sit in a small quiet room R+ as dog lies down on his own

Shape: R+ as dog lowers head, then bends elbows, then lowers back end etc.

Lure: Guide dog into position with food or toy

desired by the dog, R+ in position

How Dogs Learn

BUT KEEP IN MIND…

How Dogs Learn

Emotions & Motivation impact Learning

Take all we have studied so far into consideration…EmotionsHardwiring - MotivationPast ExperiencesLack of Experiences

Keep in mind all of this and the immediate environment have a big influence on a learners ability to learn

How Dogs Learn

THE LEARNERS …that have taught us….

Learning by observation

Local Enhancement

Emery, N. and N. Clayton (2005). Animal Cognition. The Behavior of Animals: mechanisms, function and evolution. J. Bolhuis and L. Giraldeau. London, Blackwell Publishing.

Attraction to others indirectly leads to learning about presence of an object

“Just chillin...”

“I wonder what Fred is up to... Hmm...

not much...Whoa, look at that

cookie jar”Dem. Obs.

Stimulus Enhancement


Learns about presence of object by observing the interaction of another

animal with the object

“Hmmm, what is he playing with..”

“Bug off, this is mine...” Dem. Obs.

Observational Conditioning


Emotional response to an object based on observing another animal’s response

“Hmmm, snakes must be bad...

Eeek...”

“Eeek, a snake...”Dem. Obs.

Things to note...

• The other animal is just doing their thing, their role as “demonstrator/teacher” is purely a side-effect.

• Think of wolf pups following mom & dad

• Their presence/interaction/response provides, in effect, a focus of attention that simplifies the learning problem for the observer.

• Associative learning is probably sufficient given this scaffolding

Goal Emulation...


a) Observe another animal achieve a goal by performing a specific action

b) Achieve same goal but use a different action

The difference in this case is that the person and the monkey orient the rake differently

Imitation...


a) Observe another animal achieve a goal by performing a specific & novel sequence of actions

This type of observational learning seems to occur rarely in other species than humans

b) Achieve same goal by performing the same specific & novel sequence of actions

Common forms of observational learning

Local Enhancement

Stimulus Enhancement

Observational Conditioning


Uncommon forms of observational learning

Goal Emulation

Imitation


Why?

• Why are goal emulation and imitation apparently so rare in species other than humans and maybe other apes...

• May require a type of cognitive machinery (little or big) not found in other species...

• The cost/benefit may low (i.e., the common forms work well enough given built-in behavioral structure & context)

• Just haven’t looked hard enough...

What do our slacker friends do?

• Common forms of observational learning...

• Local enhancement

• Stimulus enhancement

• Observational conditioning

• Less common forms

• Goal emulation (???)

Adler & Adler

Big idea...

• Test for evidence of observational learning in Miniature Dachshund pups of varying ages...

• Had some pups watch another pup retrieve ‘inaccessible’ food by pulling on a string that was attached to the food

• Compared time it took naive pups to retrieve food by pulling on string with the time it took pups who had experience watching other pups do it.

More details...

• 4 litters of doxie pups, 1 from each litter chosen as demonstrator for rest of littermates

• 3 days of acclimatization (???)

• 5 trials per day in which demonstrator learned how to retrieve food, and observers watched

• After 15 trials, observers were given chance to perform task

Results from Adler

Results from Adler...

• The observers generally retrieved the food substantially faster on the first three trials than their demonstrator

• Particularly strong effect on 60 and 38 day-old litters

• Evidence that they benefited from watching demonstrator

Stimulus enhancement?

• “In some cases the observer ran to pull the ribbon as soon as it was made available, with frantic activity continuing although the food was already within reach. Persistence in working and skill in manipulating the ribbon had to be learned by practice.”

Slabbert & Rasa

The big idea...

• “Pups with extended maternal care which were allowed to observe their trained mothers locating and retrieving a sachet of narcotics between the ages of 6 and 12 weeks performed the same task significantly better than non-exposed pups when tested at the age of 6 months, without further reinforcement during the interim period.”

Setup cont...

• Between 6 and 8 weeks, Group 4 pups got to observe mom retrieve drugs a total of 6 times.

• Between 9 and 12 weeks, Group 4 pups got to observe mom retrieve drugs and seem to have been able to sniff drug sachet while in mom’s mouth.

• No other group got to observe Mom performing task or had any exposure to drugs...

untrained mom

trained mom

removed at 6 weeks

group 1 group 3

stay with mom until 12 weeks

group 2 group 4

Tested performance at 6 months

• At 6 months dogs tested for performance, ranked on a scale 1 to 10 with regards to...

• interest in task, manner in which approached sachet, & way it searched and found sachet

• if found, how the dog picked it up & whether the dog carried it back to handler

• Subjective, but testers did not know background of pups...

Results...

• Pups who observed mom retrieving sachet & had opportunity to sniff mom’s mouth did better...

• Seems clear that observational learning occurred, but we don’t know the relative importance of...

• observing mom perform task, and/or

• association of smelling sachet in mom’s mouth, and/or

• association of praise with the above

•

Slabbert, J. M. and O. A. E. Rosa (1997). "Observational learning of an acquired maternal behavior pattern by working dog pups: an alternative training method?" Applied Animal Behaviour Science 53: 309-316.

More thoughts...

• Simple but Reliable Rule:

• Stuff that you smell in mom’s mouth must be good, and thus, worth learning, remembering & finding...

• I sure wish I knew

• how important observing the act of retrieval was...

• does it matter if it is mom?

Social learning: selective imitation?

Three simple types of social learning

Just because its simple, doesn’t make it any less useful

Local enhancement: “I think I’ll hang out with Harry. Hmmm, what’s that”

Stimulus enhancement: “Hey what’s Harry fooling with. That looks tasty”

Observational Conditioning: “Yikes, what is Harry reacting to, I guess I should be scared too”

These previous types of social learning can be explained via associative learning, but...

• Josep Call postulates 2 alternative mechanisms...

• The ‘cue-based’ approach. The animal learns to respond to a given stimulus in a given way, or learns to predict a given future stimulus based on observing another given stimulus.

• All about correlation, no mental model of causation, and hence little or no ability to generalize.

• The ‘knowledge-based’ approach. Here the animal extracts functional features associated with the stimulus, and builds a functional model of greater or lessor complexity that it then uses to guide its choice of behavior

• To the extent that the functional model accurately captures causation, it provides a mechanism to generalize. Call, J. (2001). "Chimpanzee social cognition." Trends in Cognitive Science 5(9): 388-

393.

Range, F., Z. Viranyi, et al. (2007). "Selective Imitation in Domestic Dogs." Current Biology 17: 1-5.

Selective imitation

• Dogs trained to pull a ring for food via mouth and paw

• Control dogs given opportunity to solve this problem on their own. 85% used their mouth to pull on the rod.

• Experimental dogs watched 8 trials of a demonstrator using their paw to push down on the rod.

• One group, demonstrator has ball in mouth

• Other group, demonstrator doesn’t have a ball in mouth

Demonstrator has ball in mouth...

Hmm... he’d being using his mouth if he could...


Trial after seeing ball in mouth

demonstrator“wicked motivated isn’t the first thing that comes to mind...”

I’m lucky, I can use my mouth...


Demonstrator’s mouth is empty...

Hmm... there must be a reason why he isn’t using his mouth...


Trial after seeing demonstrator without ball

in mouth

It is concerning that there is so much prompting by the handler

I don’t get it, but if he didn’t use mouth there must be a reason...


Clear difference in performance between the

two groupsBut why???

Possible explanations...

• The dogs really are making the kinds of inferences that I have described on the previous pages...

• If so, this is both quite remarkable and quite unexpected

• Is there some confounding factor that isn’t apparent to us that makes it appear as if the dog is making an inference when in fact they are responding to something else in the experimental set-up.

• In either case, it is a fascinating question to ponder...

Christiansen et al

The big idea...

• “...a companion dog showing intentions of predatory behaviour stimulates predatory chase in another dog, while a non-chasing companion has limited influence on this...”

• That is, predatory behavior in one dog may trigger predatory behavior in another,

• but absence of predatory behavior did not have a calming influence

The set-up

• 20 Norwegian Elkhounds (11 males, 9 females), split into 2 age groups: less than 2.5 years old, and older than 2.5 years old.

• 2 demonstrators: a chaser (a BC) and a non-chaser (a Hamilton Stoever)

• a priori tendency of test dogs to chase not tested but general behavior described via questionnaire

Just so you know...

Test...

• Test dog and demonstrator allowed time to greet.

• Test dog put in pasture with 5 sheep and demonstrator

• After initial reaction of test dog recorded, if chasing demonstrator, then cued to chase sheep

• Behavior of test dog recorded over 5 minutes

• First non-chase trial, 3 days later chase trial

Results: Norwegian Elkhounds seem awfully predisposed to chase sheep

60% of dogs attacked sheep with non-

chasing dog, 100% on 2nd trial with

chasing dog

Attack severity greater, and attack latency shorter on

2nd trial with chasing dog

Christiansen, F. O., M. Bakken, et al. (2001). "Social facilitation of predatory, sheep-chasing behaviour in Norwegian Elkhounds, grey." Applied Animal Behaviour Science 72(2): 105-114.

Results...

Christiansen, F. O., M. Bakken, et al. (2001). "Social facilitation of predatory, sheep-chasing behaviour in Norwegian Elkhounds, grey." Applied Animal Behaviour Science 72(2): 105-114.

Discussion

• “a companion dog that shows intentions of predatory behavior stimulates predatory chase in another dog, while a non-chasing companion has a limited influence on the predatory behavior of the test dog”

• “regarded as cases of social facilitation, originally defined as the performance of a behavior pattern as a consequence of the performance of the same behavior by other individuals”

• “the border collie did not lead the attack, but somehow influenced the attack motivation in test dogs. The typical constant interest towards sheep shown by border collies, i.e. showing eye display... and occasionally showing short stalk sequences may have stimulated or accelerated predatory chase in test dogs. However, the Elkhounds never followed the Border collie’s withdrawal or submissive behavior”

Social Facilitation

• This is a kind of SBRE rule...

• “perform predatory behaviors toward animals that social partners are showing predatory behavior towards”

• rely on the assumption that the others know what they are doing...

• don’t need to totally discover on own which animals are appropriate targets for predatory behavior

Social Facilitation...

My sense is that there are a ton of examples of this in real world

dog behavior...

Pongracz et al

Big idea...

• Dogs are conservative, they go with what worked in the past.

• what happens when what worked before doesn’t work any more?

• Observed behavior of human demonstrator can influence dog’s choice of behavior...

• what are they observing?

individual experience and social learning

Big idea...

• Previous success biases a dog’s choice of action even when an ‘obviously better’ choice of action is made available...

• dogs are conservative

• Dogs could use actions of a human demonstrator to guide their choice of action

Setup of experiment 1

Pongracz, P., A. Miklosi, et al. (2003). "Interaction between individual experience and social learning in dogs." Animal Behaviour 65(3): 595-603.

Door is never an option

Dogs first experience is

using door, which is subsequently

unavailable

Results from experiment 1

Pongracz, P., A. Miklosi, et al. (2003). "Interaction between individual experience and social learning in dogs." Animal Behaviour 65(3): 595.

Observing demonstrator

helps dog solve task

Previous experience with open

door seems to interfere

with choosing alternative



Dogs had a tough time

getting over the fact that the door was closed in the absence of a demonstrator

Experiment two...

• Dogs had varying previous experience with detouring

• 3 detour demonstration trial (dog gets to do detour)

• 1 detour demonstration (dog gets to do detour)

• Test trials with door open

• Trial 1 door opened while eyes covered

• Trials 2 and 3 dog sees food placed through open door



Chose to detour

Chose door

Dogs with more experience of

detouring persisted in detouring even when door was available



3 demonstrations

Dogs with a single demonstration were more focused on the door than those who

had three demonstrations

1 demonstration

Discussion

• Dogs tended to use first strategy that worked and would persist in using strategy even when a better one came along

• The more experience, the harder it was to change

• Socially acquired experiences given up slowly*

• Dogs did take advantage of watching a demonstrator with respect to taking a detour.

Discussion

• “Both of our experiments revealed the complex interaction between asocial (individual) and social learning that must be taken into account to understand how learning abilities in general contribute to increased fitness in animals. In experiment 1 social learning was advantageous in a situation where experience constrained the dogs’ behaviour. On the other hand in experiment 2, dogs facing a novel situation and exposed to socially provided information regarding access to the target were reluctant to change their behaviour, showing a preference for the more conservative (and socially learned) behaviour.”

Thoughts...

• Nice demonstration of observational learning and of conservative, to a fault, decision-making on the part of the dogs...

• I am less convinced that the way the learning occurred (social) meant that it persisted longer. The observation is conflated with the actual experience.

• They make a lot of this in the paper...

And yes...

• There may be an SBRE rule or two lurking here as well...

• go with what you know

• Is demonstration example of stimulus enhancement (end of fence) or local enhancement (places to wander)?

Follow-on study: Are dogs attending to paths or attending to corners (stimulus enhancement)???

Pongracz, P., A. Miklosi, et al. (2003). "Preference for Copying Unambiguous Demonstrations in Dogs (Canis familiaris)." Journal of Comparative Psychology 117(3): 337-343.

AmbiguousUnambiguous

Are dogs attending to paths or attending to corners (stimulus

enhancement)???

McKinley and Young

Masters of understatement

• “There has been one notable failure using this method, this involved a lar gibbon who attacked the model-rival during training sessions; this probably occurred due to the aggressive and territorial nature of this species”

• Lesson: know your animal, and don’t trust your advisor...

Big idea of model-rival...

• Animal observes trainer and a “model-rival”

• Trainer: “Can you see the SOCKS?” & hands toy to M-R

• Model-Rival: “Yes I can, thank-you for the SOCKS” & hands toy to trainer

• repeat...

• M/R is modeling desired behavior and rival for attention and/or possession of object.

Big idea of model-rival

• “fetch Mr. Squirrel” & receive a treat

• Q: in dog’s mind does the label “Mr. Squirrel” refer to the object or to the act of retrieving it and getting a cookie?

• A: who knows, but Pepperberg argues in traditional training, it is the latter, because the reward is extrinsic. This makes the label context-specific (i.e., tied to the specific action)

Big idea in model-rival

• In model-rival training, the animal’s reward for successfully performing the task is the object itself. The reward is intrinsic...

• Pepperberg argues that this makes it more likely that the label becomes associated with the object, and thus can be used in other contexts...

• “how many Mr. Squirrels are there?”

Big idea of model-rival...

• To keep dog’s interest, trainer & model-rival

• “speak in a highly animated way” [lots of motion??]

• “look at target object at all times”, but

• “voice direction and body postures directed toward dog”

• After 2 minutes, dog gets to try...

The test...

• After 2 minute training session...

• Trial

• Target object part of a group of 3 objects

• Dog told to “Go get the SOCKS”

• Trial ends when dog retrieves correct item, time noted.

• Note: order of retrieval doesn’t directly matter!!!

Results...

McKinley, S. and R. J. Young (2003). "The efficacy of the model-rival method when compared with operant conditioning for training domestic dogs to perform a retrieval-selection task." Applied Animal Behaviour Science 81(4): 357-365.

“We found no significant effects of training method on training times... or on trial times...”

Thoughts...

• Very cool example of using stimulus enhancement & maybe local enhancement to possibly help dog make association between label and object.

• interesting that dogs may be able to make this association

• Raises the whole question of intrinsic vs. extrinsic rewards

• Great to be exploring alternative approaches to training

Thoughts from Mr. Grinch...• We don’t know whether differences in training technique had any

impact on accuracy since this isn’t reported...

• why is article so silent on this point?

• What evidence is there for the following statement?

• “The m-r method resulted in the dog knowing what the object’s name was, whereas the operant method only resulted in the dogs knowing that retrieving the object results in a food reward”

Thoughts from Mr. SBRE

• Is there a Shallow But Reliable Enough Rule, or two, lurking here?

• “Stuff being handled by social partners must be good, and thus worth learning, remembering & finding...”

• “Pay attention to what social partners handle”

The slacker’s guide to the universe: simple but reliable rules

Documents

The Cognitive Dog:Savant or Slacker