The Cognitive Dog Class 11: Emotion (reward and self-rewarding behaviors...)

The Cognitive Dog

Class 11: Emotion (reward and self-rewarding behaviors...)

Agenda

• Questions?

• I will try to get the papers back this weekend.

• The neural basis of reward and self-rewarding behaviors

• Using what the dog wants to get what you want

Emotion Systems

Panksepp, J. (1998). The Foundations of Human and Animal Emotions. New York, NY, Oxford University Press.

The amydala, dopamine, and reward...

The amygdala is the star of the show

again

What we learned about fear mostly applies here as well...

LeDoux, J. (2002). Synaptic Self: How our Brains Become Who We Are. New York, NY, Penguin Books.

The soup of neurotransmitters and

their pathways

The amygdala is one of the primary cooks of this soup

Panksepp, J. (1998). The Foundations of Human and Animal Emotions. New York, NY, Oxford University Press.

The Amygdala plays a central role in CC

• ‘The BLA is required for a CS to gain access to the current motivational or affective value of the specific US that it predicts’

• The CeA ‘receives or may encode direct S-R Pavlovian associations, thereby influencing specific CRs... as well as modulating arousal and attention through diffuse projections’

Cardinal, R. N., J. A. Parkinson, et al. (2002). "Emotion and Motivation: the role of amygdala, ventral striatum and prefrontal cortex." Neuroscience and Biobehavioral Revews 26: 321-352.

One reason why the clicker may be so

effectiveclickers are ‘low road’ all the way


The amygdala, dopamine & action...

• One consequence of the amygdala being activated is that dopamine cells in the VTA are activated. Some of these cells release dopamine into a part of the brain known as the Nucleus Accumbens.

• The has the effect of amplifying signals coming from the Amygdala and this in turn allows the Amygdala to strongly activate and direct motor activity. LeDoux, J. (2002). Synaptic Self: How our Brains

Become Who We Are. New York, NY, Penguin Books.

Stimulus

Dopamine

• McConnell: ‘is vital to your brain’s ability to focus your attention—its release causes all of your brain cells to cease firing for a microsecond and then prepares them to fire in anticipation of an important event—but it’s also a key player in feelings of reward and satisfaction.’

• LeDoux: ‘Some adhere to the classic hypothesis— that it is the basis for reward. Another view is that dopamine release is important for the initiation and maintenance of anticipatory behaviors in the presence of secondary incentive stimuli. Others argue that dopamine release notifies the forebrain that something novel or unexpected has occurred, but not that reward per se has occurred. Still others propose that dopamine is involved in the switching of attention and selection of action. These are not mutually exclusive views and in fact each appears to correctly characterize certain aspects of what dopamine contributes to motivation.’

McConnell, P. (2006). For the Love of a Dog: Understanding Emotion in You and Your Dog. New York, NY, Ballantine Books.


The Nucleus Accubens plays a central role in Operant conditioning

• It helps to modulate conditioned and unconditioned responses...

• ‘It is a key site mediating the ability of Pavlovian CSs to invigorate and direct behavior.’

• It ‘mediates aspects of preparatory behaviour, temporally distant from the goal of behaviour (as opposed to consummatory behaviour, temporally close to the goal).’

Cardinal, R. N., J. A. Parkinson, et al. (2002). "Emotion and Motivation: the role of amygdala, ventral striatum and prefrontal cortex." Neuroscience and Biobehavioral Revews 26: 321-352.

Emotional habits, once learned don’t seem to require the amygdala...

• LeDoux: ‘once an emotional habit is learned, the brain systems involved in expressing it become simpler’

• ‘Characteristically, amygdala-dependent signs of emotional arousal... occur during the initial phase of avoidance learning but disappear as the avoidance response is learned. The accumbens likewise drops out once the response is learned; while it is needed to do learning, it is not necessary to perform well-learned responses.

• ‘... perhaps the accumbens... trains prefrontal cortical circuits, especially areas of the motor cortex, how to respond’

• Do many behaviors we train our dogs eventually become emotional habits and so do not require constant reinforcement once learned and transferred?

Self-motivating and self-rewarding behaviors...

Why do they choose the squirrel every

time?

After this talk they will still choose the squirrels every time, but at least you may know why...

100% chance of success...

0% chance of success...

The big idea...

• We tend to focus on goals as the explanation for behavior. That is, the behavior has a consequence and that is why it is performed

• For certain classes of behavior, the performance of the behavior is its own reward, and it is this dynamic that underlies great working dogs as well as the behaviors that drive pet owners nuts... This is speculative, but there are some tantalizing strands to the story...

• Brelands

• Leyhausen

• Coppinger

• Panksepp

Our story begins with the Brelands...

• Students of B.F. Skinner

• Left his lab in 1947 to start an animal training business drawing on the techniques developed in Skinner’s lab (applied operant conditioning)

• “observe don’t infer” taken to the extreme and through a very specific lens

• Species-specific biases seen as secondary to the universal laws of learning...

• By 1961 had trained over 6000 animals from 38 species...

• Over this period they encountered a “persistent pattern of discomforting failures”

Our story begins with the Brelands

Thank God there is no piggy bank

involved...

A surprisingly sensitive

performance for one so young... We are not

amused...

Discomforting failures...

• Raccoon...

• Conditioned to pick up a coin: easy

• Conditioned to drop coin in a container: hard

• Persistent “washing” even when it meant delay of the food reward...

• Attempted to condition dropping 2 coins: time for a new trick

Discomforting failures...

• The Dilly-dallying Pig

• Conditioned to pick up a dollar bill and deposit it in a piggy bank (savings & loan companies were evidently big customers)

• Over time, pig would “repeatedly drop it, root it, drop it again, root it along the way, toss it up in the air...”

• “They stretch out the time required for reinforcement when nothing in the experimental setup requires them to do so. They have only to do the little tidbit of behavior to which they were conditioned... to get reinforced immediately. Instead, they drag the process out... Moreover increasing the drive merely intensifies this effect”

Breland, K. and M. Breland (1961). "The Misbehavior of Organisms." American Psychologist 16: 681-684.

The Brelands’ Conclusions...

• “Hidden assumptions which led most disastrously to these breakdowns in the theory”

• “virtual tabla rasa”

• “species differences are insignificant”

• “all responses are all about equally conditionable to all stimuli”

• “It is our reluctant conclusion that the behavior of any species can not be adequately understood, predicted, or controlled without knowledge of its instinctive patterns, evolutionary history, and ecological niche”

The common pattern of misbehavior

Desired

Actual

The common pattern of misbehavior

• The startling thing about misbehavior is the degree to which the opportunity to perform species-specific “appetitive” or seeking behavior may hijack the system even to the extent that it interferes with ultimately satisfying the very drive that the behavior is presumably “intended” to facilitate.

• Performance of the behavior is its own reward: the means become the end and the end is all but forgotten

• Trush in advertising (aka alternative explanation): A contributing factor may have been that tricks involved a focal object that they had to eventually give up or ignore in order to get the treat

The intensity of the misbehavior increased over time

• At least 2 explanations why

• The Brelands noted that increasing the level of the putative drive increased the intensity of the behavior

• There may be a feedback loop: the more the animal does it, the better it feels.

Coppinger: Border collies may give eye because, well, it feels good to give eye...

Coppinger: sled dogs run because it feels good to run...

• A common theme in performance sports with dogs is that the behavior itself is in some sense the reward...

• Sled dogs

• Herding

• Earthdog

• Field events

• Coursing

Coppinger: breed typical display of motor patterns

• The motor patterns themselves are “self-motivating” and “self-rewarding”, although breeds may differ on the levels of motivation and reward, and thus on the tendency, intensity, and threshold stimulus associated with the motor patterns

• The diversity arises from diversely “shaped” brains caused by a diversity of gene action...

• genes interacting with developmental environment

• the diversity is a product of natural AND artificial selection

• Exaggerated and suppressed so as to achieve goal...

Coppinger: breed typical display of motor patterns.

Coppinger, R. and L. Coppinger (2001). Dogs: A Startling New Understanding of Canine Origin, Behavior, and Evolution. New York, NY, Scribner.

Frequency of display and tendency to move on to

next motor pattern varies consistently

across some breeds

Leyhausen: kittens may pounce because, well, it feels good to pounce...

Leyhausen and his kittens...

• Leyhausen observed that kittens performed & perfected predatory motor patterns, e.g., pounce, on their litter-mates long before they were put to the test with real prey

• He hypothesized that appetitive behaviors were likely to be self-motivating & self-rewarding so that the animal would

• be motivated to practice, and perfect, behavior in a non-functional & safe context, e.g., play.

• still perform the behavior in the face of the fact that behaviors early in the predatory sequence fail more often than they succeed even in a mature predator.

Lorenz, K. and P. Leyahusen (1973). Motivation of Human and Animal Behavior: An Ethological View. New York, NY, Van Nostrand Reinhold Co.

Panksepp: The Seeking System

The dopamine pathway

Donahoe, J. W. and D. C. Palmer (1994). Learning and Complex Behavior. Boston MA, Allyn and Bacon.

Rats would work in order to receive electrical stimulation to part of their brain...

• James Olds & Peter Milner first discovered that rats would work (press a lever, etc...) in order to receive electrical stimulation to a part of the hypothalmus (LH)

• The interpretation at the time...

• Invoked neural representation of a specific reward...

• e.g. “as good as eating the cookie”

• Exploratory behavior was secondary despite the fact that the observed behavior & affect was more akin to search/exploratory behavior than consummatory...

•

Panksepp’s reinterpretation of the findings...

• Panksepp took what everyone observed, but discounted, as the primary lesson, in fact, to be drawn from self-stimulation...

• The subjective feeling was one of “intense interest, engaged curiousity, and eager anticipation”

• The observable behavior(s) were those associated with active exploration of the environment. Indeed, cells in the LH fired when engaged in appetitive motor patterns, and stopped when in a position to handle or consume the object of interest.

• Species-specific appetitive behaviors

• His conclusion: it was the seeking not the finding that was rewarding...

The seeking system is invoked by...

• Regulatory imbalances

• External Stimuli

• Cues associated with incentives

Maybe misbehavior is just self-motivated, self-rewarding behaviors hijacking the animal...


This phenomena can be our friend...

Self-reinforcing behaviors???

• Things dogs do that may have roots in predatory (appetitive) motor patterns

• Tugging

• Chasing, Eye-stalk, Pounce

• Shaking & Ripping, Chewing

• Digging

• Retrieving

• Pulling

Some summary thoughts...

• You want a dog with a good amygdala and the right soup of neurotransmitters and hormones.

• You won’t go wrong if you think of the amygdala running ahead of the cognitive machinery, and it is very easy for the amygdala to take over...

• When training think about what will work well on the low road to the amygdala

• Remember that many behaviors, especially behaviors one thinks of as predatory behaviors may be self-motivating & self-rewarding...

• This can be a curse or a blessing or both.

You can know me, you can love me, but you will

never understand me

2 out of 3 ain’t bad...

Using Canine Evolution and Perceptionto our Advantage

Use your dogs’ dogginess to your advantage …


Using your dogs’ dogginess to your advantage

Dogginess = reward Often this is what drives you crazy Is your dog tuned into sounds – maybe squeaky

toys or other sound producing toys Dog likes to chase things – maybe a ball or disc Dog loves to hunt squirrels or to sniff –these can

be rewards


Examples of breed tendencies: Hounds – sight, scent

Problems: hunting, running away, sniffing Possible rewards: food, sniffing, chasing a toy,

running with you Terriers – dig, hunt, act tough thru fear

Problems: digging, hunting, possessive issues, fear

Possible rewards: food, digging, hunting, tugging, squeaky toys, and hose!


Herding - chase, bark, nip, Problems: chasing kids, bikes, barking

excessively Possible rewards: food, toys to chase, tug of war,

herding

Sporting - hunt, high energy, retrieving Problems; easily distracted by scents, too much

energy, possessive issues Possible rewards: food, retrieving, sniffing, tug


Northern types – pull, run, Problems: pulling on leash, running away Possible rewards: food, running with you, carting,

ski/bike pulling, sledding Fighting – quick & high arousal

Problems: high arousal, aggression towards animals

Possible rewards: food, tugging, chasing a toy Guarding – confidence, territorial

Problems: guarding territory Possible rewards: food, tugging, chasing a toy


Reward Examples Sydney and the chickens

Steiff and the hose


Every dog is different Experiment and be creative when finding a

motivator for your dog Some unusual dog rewards: plastic bottles,

paper towel tubes, plastic bags (sight hounds)

Other motivators ? Limited only by

imagination


An important point to take home Innate behaviors can be so self rewarding that

even an often high value reward will not motivate Misbehavior paper by

Brelands and more

recently Panksepp


When the hunt takes over your dog …

use the

hunt as a

reward!


Using sniffing as a rewardSniff reward example

Chief the Visla Could not eat in class when initially outside Waited with short leash on pavement for a

glance at owner Click attention and run to grass for a 20

second sniff Repeated for about 3-4 minutes and attention

became immediate


Sidebar: Dogs need to sniff People need to look Information seeking –

new situations When possible give them a chance to investigate

an area with their nose Can be very helpful for performance dogs with fear

issues


More Motivators beyond food:

PLAY Play is a great relationship builder Most dogs like to play Play can tap into innate and self

rewarding

behaviors


More Motivators beyond food:

LIFE REWARDS Using daily activities as rewards for

controlled behavior Readily available in day to day life Dogs differ in what is important to them


Using Motivators beyond food:

PLAY Observe what your dog

likes and use to your advantage Try the old standbys – fetch & tug You can teach dogs to play with you Find the right game for your dog... Get on the floor and play bow! - try it and see what

happens



PLAY Click and treat into play –

Tugging and retrieving Food toys help transfer

food dogs to play with toys Have pockets to put food into them Cleanrun.com is a good source


Using Motivators beyond food:PLAY Hints for dogs that only play at home

Play with distractions at home Begin with familiar not too distracting places Slowly add new places Very short sessions Use the absolute favorite type of toy

Experiment – don’t give up easily



LIFE REWARDSDogs have daily needs and desires Importance will vary from dog to dog

meals going outside going for a walk play petting chew items ride in the car etc.



LIFE REWARDSExcellent for self control behaviors

Sit, down, stay, not jumping, bolting through doors

Examples: Sit to go out the doorSit to greet peopleWait quietly to come out of crate or pen or be leashedNo behavior = no reward


Other Factors Management: prevent your dog from learning

what you do not want them doing Dogs are learning all the time


Management If there is an opportunity to get what they want,

they will generally take it especially if it has worked before Counter surfing , jumping up, running away,

perpetuating fear aggression Leashes, confinement (crates, gates) Training


Use management to your advantage

Be flexible & open minded in yourideas on motivation

You and the dogs youencounter will enjoy more success…


Always Keeping in Mind - Every dog is different…

Documents

The Cognitive Dog Class 11: Emotion (reward and self-rewarding behaviors...)