View
223
Download
3
Category
Preview:
Citation preview
The fussy brain:
What makes one option more attractive than another?
Steve Fleming and Louise Whiteley
Value
Some decisions are about information gathering, where what matters is being accurate. Many everyday decisions are about what is valuable to us now, and in the future…
OR ?
What is a decision?
DecisionShort- vs
Long-termgain
Context
Risk
Information gathering
Prior Beliefs
Value
Bentham and probability
Jeremy Bentham believed that using “felicific calculus” it was possible to work out the best action to take
“Nature has placed mankind under the guidance of two sovereign masters; pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do”
Jeremy Bentham, 1748-1832
Darling’s investment – predicting the future
The value of the share can rise or fall…
Down5%
Up 10%p = 0.2
p = 0.8
Darling’s investment – predicting the future
Down5%
Up 10%p = 0.2
p = 0.8
Expected value of share = weight each outcome by its probability, then add them all up
Darling’s investment – predicting the future
Down5%
Up 10%p = 0.2
p = 0.8
Expected value of share = weight each outcome by its probability, then add them all up
EV(share) = outcomes p(outcome) x r(outcome)
= (0.2 x 10) + (0.8 x -5) = -2
Darling’s investment – discounting the future
Down20%
Up 25%p = 0.2
p = 0.8Time
Value
Down20%
Up 15%p = 0.2
p = 0.8Time
Value
Share 1
Share 2
In six months…
In six weeks…
OR
Darling’s investment – discounting the future
EV(share) = outcomes p(outcome) x r(outcome)
EV(share) = outcomes λ x p(outcome) x r(outcome)
Time
Value
Measuring impulsivity…
Time
Value
What is a decision?
DecisionShort- vs
Long-termgain
Context
Risk
Information gathering
Prior Beliefs
Value
+ Learning
Investigating value in the brain
1.Find neurons that signal our preferences
2.Work out how these neurons learn from experience to predict future values
3.See how these neurons are affected by probability
4.See how these neurons are affected by when you get the reward
Neurons representing value of choice…
vs.
We want to know if OFC neurons can keep track of different preferences
V(pineapple) V(orange)
orangepineapple
during instruction
just before reward
Preferences in the OFC
• Different groups of neurons within OFC are associated with different types of reward (e.g. orange vs. pineapple)
• OFC neurons also know how much reward is on offer - e.g. six apples vs. one piece of cake
Padoa-Schioppa & Assad (2006)
2. Work out how neurons predict future values
• Learn from the past!
New value = prediction + new information
= difference between prediction and what happened…
So:
New value = prediction + α(outcome – prediction)
How does the brain predict future values?
(outcome – prediction)
Schultz et al. (1997) Science
Reward unpredicted, reward occurs
Reward predicted, reward occurs
Reward predicted, reward absent
Knutson et al. (2005)
OFC Basal ganglia
EV = outcomes p(outcome) x r(outcome)
How does the brain respond to probability?
Would you like a) £900 now or b) £1000 in one month’s time?
4. See how these neurons are affected by when you get a reward
Brain data help us refine our theory
OFC
Kable & Glimcher (2007)
• Two theories: –a) brain region knows about “absolute” value, communicates it to somewhere else which knows about how far away it is in time–b) discounting the future is inherent to our value system
What is involved in making a decision?
What happens when things get more complicated…?
DecisionShort- vsLong-term
gain
Context
Risk
Information gathering
Prior Beliefs
Value
Many decision systems in parallel
We’ve been focusing on how the brain learns values from experience, building up habits that can be used again
Many decision systems in parallel
Sometimes, we can’t learn habits, and need to look ahead in a more sophisticated way…
Many decision systems in parallel
And sometimes we don’t need to bother - we have innate values attached to things like food and shelter
Bentham again…
“the game of push pin is of equal value with poetry”
vs. J.S. Mill…
“it is better to be … Socrates dissatisfied than a fool satisfied”
Complicated value
Many decision systems in parallel
In the next talk we hear more about these three systems, about how the brain chooses which system to use, and how
this can lead us astray…
What is a decision?
DecisionShort- vs
Long-termgain
Context
Risk
Information gathering
Prior Beliefs
Value
Recommended