Symposium: Decision-making in health and disease - Part 2

Given at Gresham College on 28th May 2009

Introduction By Elisabeth Rounis and Louise Whiteley

What affects a decision?

OR ?

What affects a decision?

OR ?

What affects a decision?

You have:

What would you rather…

OR ?

What affects a decision?

OR

What affects a decision?

OR

Healthy? Or not?

What affects a decision?

Vs.

Wartime Peacetime

Big decisions…

Forming a decision

Decision

Information

Behaviour

Decision

Decision

Behaviour

Neuronal recording

The Start: Building a perceptual decision

100coh_circle.mov

http://monkeybiz.stanford.edu/movies/0coh_circle.qt

0coh_circle.mov

LIP

. .Are the dots moving left or right?

0coh_circle.mov

So what happens when we actually see something?

Representation of sensory evidence

Value

B

A

Compute V(A) and V(B)

Choose

Value

B=£500

A=£1000

V(A) > V(B), so choose A

Discounting

B

A

Compute V(A) and V(B)

Choose

Time

Value

Now!

3 weeks…

Discounting

B = £500

A = £1000

V(Bt=now) > V(At=3 weeks) so choose B

3 weeks…

Now!

Relative wealth

B

A

Compute V(A) and V(B) relative to your wealth now and in 3 weeks

Choose

Time

ValueNow!

3 weeks… Utility

Value

Relative wealth

B = £500

A = £1000

Now, V(Bt=now, Wt=now) < V(At=3 weeks, Wt=3 weeks) so choose A

Plus pension of £1000 in 3 weeks!!

Probability

B

A

Weight V(A) and V(B) by the probability they will occur

ChooseU = p * V

x p(A)

x p(B)

Probability

B = £500

A = £1000

In other words… U(Bt=now, Wt=now) > U(At=3 weeks, Wt=3 weeks) so choose B

U = p * V

Risky bet

Safer bet

p = 0.1

p = 0.9

Now, V(Bt=now, Wt=now)*0.9 > V(At=3 weeks, Wt=3 weeks)*0.1 so choose B

Deciding what we’re seeing

A

B p = 0.9

p = 0.1

On balance, we think we saw B

Choose

Deciding what we’re seeing

Tumour

Healthy p = 0.7

p = 0.3

p(healthy|x-ray) > p(tumour|x-ray), so thought to be healthy…

Training a medical student…

Priors

A

B p = 0.7 x prior

p = 0.3 x prior

Now we are not so sure…

Choose B is only rarely seen, small prior belief

Priors

Tumour

Healthy

Smokes 40 a day

p = 0.3

p = 0.7

Now, p(healthy|x-ray) * p(healthy) < p(tumour|x-ray) * p(tumour), so more likely a tumour

Training a medical student…

Value (again…)

A

B p = 0.7 x prior x value

p = 0.3 x prior x value

Better be safe than sorry – decide on “A”

Choose Detecting A is very important, detecting B is less so

Value (again…)

Tumour

Healthy

Smokes 40 a day

p = 0.3

p = 0.7

Treat

All clear

Treat

All clear

V = 100

V = -500

V = -20

V= 0

In the real world…

1) U(“tumour”|tumour) = p(tumour) * p(tumour|x-ray) * V(treat, tumour)

2) U(“healthy”|tumour) = p(healthy) * p(healthy|x-ray) * V(all clear, tumour)

Then, U(“tumour”) = 1+3, U(“healthy”) = 2+4… choose which is bigger!

3) U(“tumour”|healthy) = p(healthy) * p(healthy|x-ray) * V(treat, healthy)

4) U(“healthy”|healthy) = p(healthy) * p(healthy|x-ray) * V(all clear, healthy)

Putting it all together

Decision

Value

Discounting

Relative wealthRisk

Perceptualuncertainty

Priors

Decision

Value

Discounting

Relative wealthRisk

Perceptualuncertainty

Priors

?

Behaviour

Neuronal recording

Functional Imaging

Functional vs Structural

Task-related activity Structure ?Pathology

Need good hypotheses!

So… how is decision made in the brain???

An appropriate behavioural paradigm

What is a decision, and what’s going on in the brain?

Elisabeth Rounis and Louise Whiteley

The mathematical brain:

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Choosing between different options….

What is involved in making decisions?

OR ?

Value

DecisionShort- vs. Long-term

gain

Context

Risk

Information gathering

Priors

Value

What is involved in making decisions?

OR

?

Short- vs. Long-term gain

DecisionShort- vs Long-term

gain

Context

Risk

Information gathering

Prior Beliefs

Value

What is involved in making decisions?

You have:

What would you rather… OR ?

Context

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

What is involved in making decisions?

OR

Risk

DecisionShort- vs

Long-termgain

ContextRisk

Information gathering

Prior Beliefs

Value

What is involved in making decisions?

Healthy? Or not?

Information gathering

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

What is involved in making decisions?

Vs.

Wartime Peacetime

Prior beliefs

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

What is involved in making a decision?

A model that helps us understand decision making, predict behaviour, and know kind of signals to look for in the brain…

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

How do we link brain, behaviour, and theory?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Do a behavioural study…

Non-Dieters Dieters

What happens in the brain? What are the

differences in decision between dieters and

non-dieters?

Brain has specialised areas that are interconnected… but what do these areas do?

Thinking, planning, moving

Feeling, recognising

Seeing

Understanding

‘Higher’ Order areas located in front…

… Recording directly from brain cells (‘neurons’)

Neuron 1

Neuron 2

Ave

rage

act

ivity

Time

dendrites

soma

axon

synapses

Functional Imaging of whole brain regions

What is involved in a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Information gathering

OR

Healthy? Or not?

Information gathering

• Enigma code breakers - how much time can you sacrifice to gather more information?

Investigating information gathering in the brain

1. Think of a really simple decision

2. Find neurons in the brain that carry information important for our decision

3. Find neurons in the brain that add up this information over time

1. A really simple decision…

Which overall direction are the dots moving in?

0% coherence (random)

100% coherence(all in one direction)

50% coherence

The more random dots there are, the longer you need to work out the direction of the non-random onesi.e. the more information you need to gather…

Random dots

Random dots… Can you see a direction?

Random dots… now they’re moving right!

2. Neurons that care about motion…

First, we need to find an area where the brain cells (neurons) carry information about the direction of motion…

Area MT neurons respond more to a ‘preferred’ direction Stimulating neurons that prefer ‘down’ produces ‘motion hallucinations’

MTa

ctiv

ity o

f ne

uro

ns preferred direction

Britten et al. 2002, Huk and Shadlen 2005

3. Tracking information, adding it up…

MT

LIP

Gold and Shadlen 2007

Video of activity in LIP

Video of activity in LIP

Video of activity in LIP

Roitman and Shadlen 2002

• Note the cell is always active but more so in the presence of the targets and as evidence accumulates

• Activity is lower if decision-maker has to choose a target that is not in the preferred direction of the cell

So we’ve looked at dots, but there’s lots of other stuff in the world too - a range of brain areas track and gather information

What is involved in making decisions?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Prior Knowledge

Healthy? Or not?

What if we now find out the patient smokes 50 a day?

Prior Knowledge

Prior knowledge affects perception

What colour is a banana? YELLOW!!!

If you show people lots of bananas of different shades along the blue-yellow spectrum and ask them which one is grey?

they pick a slightly blue one, because our expectation that they will be yellow influences perception

true greyjudged grey Hansen et al. 2006

So what about prior beliefs in the brain?

MT

LIP • This is still under investigation!

• Some candidates have been suggested, including ‘action’ areas of the visual system

• Understanding prior beliefs in the brain might help us decide between models

SC

Perceptual decision making…

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Information gathering

Is Wally on the right or the left hand side of the beach?

Adding prior information

He’s definitely next to one of the boats…

Next we consider the value of different options

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Any questions…?

A candidate brain area…

1 target

2 targets

4 targets

DimPossible Targets Select

8 targets

Accumulation of evidence over time is lower with more targets to choose from (ie more uncertainty as to probability of target location)

SC

act

ivity

at

‘DIM

’

TimeBasso and Wurtz 1998

Let’s ‘dim’ the lights…

A candidate brain area…

Accumulation of evidence over time is lower with more targets to choose from (ie more uncertainty as to probability of target location)

SC

act

ivity

at

‘DIM

’

TimeBasso and Wurtz 1998

Let’s ‘dim’ the lights…

MT

LIP

SC

The fussy brain:

What makes one option more attractive than another?

Steve Fleming and Louise Whiteley

Value

Some decisions are about information gathering, where what matters is being accurate. Many everyday decisions are about what is valuable to us now, and in the future…

OR ?

Predicting the future

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Bentham and probability

Jeremy Bentham believed that using “felicific calculus” it was possible to work out the best action to take

“Nature has placed mankind under the guidance of two sovereign masters; pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do”

Jeremy Bentham, 1748-1832

Darling’s investment – predicting the future

The value of the share can rise or fall…

Down5%

Up 10%p = 0.2

p = 0.8

Darling’s investment – predicting the future

Down5%

Up 10%p = 0.2

p = 0.8

Expected value of share = weight each outcome by its probability, then add them all up

Darling’s investment – predicting the future

Down5%

Up 10%p = 0.2

p = 0.8

Expected value of share = weight each outcome by its probability, then add them all up

EV(share) = outcomes p(outcome) x r(outcome)

= (0.2 x 10) + (0.8 x -5) = -2

Darling’s investment – discounting the future

Down20%

Up 25%p = 0.2

p = 0.8Time

Value

Down20%

Up 15%p = 0.2

p = 0.8Time

Value

Share 1

Share 2

In six months…

In six weeks…

OR

Darling’s investment – discounting the future

EV(share) = outcomes p(outcome) x r(outcome)

EV(share) = outcomes λ x p(outcome) x r(outcome)

Time

Value

Measuring impulsivity…

Time

Value

How are these values learnt?

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

+ Learning

Investigating value in the brain

1.Find neurons that signal our preferences

2.Work out how these neurons learn from experience to predict future values

3.See how these neurons are affected by probability

4.See how these neurons are affected by when you get the reward

1. Find neurons that signal our preferences

OFC – Orbitofrontal cortex

MT

LIP

OFC

Neurons representing value of choice…

vs.

We want to know if OFC neurons can keep track of different preferences

V(pineapple) V(orange)

orangepineapple

during instruction

just before reward

Preferences in the OFC

• Different groups of neurons within OFC are associated with different types of reward (e.g. orange vs. pineapple)

• OFC neurons also know how much reward is on offer - e.g. six apples vs. one piece of cake

Padoa-Schioppa & Assad (2006)

2. Work out how neurons predict future values

• Learn from the past!

New value = prediction + new information

= difference between prediction and what happened…

So:

New value = prediction + α(outcome – prediction)

How does the brain predict future values?

(outcome – prediction)

Schultz et al. (1997) Science

Reward unpredicted, reward occurs

Reward predicted, reward occurs

Reward predicted, reward absent

Changing our predictions with new information

Time

Read label

It’s corked

Taste…

New information in the brain…

Seymour et al. (2004)

Basal ganglia

3. See how these neurons are affected by probability

p = 0.8 p = 0.2

LIP

Platt & Glimcher (1997)

How does the brain respond to probability?

Knutson et al. (2005)

OFC Basal ganglia

EV = outcomes p(outcome) x r(outcome)

How does the brain respond to probability?

Would you like a) £900 now or b) £1000 in one month’s time?

4. See how these neurons are affected by when you get a reward

Short- and long-term gain in the brain

Kable & Glimcher (2007)

OFC

Brain data help us refine our theory

OFC

Kable & Glimcher (2007)

• Two theories: –a) brain region knows about “absolute” value, communicates it to somewhere else which knows about how far away it is in time–b) discounting the future is inherent to our value system

What is involved in making a decision?

What happens when things get more complicated…?

DecisionShort- vsLong-term

gain

Context

Risk

Information gathering

Prior Beliefs

Value

Many decision systems in parallel

We’ve been focusing on how the brain learns values from experience, building up habits that can be used again

Many decision systems in parallel

Sometimes, we can’t learn habits, and need to look ahead in a more sophisticated way…

Complicated or one-off decisions…

Many decision systems in parallel

And sometimes we don’t need to bother - we have innate values attached to things like food and shelter

Bentham again…

“the game of push pin is of equal value with poetry”

vs. J.S. Mill…

“it is better to be … Socrates dissatisfied than a fool satisfied”

Complicated value

Many decision systems in parallel

In the next talk we hear more about these three systems, about how the brain chooses which system to use, and how

this can lead us astray…

Any questions...?

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Neurons representing value of choice…

Padoa-Schioppa & Assad (2006)