Abstract

Predictive coding models and the free-energy principle, suggests that cortical activity in sensory brain areas reflects the precision of prediction errors and not just the sensory evidence or prediction errors per se. If we assume that neuronal activity encodes a probabilistic representation of the world that optimizes free-energy in a Bayesian fashion, then, because free-energy bounds surprise or the (negative) log-evidence for internal models of the world, this optimization can be regarded as evidence accumulation or (generalized) predictive coding. Crucially, both predictions about the state of the world generating sensory data and the precision of those data have to be optimized. In other words, we have to make predictions (test hypotheses) about the content of the sensorium and predict our confidence in those hypotheses. I hope to demonstrate the meta-representational aspect of inference using simulations of visual searches and action selection - to illustrate their nature and promote discussion about its role in high-order cognition.

November 29th 4:30 – 6:00pm Old LibraryKarl Friston

Meta-cognition, prediction, precision (Discussant, Andreas Roepstorff, Aarhus)

SEMINARS ON META-COGNITION, 2012–2013

The basic idea: active inference and free energy

Beliefs about beliefs: beliefs about uncertainty

Beliefs about beliefs: beliefs about precision and agency

“Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism” - von Helmholtz

Thomas Bayes

Geoffrey Hinton

Richard Feynman

From the Helmholtz machine to the Bayesian brain and self-organization

Hermann Haken

Richard GregoryHermann von Helmholtz

tem

pera

ture

What is the difference between a snowflake and a bird?

Phase-boundary

…a bird can act (to avoid surprises)

( , ) ss a g ψ ω

Hidden states in the world Internal states of the agentSensations

argmin ( , )F s

argmin ( , )a F s μ a

( , ) xa ψ f ψ ω

ω

Action

External states

FluctuationsPosterior expectations

The basic ingredients

What we need to explain: how do we minimise the dispersion of sensory states (homoeostasis)?

ln ( ( ) | ) [ ( | )]p s t m dt H p s m

( , , ) ln ( | ) [ ( | ), ( | )]

[ ln ( , )] [ ( | )]

KL

q

F s m p s m D q p s

E p s H q

( ) ln ( ( ) | ) [ ( | )]dtF t dt p s t m H p s m

The principle of least action

The principle of least free energy (minimising surprise)

Self organisation

Ergodic theorem

Bayesian inference

Maximum entropy principle

How can we minimize surprise (prediction error)?

Change sensations

sensations – predictions

Prediction error

Change predictions

Action Perception

…action and perception minimise free energy

( )s g( )a t

Prior distribution

Posterior distribution

Likelihood distribution

temperature

( )t

Action as inference – the “Bayesian thermostat”

( )t

( | )p s

20 40 60 80 100 120

s

( | )p s( )p

2 2

2 2

argmin ( , , ) argmin ( ( ) ( )) ( )

argmin ( , , ) argmin ( ( ) ( )) ( )

s

sa a

F s s a g

a F s s a g

Perception

Action

( , ) ss a g ψ ω

Hidden states in the world Internal states of the agentSensations

argmin ( , )F s

argmin ( , )a F s μ a

( , ) xa ψ f ψ ω

ω

Action

External states

FluctuationsPosterior expectations

How might the brain minimise free energy (prediction error)?

…by using predictive coding (and reflexes)

(1) (1) (1) (1)

(1) (1) (1) (1) (1)

( 1) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

( , )

( , )

( , )

( , )

v

x

i i i i iv

i i i i ix

s g x v

x f x v

v g x v

x f x v

( ) ( ) ( ) ( ) ( 1) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( 1)

( ) ( ) ( ) ( )

(1) (1)

( ( , ))

( ( , ))

( )

i i i i i i i iv v v v v x v

i i i i i i i ix x x x x x v

i i i i iv v v v

i i i ix x x

a v v

g

f

D

D

a

D

( , , )

( , , )

( , )

( , )

v

x

aa F s

F s

s g x v a ω

x f x v a ω

D

Free energy minimisation Generative model Predictive coding with reflexes

(1)x

(1)v

(2)x

(2)v

(2)x

(3)v

(2)v

(1)x

(1)v (0)v

(1)x

(1)v

(2)x

(2)v

(2)x

(3)v

(2)v

(1)x

(1)v

(0)v

Expectations:

Predictions:

Prediction errors:

( ) ( ) ( ) ( ) ( )

( 1) ( ) ( ) ( ) ( )

( , )

( , )

i i i i ix

i i i i iv

Dx f x v

v g x v

Generative model

Model inversion (inference)

A simple hierarchy

(0)

( )pa i i

s v

( ) ( ) ( ) ( ) ( 1)

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( 1) ( )

( ) ( ) ( ) ( ) ( ) ( )

( , )

( , )

( )

( )

i i i i iv v v v

i i i ix x x

i i i ix v

i i i ix v

i i i i i iv v v v v

i i i i i ix x x x x

D

D

g g

f f

g

f

D

Outward prediction stream

Inward errorstream

From models to perception

Attentionoccipital cortex geniculate

visual cortex

retinal input

pons

oculomotor signals

( ) ( ) ( ) ( ) ( 1)

( ) ( ) ( ) ( )

i i i i iv v v v

i i i ix x x

D

D

( ) ( ) ( ) ( ) ( 1) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ( , ))

( ( , ))

i i i i i i i iv v v v v x v

i i i i i i i ix x x x x x v

g

f

D

( )i

( )i

Prediction error (superficial pyramidal cells)

Conditional predictions (deep pyramidal cells)

Top-down or backward predictions

Bottom-up or forward prediction error

proprioceptive inputreflex arc

David Mumford

Predictive coding with reflexes Action

Perception

(1)a va s

Biological agents resist the second law of thermodynamics

They must minimize their average surprise (entropy)

They minimize surprise by suppressing prediction error (free-energy)

Prediction error can be reduced by changing predictions (perception)

Prediction error can be reduced by changing sensations (action)

Perception entails recurrent message passing in the brain to optimize predictions

Action makes predictions come true (and minimizes surprise)

Beliefs about beliefs: beliefs about uncertainty

Perception as hypothesis testing – action as experiments

But how do we think action will change our beliefs?

Searching, salience and saccades

( )s g

2 2

2 2

argmin ( ( ) ( )) ( )

argmin ( ( ) ( )) ( )

argmin ?

s

sa

s a g

a s a g

( )a t

Where do I expect to look?

( , ) ( | ) ( | )

[ ln ( ( ) | )] [ ( | ( ))]t t

H S H S m H S

E p s t m E H S s t

( ) [ ( | , )]H q S ( )s t S

saliencevisual inputstimulus sampling

Sampling the world to minimise uncertainty

Perception as hypothesis testing – saccades as experiments

( )t

Free energy principle minimise uncertainty

( ) argmin{ [ ( | , )]}t H q

( , ) ss a g ψ ω

Hidden states in the world Internal states of the agentSensations

argmin ( , )F s

argmin ( , )a F s μ a

( , ) xa ψ f ψ ω

ω

Action

External states

FluctuationsPosterior expectations

argmin [ ( | , )]H q

Prior expectations

,x p Frontal eye fields

ps

,v p

qs

,x pu

u

,x q

,x q

,v q

px

Pulvinar salience mapFusiform (what)

Superior colliculus

Visual cortex

oculomotor reflex arc

( )S

pxParietal (where)

a

200 400 600 800 1000 1200 1400-2

0

2Action (EOG)

time (ms)

200 400 600 800 1000 1200 1400

-5

0

5

Posterior belief

time (ms)

Saccadic fixation and salience maps

Visual samples

Conditional expectations about hidden (visual) states

And corresponding percept

Saccadic eye movements

Hidden (oculomotor) states

Beliefs about beliefs: beliefs about precision

If beliefs cause movement, how can I move when sensory evidence compels me to believe that I am not moving?

Sensory attenuation, illusions and agency

ss

ps

evix

14

8

8

( )

~ (0, )

~ (0, )

p is

s i e

i i x

s

x

ss

s

a

e I

e I

xω

x v

x x x ω

ω

ω

N

N

14

14

4

6

~ (0, )

~ (0, ) 8 ( )

~ (0, )

p is

s i e

i i ix

e e e

iv

e

s

x i i

v

s xs

s x x

x v xx

x v x

vv

v

e I

e I x v

e I

N

N

N

a

Generative process Generative model

Making your own sensations

x

ss

xv

v

ps

,v p

a

,v s

Motor reflex arc

thalamus

sensorimotor cortex

prefrontal cortex

ss

descending predictions

ascending prediction errors

descending modulation

High sensory attenuation

psss

ix

iv a

5 10 15 20 25 30

-0.5

0

0.5

1

1.5

2

prediction and error

Time (bins)5 10 15 20 25 30

-0.5

0

0.5

1

1.5

2

hidden states

Time (bins)

5 10 15 20 25 30

-0.5

0

0.5

1

hidden causes

Time (bins)5 10 15 20 25 30

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time (bins)

perturbation and action

ex

ev

Low sensory attenuation

5 10 15 20 25 30

-0.5

0

0.5

1

1.5

2

prediction and error

time5 10 15 20 25 30

-0.5

0

0.5

1

1.5

2

hidden states

time

5 10 15 20 25 30

-0.5

0

0.5

1

hidden causes

time5 10 15 20 25 30

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

time

perturbation and action

10 20 30 40 50 60-0.5

0

0.5

1

1.5

2prediction and error

Time (bins)10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2hidden states

Time (bins)

10 20 30 40 50 60-0.5

0

0.5

1

1.5

2hidden causes

Time (bins)10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2

Time (bins)

perturbation and action

10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2

hidden states

Force matching illusion

10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2

prediction and error

Time (bins) Time (bins)

Sensory attenuation

10 20 30 40 50 60

-0.5

0

0.5

1

1.5

hidden causes

Time (bins)10 20 30 40 50 60

-0.5

0

0.5

1

1.5

Time (bins)

perturbation and action

0 0.5 1 1.5 2 2.5 30

0.5

1

1.5

2

2.5

3

External (target) force

Self-

gene

rate

d(m

atch

ed) f

orce

External (target) force

Self-

gene

rate

d(m

atch

ed) f

orce

Simulated Empirical (Shergill et al)

Failures of sensory attenuation, with compensatory increases in non-sensory precision

A failure of sensory attenuation and delusions of control

10 20 30 40 50 60-0.5

0

0.5

1

1.5

2

2.5

3

3.5prediction and error

Time (bins)10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2

2.5

3

3.5hidden states

Time (bins)

10 20 30 40 50 60-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5hidden causes

Time (bins)10 20 30 40 50 60

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

Time (bins)

perturbation and action

Thank you

And thanks to collaborators:

Rick AdamsAndre Bastos

Sven BestmannJean DaunizeauMark EdwardsHarriet BrownLee HarrisonStefan KiebelJames Kilner

Jérémie MattoutRosalyn Moran

Will PennyKlaas Stephan

And colleagues:

Andy ClarkPeter Dayan

Jörn DiedrichsenPaul FletcherPascal Fries

Geoffrey HintonJames HopkinsJakob Hohwy

Henry KennedyPaul Verschure

Florentin Wörgötter

And many others

( , ) ( | ) ( | )

[ ln ( ( ) | )] [ ( | ( ))]t t

H S H S m H S

E p s t m E H S s t

Searching to test hypotheses – life as an efficient experiment

Free energy principle minimise uncertainty

( ) argmin{ [ ( | , )]}t H q

310 s

010 s

310 s

610 s

1510 s

Perception and Action: The optimisation of neuronal and neuromuscular activity to suppress prediction errors (or free-energy) based on generative models of sensory data.

Learning and attention: The optimisation of synaptic gain and efficacy over seconds to hours, to encode the precisions of prediction errors and causal structure in the sensorium. This entails suppression of free-energy over time.

Neurodevelopment: Model optimisation through activity-dependent pruning and maintenance of neuronal connections that are specified epigenetically

Evolution: Optimisation of the average free-energy (free-fitness) over time and individuals of a given class (e.g., conspecifics) by selective pressure on the epigenetic specification of their generative models.

Time-scale Free-energy minimisation leading to…