Paul Cisek Model - No "Decision" "Decision-Making"

The vanishing central executive

Distributed neural mechanisms of decision-making

The vanishing central executive

Distributed neural mechanisms of decision-making

Paul CisekPaul Cisek

Summer School in Cognitive SciencesSummer School in Cognitive SciencesEvolution and Function of ConsciousnessEvolution and Function of Consciousness

July 4, 2012July 4, 2012

Our question:Our question:

• When, where, how and why — since the origin of life on Earth When, where, how and why — since the origin of life on Earth about 4 billion years ago — did organisms' input/output about 4 billion years ago — did organisms' input/output functions become conscious input/output functions?functions become conscious input/output functions?

• Why “input/output” functions?Why “input/output” functions?

But first, another question:But first, another question:

What is behavior?What is behavior?

“The whole neural organism, it will be remembered, is, physiologically considered, but a machine for converting stimuli into reactions” (James, 1890, p. 372).

BehaviorBehavior: An analysis of the world, : An analysis of the world, followed by deliberation and planning, followed by deliberation and planning, followed by execution of the plan.followed by execution of the plan.

““sense, think, act”sense, think, act”

motoroutput

sensoryinput

ActionPerception

representationof the world

Cognition

representationof the motor plan

William James

Psychological architecture for behaviorPsychological architecture for behavior

motoroutput

sensoryinput

ActionPerception

representationof the world

Cognition

representationof the motor plan

• University courses• Textbooks• Journals• Conferences• Academic departments• Grant review committees• Scientists• Questions we ask• Theories we propose

Q: From where does this view originate?

“Dualism”“Dualism”

• Philosophy: The mind is non-physicalPhilosophy: The mind is non-physical– This forces interfaces between non-

physical mind and physical world

• Psychology: Study of the psychePsychology: Study of the psyche– Structuralism: The mind is studied

through introspection

Perception Actionsensoryinput

motoroutputMind

Socrates

Descartes

John Locke

Wilhelm Wundt

http://en.wikipedia.org/wiki/File:Locke-John-LOC.jpg

BehaviorismBehaviorism

– Stop this metaphysical nonsense…


motoroutputMind

John Watson

BehaviorismBehaviorism

– Stop this metaphysical nonsense… – Perception and Action are directly linked– Subject matter: Learning laws which establish the linkage


motoroutput

John Watson Ivan Pavlov Edward Thorndike B.F. Skinner

CognitivismCognitivism

– Internal processes are indispensable


motoroutput

Tolman

CognitivismCognitivism

– Internal processes are indispensable– Cognition takes the mind’s place– A fully physical process – but what

kind?– “Information processing”

• Definition of “information”• Definition of “processing”

– Cognition is a computational process• Linguistics• Language of thought


motoroutput

Cognition

Tolman

Shannon

Turing

Chomsky

Fodor

Computational view of the brainComputational view of the brain

• The “computer metaphor”The “computer metaphor”– Cognition is like computation:

Rule-based manipulation of representations (Newell & Simon, Pylyshyn)

– The mind is the software (Block)

– Studies of mental phenomena may be conducted independently of studies of brain physiology

• Less to worry about• Not so much known (yet) about the brain• Historical separation between psychology and biology

Newell & Simon

What kinds of representations?What kinds of representations?

• ““Descriptive” representationsDescriptive” representations– Capture knowledge about the world

and the organism– Explicit– Objective, accurate to external

reality, uncontaminated by internal states

– Examples:• Reconstructed visual image• 3-D map of the world• Labeled objects• Desired path of the hand in space

Descriptive representations

delineate the conceptual borders

between the processes that

construct them and the

processes that use them.

David Marr

input/output

^

input/output

^

Can we use this to understand the brain?Can we use this to understand the brain?

• Cognitive NeuroscienceCognitive Neuroscience– How are psychological /

cognitive functions produced by the brain?

– Ex: Decision-making– Based on the concepts of

cognitivism• Computation, descriptive

representations, working memory, attentional filters, motor programs, etc.

Michael Gazzaniga

Where is the central representation?Where is the central representation?

• The visual systemThe visual system– Two visual processing streams:

• ventral “what” • dorsal “where”

– Separate regions analyze color, motion, form, etc.

– Separate regions for near and far space

• Binding problemBinding problem– How to create the unified

representation of the world that is needed as input for cognition?

where

what

• Primary sensory and motor regionsPrimary sensory and motor regions

• ““Association” regionsAssociation” regions– Appear to first encode sensory,

then motor representations– Even true at the level of

individual neurons

Example: Lateral intraparietal areaExample: Lateral intraparietal area– is it “attention”?

(input to cognition)– or “intention”?

(output of cognition)– How could it be both?– Could it be cognition?

Perception, Cognition, & Action Systems?Perception, Cognition, & Action Systems?

Cognition

Cognition

Cognition?

• Decision-makingDecision-making– Neural correlates in prefrontal

and orbitofrontal cortex– also in parietal cortex– Premotor cortex– Supplemental motor area– Frontal eye fields– Basal ganglia– Even the superior colliculus

– Activity reflects decision everywhere at about the same time (~150ms)

Where is the central executive?Where is the central executive?

Conceptual challengesConceptual challenges

• The binding problemThe binding problem– How to create the unified representation of the world that is needed

as input for cognition?

• The problem of meaningThe problem of meaning– How does a computational process know the meaning of the

representations that it manipulates?– “Chinese Room” (Searle)– The “symbol grounding problem” (Harnad)– Representations are purely syntactic, they have no intrinsic

semantics, no meaning to the system that uses theminput/output

^

• Some observations:Some observations:1. This model inherits its structure from

mind-body dualism2. Was designed to explain the abstract

problem-solving behavior of adult humans3. Its concepts were developed under the

explicit assumption that the substrate doesn’t matter

• Perhaps it should not be surprising that this model has difficulty Perhaps it should not be surprising that this model has difficulty explaining neural data…explaining neural data…

Psychological architecture for behaviorPsychological architecture for behavior

motoroutput

sensoryinput

ActionPerception Cognition

EvolutionEvolution

• Two key concepts:Two key concepts:

– Natural selection• What is the selective advantage of X?

– Descent with modification• What are the phylogenetic origins of X?

Darwin

The long history of behaviorThe long history of behavior

Are brains input/output devices?Are brains input/output devices?

• What else could they be?What else could they be?

What kinds of devices are living systems?What kinds of devices are living systems?

• Control systemsControl systems::– Ex: Biochemistry

• Suppose there is some substance A necessaryfor survival

• Suppose there’s a catalyst for creating A whose action is regulated inversely by the concentration of A

• Feedback control system• Exploits consistencies in the laws of chemistry• Control loop within the organism: “Physiology”

A

What kinds of devices are living systems?What kinds of devices are living systems?

• Control systems can extend beyond the skinControl systems can extend beyond the skin– Ex: Kinesis

• Suppose substance B cannot be produced within the body, must be absorbed from the world

• If the local concentration of substance B falls below desired levels, move randomly

• Exploits statistics of nutrient distributions(assumes that there is more elsewhere)

• Control loop that extends outside the skin: “Behavior”

– Reliable motor-sensory contingencies exist• Statistics of food distributions (move → find food)• Laws of optics and mechanics (contract muscle → arm moves)• Laws of interaction (you show teeth → I back off)

• Animals are constantly doing whatever brings them to the most Animals are constantly doing whatever brings them to the most desirable situation (full stomach, safety, etc.)desirable situation (full stomach, safety, etc.)

• “Behavior: The control of perception” (Powers, 1973)

AB

Concentration of [B]

Different ways of looking at behaviorDifferent ways of looking at behavior

1.1. Given a perception, produce the best actionGiven a perception, produce the best action

“The whole neural organism, it will be remembered, is, physiologically considered, but a machine for converting stimuli into reactions” (James, 1890).

2.2. Of the possible actions, produce that which Of the possible actions, produce that which results in the best perceptionresults in the best perception

“What we have is a circuit… the motor response determines the stimulus, just as truly as sensory stimulus determines movement” (Dewey, 1896).

John Dewey

William James

EthologyEthology

• Studies of animal behavior in Studies of animal behavior in the wildthe wild

• Species-specific behavioral Species-specific behavioral nichesniches

• ““Closed-loop” sensorimotor Closed-loop” sensorimotor controlcontrol

• Key stimuliKey stimuli

Von Uexküll

Lorenz & Von Holst

Tinbergen

http://en.wikipedia.org/wiki/File:Uex_photo_full.jpg

http://upload.wikimedia.org/wikipedia/commons/9/95/Lorenz_and_Tinbergen1.jpg

What kinds of representations?What kinds of representations?

• ““Descriptive” representationsDescriptive” representations– Capture knowledge about the world

and the organism– Explicit– Objective, accurate to external

reality, uncontaminated by internal states

– Examples:• Reconstructed visual image• 3-D map of the world• Labeled objects• Desired path of the hand in space

• ““Pragmatic” representationsPragmatic” representations– Used to guide interaction between

the world and the organism– Implicit– Subjective, mix external reality and

internal state, often correlate with many variables at once

– Examples:• Salience map• Motor signals to the limb• Subject-dependent opportunities for

action (“affordances”)

David Marr J.J. Gibson

Example: Decision-makingExample: Decision-making

What to do?What to do?Move the queen?Protect the pawn?Threaten the

knight?

““Selection”Selection”

How to do it?How to do it?Which grasp point?What trajectory?How to avoid

obstacles?

““Specification”Specification”

• Classical model:Classical model:– First decide what to do (select) then plan the movement (specify)– Sense, think, act

Decision-making in the wildDecision-making in the wild

• The world presents animals with multiple opportunities for action (“affordances”)The world presents animals with multiple opportunities for action (“affordances”)• Cannot perform all actions at the same timeCannot perform all actions at the same time• Real-time activity is constantly modifying affordances, introducing new ones, etc.Real-time activity is constantly modifying affordances, introducing new ones, etc.

Action specification and selection must occur in parallel

Sensorimotor contingencies influence how selection should be done

Cel

l act

ivity

Direction

Dis

tanc

e

Cel

l act

ivity

Specification and selection in parallelSpecification and selection in parallel

• Action SpecificationAction Specification: Activation of parameter regions corresponding : Activation of parameter regions corresponding to potential actionsto potential actions

• Action SelectionAction Selection: Competition between distinct regions of activity: Competition between distinct regions of activity

A population of tuned neurons

What are the neural substrates?What are the neural substrates?

attention Specification in the dorsal visual stream– Cells sensitive to spatial visual information

(Ungerleider & Mishkin …)– Involved in action guidance (Milner &

Goodale)– Divergence into separate sub-streams, each

specialized toward different kinds of actions (Stein; Andersen; Colby & Goldberg; Matelli & Luppino ...)

– An increasing influence of attentional effects, enhancing information from particular regions of interest (Duncan & Desimone; Posner & Gilbert; Treue; Boynton ...)

– Parietal representation of external world is “sparse” (Goldberg)

Specification in the dorsal visual stream– Cells sensitive to spatial visual information

(Ungerleider & Mishkin …)– Involved in action guidance (Milner &

Goodale)– Divergence into separate sub-streams, each

specialized toward different kinds of actions (Stein; Andersen; Colby & Goldberg; Matelli & Luppino ...)

– An increasing influence of attentional effects, enhancing information from particular regions of interest (Duncan & Desimone; Posner & Gilbert; Treue; Boynton ...)

– Parietal representation of external world is “sparse” (Goldberg)

potential actions

attention

Fronto-parietal systemFronto-parietal system– Activity related to potential motor

actions (Andersen; Georgopoulos; Kalaska; Wise; Hoshi & Tanji)

– Competition between potential actions– Various biasing factors

• attention (Goldberg; Steinmetz)• behavioral relevance

(Mountcastle; Seal & Gross)• probability (Glimcher; Shadlen)• reward (Glimcher; Olson)

attention

behavioralbiasing

potential actions

Basal gangliaBasal ganglia– Cortico-striatal-pallido-thalamo-cortical

loops (Alexander; Middleton & Strick)– Selection of actions from among

alternatives (Mink; Redgrave et al.)– Reward (Hikosaka; Schultz)

cognitivedecision-making

attention

behavioralbiasing

objectidentity

potential actions

Prefrontal cortexPrefrontal cortex– High-level decisions

based on knowledge about object identity (Fuster; Miller; Tanji…)

– Receives ventral stream information on object identity (Sakata…)


objectidentity

attention

behavioralbiasing

potential actions

ExecutionExecution– Fast visual feedback

(Prablanc; Desmurget)– Forward models

(Ito; Wolpert; Miall)

motorcommand

visual feedback

predictedfeedback


objectidentity

attention

behavioralbiasing

potential actions

motorcommand

visual feedback

predictedfeedback


objectidentity

attention

behavioralbiasing

potential actions

motorcommand

visual feedback

predictedfeedback

“Affordance competition hypothesis”“Affordance competition hypothesis”

• Continuous specification of currently available potential actionsContinuous specification of currently available potential actions

• Competition between potential action representations in fronto-Competition between potential action representations in fronto-parietal regionsparietal regions

• Biasing from frontal and subcortical areasBiasing from frontal and subcortical areas

• Decision is made through a “distributed consensus”Decision is made through a “distributed consensus”

Cisek (2007) Cisek (2007) Phil.Trans.Royal Soc. B.Phil.Trans.Royal Soc. B.

Behavior

Behavior

Actionspecification

Actionselection

Perception Cognition Action

auditionvision

planning

attention forwardmodels

inversekinematics

decisionmaking

trajectorygeneration

visionof space

objectrecognition

propositionallogic

reinforcementlearning

actionsequencing

proprioception

attention

forwardarm models

inverse armkinematics

decisionmaking

visionof nearby

space

objectrecognition

reaching

key stimulusdetection action

sequencing

propositionallogic

grasping running affect

reinforcementlearningproprio-

ception

PredictionsPredictions

• Multiple potential actions can be specified simultaneouslyMultiple potential actions can be specified simultaneously

• Biased competition between potential actionsBiased competition between potential actions

• Everything occurs in parallelEverything occurs in parallel

Neural activity specifies multiple actionsNeural activity specifies multiple actions

Classic model:Classic model:– Store information, decide,

then plan one action

Affordance competition:Affordance competition:– Specify both actions,

then select one

Cell PD

Cell PD

TimeTime

RostralRostralPMdPMd

CaudalCaudalPMdPMd

PrimaryPrimaryMotorMotorCortexCortex

Cisek & Kalaska (2005) Neuron





Biased choice taskBiased choice task

1-TARGET

CHT

2-TARGET

CHT

GO

FREE

67%

FORCED

GO

33%

DELAY

GO

THT

Reward: 1

THT

Reward: 1

DELAY

2 drops

3 drops

1 drop

THT

Reward: 3

AlexandrePastor-Bernier

Neural activity in premotor cortexNeural activity in premotor cortex

• No effect of valueNo effect of valuein 1T taskin 1T task

• However, if another However, if another target is present, then target is present, then activity activity increasesincreases with with value of preferred value of preferred targettarget

• If value of preferred If value of preferred target is constant, target is constant, activity activity decreasesdecreases with with value of other targetvalue of other target

• Activity decreases with Activity decreases with distance between distance between targetstargets

Pastor-Bernier & Cisek (2011) J. Neurosci.

Distance-dependent interactionsDistance-dependent interactions

• More activity when targets are More activity when targets are closercloser

• Compare the strength of the Compare the strength of the competition as a function of competition as a function of target distancetarget distance– As distance increases, slope is

increasingly negative

• The competition is strongest The competition is strongest between cells with the largest between cells with the largest difference in preferred directionsdifference in preferred directions

Why should it matter that distance matters?Why should it matter that distance matters?

• The distance effect suggests that the decision is madeThe distance effect suggests that the decision is madewithin the sensorimotor systemwithin the sensorimotor system– If decisions were purely cognitive (“I prefer to get 3 drops of juice

over 1 drop”), then they should be determined in an abstract space– The dynamics of the competition which determines choice depend

on the spatial relationship between the movements themselves






objectidentity

attention

behavioralbiasing

potential actions

motorcommand

visual feedback

predictedfeedback


objectidentity

attention

behavioralbiasing

potential actions

motorcommand

visual feedback

predictedfeedback

TimingTiming

• An animal is constantly interacting with the worldAn animal is constantly interacting with the world– Continuous sensorimotor control of ongoing actions– Continuous specification of alternative actions– Continuous evaluation of value– Continuous tradeoffs between persisting in a given activity or

switching to a different, currently available one

• Specification and selection must normally occur in parallelSpecification and selection must normally occur in parallel

• However, if we put the animal in the labHowever, if we put the animal in the lab– Time is broken into discrete “trials” each of which begins with a

stimulus and ends with a response– The stimulus is deliberately made independent from the response

• What should we see?What should we see?


objectidentity

attention

behavioralbiasing

potential actions

motorcommand

visual feedback

predictedfeedback

potential actions

Wave 1. Fast feedforward sweepWave 1. Fast feedforward sweep


objectidentity

attention

behavioralbiasing

potential actions

Wave 2. Attentional/Decisional modulationWave 2. Attentional/Decisional modulation

Two waves of activityTwo waves of activity

• Measured LFPs from various regions of cerebral cortexMeasured LFPs from various regions of cerebral cortex

• Monkeys performed a conditional GO / NOGO taskMonkeys performed a conditional GO / NOGO task

Ledberg et al. (2007) Cerebral Cortex

Two waves of activityTwo waves of activity

1.1. Fast feedforward sweepFast feedforward sweep• Activation in ~50ms

throughout dorsal stream and frontal cortex

2.2. Attentional/DecisionalAttentional/Decisional• About 150ms post-

stimulus, discrimination of Go/Nogo throughout the cortex

Summary 1: Experimental dataSummary 1: Experimental data

• Simultaneous specification of multiple potential actionsSimultaneous specification of multiple potential actions– Arm reaching system (PMd, PRR, M1)– Grasping system (AIP, PMv)– Saccade system (LIP, FEF, Superior colliculus)

• Biased competitionBiased competition– Potential actions compete against each other within sensorimotor

maps, influenced by a variety of biasing factors (e.g. reward)– NOTE: Similar mechanism as attention (Duncan & Desimone)

• “Attention” is selection near sensors, “decision” is selection near effectors

– Influences depend on geometry – decisions are not simply abstract• These are “pragmatic” representations, not “descriptive”

– Decision is made through a “distributed consensus”

• Parallel specification and selection systemsParallel specification and selection systems

Summary 2: Theoretical conceptsSummary 2: Theoretical concepts

• ““Affordance competition hypothesis”Affordance competition hypothesis”– Instead of serial Perception, Cognition, & Action modules, we have parallel

specification and selection systems– Better match to neural data– Better suited to the kinds of tasks that dominated animal behavior

• ““Pragmatic representations”Pragmatic representations”– Neural activity aimed not at describing the world, but at mediating

interaction with the world– Correlation with external and internal variables is necessary, but mixtures

are useful (e.g. spatial direction mixed with reward values)– Conjecture: Most, but not all, neural activity is of this kind

• “Descriptive” representations (e.g. in the ventral stream) emerged in evolution as specializations of pragmatic representations for advanced selection

• Cognitive advances evolved through hierarchical elaborationCognitive advances evolved through hierarchical elaboration– Diversification of fronto-parietal loops, cortico-striatal circuits, cortico-

cerebellar circuits, into progressively anterior/abstract systems– Interaction lays the foundation for cognition (Piaget)

Summary 3: Philosophical implicationsSummary 3: Philosophical implications

• There is no central executiveThere is no central executive– Decisions emerge through a distributed consensus

• Classic problems in a different contextClassic problems in a different context– Binding problem:

• Activity of separate streams is coherent by virtue of dealing with the same world

– Symbol grounding problem:• Interaction has meaning by virtue of influencing the variables critical for life• Symbols are specializations (“shorthand notation”) that emerged late in evolution,

already within the context of grounded interaction

– The “Hard” problem• Feeling is different than doing

– Being inside the loop is different than observing it from the outside– Private language, beetle in box, squirrel in head, 1st person perspective, the “Umwelt”

• The computer metaphorThe computer metaphor– With all due respect to Alan Turing, the computer metaphor is misleading as

a model for the brain– What matters is control (Wiener, Ashby, Powers, Gibson, Dewey, etc.)

““The great end of life is not The great end of life is not knowledge but action”knowledge but action”

– T. H. Huxley (1825-1895)

““Your head is there to move Your head is there to move you around”you around”

– R.E.M. (1980-2011)

THANK YOUTHANK YOU

• Lab membersLab members– Marie-Claude Labonté– Alexandre Pastor-Bernier– David Thura– Ignasi Cos– Matthew Carland– Jessica Trung

• AlumniAlumni– Jean-Philippe Thivierge– Thomas Michelet– Valeriya Gritsenko

FOUNDATIONTHEEJLBFOUNDATIONTHEEJLB

[email protected]

http://www.innovation.ca/index.cfm

Business

Paul Cisek Model - No "Decision" "Decision-Making"