Reading in the brain 2. Masking, subliminal reading, and ... · Our research strategy - The contrastive method: « … contrasting pairs of similar events, where ... SAME LOCATION

Reading in the brain

2. Masking, subliminal reading, and the mechanisms of

conscious access

Stanislas Dehaene

Collège de France,and

INSERM-CEA Cognitive Neuroimaging Unit

NeuroSpin Center, Saclay, Francewww.unicog.org

Non-conscious

Conscious

Image by Claire Sergent

The visual word form area

A subpart of the occipito-temporal visual system that systematicallyresponds to writtenwords.

A hierarchicallyorganized region

A short demonstration of digit masking

Two central issues in consciousness research

1. The depth of subliminal processing- How deeply can subliminal stimuli be processed into the visual system?

2. The nature of conscious access- What prevents subliminal stimuli from becoming conscious?- What processes occur when a stimulus is consciously accessed, and

when do conscious and non-conscious processing diverge?

Our research strategy- The contrastive method: « … contrasting pairs of similar events, where

one is conscious but the other is not. » (Baars, 1989)- The primacy of the subjective: « …the first crucial step is to take

seriously introspective phenomenological reports. (…) They constitute primary data that need to be measured and recorded along with other psychophysiological observations » (Dehaene & Naccache, 2001)

In summary, look for:objective neurobiological correlates of subjective processes

Part I. The fate of a non-conscious stimulus

• Masked words and digits elicit activity at a series of hierarchical levels, including visual, semantic and motorstages.

• fMRI priming can help decipher how information is codedin a given region.

time

RADIO29 ms

29 ms

271 ms

29 ms

Making a word invisible: the masking paradigm

Dehaene et al., Nature Neuroscience, 2001

Invisible word

Time

radio

RADIO29 ms

29 ms

271 ms

500 ms

29 ms

Left fusiform(-44, -52, -20)

SameCase

DifferentCase

595

600

605

610

615

620

625

Res

pons

e tim

e (m

s)

Same wordDifferent word

Behavioral priming

Same wordDifferent word

0

0.1

SameCase

DifferentCase

Act

ivat

ion

(%)

C

fMRI priming

0

0.1

similar dissimilar

Left fusiform-48, -52, -12

Perc

ent s

igna

l cha

nge

Case-invariant priming independent of letter

similarity

e.g. COUP-coup vs RAGE-rage

Dehaene et al, Nature Neuroscience, 2001; Psychological Science, 2004Invariance for case in the visual word form area

Single letter?

#REFLET#reflet

TREFLE##reflet

#TREFLE#reflet

anagram

REFLET##reflet

#PATERE#reflet

EPATER##reflet

same word different word

same

different

Prime-Target Relation

wordlocation

RADIOradio

What are the coding units underlying orthographic priming?

Bigram?

Morpheme or whole word?

Dehaene et al., Psychological Science, 2004

#REFLET#reflet

REFLET##reflet

#TREFLE#reflet

TREFLE##reflet

#PATERE#reflet

EPATER##reflet

same word anagram different word

same

different


wordlocations

595

600

605

610

615

620

625

630

sameword

anagram differentword

resp

onse

tim

e (m

s)

same locationdifferent location

Behavior


#REFLET#reflet

REFLET##reflet

#TREFLE#reflet

TREFLE##reflet

#PATERE#reflet

EPATER##reflet


same

different


wordlocations

LETTERS AT THE SAME LOCATION

same locationdifferent location

both locations

0

0.1

0

0.1

Left posterior fusiform right posterior fusiform

sameword

anag diffword

sameword

anag diffword

fMRI : location-specific priming


5

0

0.1

0.2

Left middle fusiform (y=-56)

sameword

anag diffword

#REFLET#reflet

REFLET##reflet

#TREFLE#reflet

TREFLE##reflet

#PATERE#reflet

EPATER##reflet


same

different


wordlocations

LETTERS AT ANY LOCATION LARGER UNIT (WHOLE WORD?)

fMRI : location-independent priming

0

0.1

Left middle fusiform (y=-48)

sameword

anag diffword


Coded units

Local bigrams

Small words and recurring

substrings (e.g. morphemes)

Leftoccipito-temporal

sulcus?(y ≈ -56)

Left occipito-temporal

sulcus?(y ≈ -48)

Putative area

Bank of abstract letter

detectors

Bilateral V8?(y ≈ -64)

Oriented bars

Local contrasts

Local contours(letter fragments)

Letter shapes(case-specific)

BilateralLGN

BilateralV1

BilateralV2

Bilateral V4?

Receptive fieldsize and structure

+- -

Examples of preferred stimuli

En

extentCONTENT

E

Local combination detectors: A model of invariant visual word recognition

Dehaene et al.TICS, 2005

Priming within and across scriptsin Japanese subjects

Design:• Targets and primes can appear in Kanji or in Kana• Task = semantic classification (natural/man-made)

Nakamura, Dehaene et al., JOCN, 2005

Repetition priming in Japanese

Within and cross-script priming in response times

Repetition priming with Kanji primes and Kanji targets

Nakamura, Dehaene et al., JOCN, 2005

VWFA: Script-specific priming

Left middle temporal region:Cross-script priming (semantic?)

Visual word form area

bilateral intraparietal sulci

Motor lateralized readiness potential

Left middle temporal gyrus

Kanji-Kana

Sofa-couch

Evidence for extensive subliminal processing using fMRI priming

Orthographic priming Left fusiform gyrus (Dehaene et al,

2001; Devlin et al, 2004)Semantic priming

Numerical proximity in bilateralintraparietal sulci (Naccache and Dehaene, 2001)

Semantic proximity of words in leftmiddle temporal gyrus (Devlin et al, 2004; Nakamura, Dehaene et al, 2005)

Amygdala activation by maskedemotional words (Naccache et al, 2005)

Motor primingBilateral motor areas (Dehaene et

al., 1998)

Part II. The nature of conscious access

• Conscious access is associated with a sudden activation of a distributed parieto-prefrontal network

time

RADIO29 ms

Making a word visible again


71 ms

71 ms

71 ms

71 ms

time

RADIO29 ms

71 ms

71 ms

71 ms

Making a word visible again


71 ms

visible words masked words

2.26 3.33t scale

p value0.02 0.0025

x = -38

29 17

6.3 20.8t scale

p value10-5 3.10-12

x = -38

29 5 1745

left fusiform gyrus(-48, -60, -12)

-0.1

0.3

-5 0 5 10 15

time (s)

% s

igna

l cha

nge

visible wordsmasked words

Consciousness correlates with-Local amplification in the relevant processors

-Global parieto-frontal activity (Dehaene et al, Nature Neuroscience 2001)

Baar’s (1989) theory of a conscious global workspace:

An architecture mixing parallel and serial processing

…the intellectual activity, the will and self-consciousness…are the result of a combined action of a large number of mnemonic spheres

(Ramon y CAJAL, Histologie, p.878)

What is the neural basis of the global workspace?

Prefrontal cortex and temporo-parietal association areas form long-distance networks

V1

TE

PFC

Pat Goldman-Rakic(1980s):

long-distance connectivity of dorso-

lateral prefrontal cortex

Guy Elston (2000)Greater arborizations and spine density

Von Economo (1929):Greater layer II/III thickness

processorsmobilizedinto the

consciousworkspace

hierarchy of modularprocessors

The global neuronal workspace model

Dehaene, Kerszberg & Changeux, PNAS, 1998Dehaene & Changeux, PNAS, 2003; PLOS, 2005inspired by Mesulam, Brain, 1998

automaticallyactivated

processors

high-level processorswith strong

long-distanceinterconnectivity

Dehaene, Sergent, & Changeux, PNAS, 2003

Detailed simulations of the global neuronal workspaceusing a semi-realistic network of spiking neurons

(Dehaene et al., PNAS 2003, PLOS Biology, 2005)

T1 T2

Area A1

Area B1

Area C

Area D

Area A2

Area B2

Feedforward

Feedback

Thalamocortical column

to/fromhigherareas

to/fromlowerareas

Thalamus

Cortex

supragranular

layer IV

infragranular

Spiking neurons

NMDA

AMPAGABA

NaP

KS

Spike-rateadaptation

Leak

neuromodulatorcurrent

Optional cellular oscillator currents

0 100 200 300 400 5000

20

40

60

« Ignition » of the global

workspace by target T1

Failure of ignition by target T2

Three distinguishable states of subliminal, preconscious and conscious processing

Dehaene, Changeux, Naccache, Sackur, & Sergent, TICS, 2006²

GlobalWorkspace

T1Conscious

high strengthand attention

T2Preconscioushigh strengthno attention

T3Subliminal

weak strength

T1 versus T3 : unmasked versus masked stimuli(both attended)

Unmasked words (T1) Masked words (T3)

GlobalWorkspace

T1Conscious


T3Subliminal

weak strength

Contrast betweenconscious and subliminal processing

16ms

Delay: 0-150ms

prime

mask+

.

M6MEE

+

..

.

+6.

. .

OBJECTIVE JUDGEMENT

Compare prime to 5

SUBJECTIVE JUDGEMENT

Rate Prime

Visibility

Not seenMaximalvisibility

250 ms

Time

DIGIT MASKING PARADIGM

Antoine Delcul

16ms

Delay: 0-150ms

prime

mask+

.

M6MEE

+..

.

+6.

. .

Time

Exploring the cerebral mechanismsof the non-linear threshold in conscious access

(Delcul and Dehaene, PLOS Biology 2007)

Logic = Use this sigmoidal profile as a « signature » of consciousaccess. Which ERP components show this profile?

Subjective visibility Objective performance

% of responsedelay

1

3

5

7

9 11

13

15

17

19

21

0ms

16m

s

33m

s

50m

s

66m

s

83m

s

100m

s

150m

s

0

10

20

30

40

50

60

70

80

90

100

Subjective ratingDelay

subliminal processing conscious processing

at threshold

Early visual areas Higher visual areas Prefrontal areas

subliminal

conscious

weak masking

strong masking

Masking strength

time following stimulus onset (ms)

Predictions of the global neuronal workspace model

-100 0 100 200-2

-1

0

1

2

3

4

5mask subtracted, aligned on mask onset

N1

P1a

100 ms

33 ms50 ms66 ms83 ms

16 ms

C

Apparent P2after

mask subtraction

Time (ms)

Ampl

itude

(μV)

The Mask-SubtractionMethod

Separation of activation evoked by the mask and by the target

-100 0 100 200-2

-1

0

1

2

3

4raw data, aligned on mask onset

100 ms

maskonly


16 ms

A

Alignement on

target onset

Mask subtraction

Mask-evoked N1

Time (ms)

Ampl

itude

(μV)

0 100 200 300-2

-1

0

1

2

3

4

5mask subtracted, aligned on target onset

N1

P1a

D

Time (ms)

Ampl

itude

(μV)

Apparent P2after

mask subtraction

Conclusions:-Masking leaves the P1a and N1 essentially intact (exceptat the shortest SOA)-Masking reduces the P1b (less inter-hemispherictransfer)-Masking interrupts the N2-As the masking delayincreases

-target activation become increasinglystronger and longer lasting-Mask activation decreasescorrespondingly

- Only the P3 shows a non-linearity similar to behavioralreport

Brain mechanisms of masking

m-N2

m-N1

180 ms 0 50 100

180

200

220

240

260

280

16 33 50 66 831000

1

2

3

4

5

Latency

P1a

P1b

N1

N2

P3

100ms

140ms

167ms

227ms

370ms

Left target

-6-5-4-3-2-1012345μV

Amplitude

0 50 100

20

40

60

80

100

120

16 33 50 66 831000

0.5

1

1.5

0 50 100

50

100

150

16 33 50 66 831000

0.5

1

1.5

2

0 50 100

140

160

180

200

220

240

16 33 50 66 831000

1

2

3

4

0 50 100

300

350

400

16 33 50 66 831000

1

2

3

4

5

Target evoked components

Mask evoked components

0 50 100

200

250

300

16 33 50 66 831000

1

2

3

4

200 ms

Right target

0 50 10060

80

100

120

140

160

180

16 33 50 66 831000

0.5

1

1.5

2

0 50 100

180200

220240

260280

16 33 50 66 83100

01

23

45

0 50 100

200250

300

16 33 50 66 83100

01

23

4

100 200 300 4000

2

4

6

8raw data, aligned on target onset

P3: sudden non linear divergencearound 270 ms

Amplitude (microV)

100 200 300 400-10

-8

-6

-4

-2

0raw data, aligned on target onsetFronto-polar electrodes

Suddenonset

100 ms

maskonly


16 ms

Central electrodes

Suddenonset

370 ms

100 200 300 400-1

0

1

2

3

4

5

6raw data, aligned on target onset

maskonly

seennot seen

Central electrodes

t-test at t=341 ms post target

+4

0

-4

t value

A second independent criterion for conscious access:Only the amplitude of the P3 distinguishes

seen versus not-seen trials at a fixed delay (50 ms)(9 subjects only)

Note, however, that the unseenstimuli evoke strong activation atSOA=50 ms.

All components showed a significant difference:Not-seen > mask-only

EM M

E

A late non-linearity underlying conscious access during masking(Delcul et Dehaene, PLOS Biology 2007)

100 ms


16 ms

First phase: local and linear

9

Variable delay

16 ms

250 ms

Delay:

Second phase: global and non-linear

(amplification)

Parietal

FusiformFrontal

Distributed visual, parietaland prefrontal activity atthe peak of the P3 (conscious trials)

Test of the theory2. with two competing stimuli during AB

GlobalWorkspace

T1Conscious


T2Preconscioushigh strengthno attention

T3Subliminal

weak strength

Conscious access and non-conscious processingduring the attentional blink

Sergent, Baillet & Dehaene, Nature Neuroscience, 2005

02 4 6 8 10 12 14 16 18 20

1 2 3 4 6 8

0

10

20

30

40

50

60

70

80

90

Perc

ent o

f tria

ls

Subjective visibility

Lag

Bimodal distribution of T2 visibility

Visibility ratingT1-T2 Lag

Seen

Not seen

HRQFCINQ

Target 2

time

Variable T1-T2 lag

Target1

CVGRXOOX

Sergent et al., Nature Neuroscience, 2005

UNSEEN T2(minus T2-absent trials) DIFFERENCE

Time course of scalp-recorded potentialsduring the attentional blink

SEEN T2(minus T2-absent trials)

Timing the divergence between seen and not-seen trialsin the attentional blink (Sergent et al., Nature Neuroscience 2005)

Seen

Not seen

Unchanged initial processingP1 : 96 ms

-2 μV +2 μV

N1 : 180 ms

-4 μV +4 μV

Abrupt divergence around 270 ms…

Seen

Not seen

N2 : 276 ms

-2 μV +2 μV

N3 : 300 ms

-3 μV +3 μV

Late non-conscious processingN4 : 348 ms

-3 μV +3 μV

All-or-none ignition

P3a : 436 ms

-2 μV +2 μV

P3b : 576 ms

-2 μV +2 μV

Dorsolateral prefrontal

Differencebetween seen and

not seen trials

436 ms

Middle temporal

Inferior frontal

Anterior cingulate

seennot seen

t = 300 ms

t = 436 ms

t = 436 ms

Recording of ‘virtualsources’ at variouscortical locations

fMRI:Marois et al, Neuron 2004

Inferior temporal

MEG:Gross et al, PNAS 2004

Target T1

Target T2

Response 1

Response 2

Test of the theory3. with two competing stimuli during PRP

Sigman and Dehaene, PLOS Biology, 2005

Pashler’s « central bottleneck » model of the psychological refractory period

T1 T2

Workspace state for task 1

R2

R1

Workspace state for task 2

Pashler’s central bottleneck model

Response 1

Response 2

Target 1

Target 2

time

Task 1

Task 2

0

InterferenceRegime

IndependentRegime

T1-T2 SOA

Event-related potentials dissociate parallel and serial stages during dual-task processing

Sigman and Dehaene, submitted

-1000 -500 0 500 1000 1500 20000

0.5

1

1.5

2

2.5

V A

N1 P3 N1 P3

Visual events Auditory events

Separation of ERPs at long T1-T2 delays

0 300 900 1200

600

1000

1400

1800

RT2

RT1

Response times

T1-T2 delay

Task 1: number comparison of a visual Arabic numeral with45, respond with right hand

Task 2: pitch judgment on an auditory tone, respond with lefthand

- 300 ms delay in or outside of the interference regime

-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1

-0 .5

0

0 .5

1

1 .5

P3

Event-related potentials dissociate parallel and serial stages during dual-task processing

N1

-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1

-0 .5

0

0 .5

1

1 .5

P3-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0

-0 .5

0

0 .5

1

1 .5

-5 0 0 0 5 0 0 1 0 0 0 1 5 0 0-1

-0 .5

0

0 .5

1

1 .5

N1


-1000 -500 0 500 1000 1500 20000

0.5

1

1.5

2

2.5

V A

N1 P3 N1 P3

Visual events Auditory events

Separation of ERPs at long T1-T2 delays

0 300 900 1200

600

1000

1400

1800

RT2

RT1

Response times

T1-T2 delay




Functional MRI can be sensitive to small delays, in the order of a few hundreds of milliseconds

Sigman, M., Jobert, A., LeBihan, D., & Dehaene, S. (2006). Parsing a sequence of brainactivations at psychological times using fMRI. NeuroImage, 2006

•The phase of the BOLD response can beestimated with high precision even when the TR exceeds 2 seconds• Injected delays of ~200 ms can bedistinguished with whole-brain imaging• Changes in onset, duration or both can beseparated

1 2 3 46

6.5

7

7.5

8

8.5

Pha

se(s

ec)

Injected delay (multiples of 500 ms)

Slope=0.512 sMenon et al, 1998

Separating parallel and serial stages during dual-taskprocessing with high temporal resolution fMRI





- slow fMRI (one trial = 14.4 s)

z=54 z=6 z=-6 0 1.2

5.2

6.7

0 1.2

4.4

5.9

z=54 0 1.2

5

6.5

z=54 z=6z=30 0 1.2

4.9

6.4

z=46z=34z=20 0 1.2

6

Injected T1-T1 delay (seconds)

Measured BOLD phase (seconds)

Task 1network

Task 2perceptual

Task 2Decision/motor

Shared betweentask 1 and task 2

High-level control

PRP

VSTM/MOT

Att. Blink

Marois & Ivanoff, TICS, 2005review of imaging studies of bottleneck tasks

Locating the sites of processing bottlenecks:parieto-prefrontal networks

Dux, Ivanoff, Asplund & Marois, Neuron, 2007

Conclusion: Towards a neuronal understandingof consciousness

Non-conscious processing is extensive in the human brainBrain activity can remain non-conscious for at least two reasons:– bottom-up strength is insufficient (e.g. masking)– Top-down attention is distracted (e.g. attentional blink)

A representation becomes conscious whenever it wins the central competition and ignites a distributed, self-sustained assembly of neurons in prefrontal, cingulate and other cortical association areasConscious access corresponds to a sharp and relatively late (~270 ms) dynamical phase transition in neural network activity. Although I have only talked about access to consciousness, changes in viligance (intransitive consciousness) relate to neuromodulation of the same network (S. Laureys, P. Maquet).

Vegetative state Coma Slow-wave sleep General anesthesia

Documents

Reading in the brain 2. Masking, subliminal reading, and ... · Our research strategy - The contrastive method: « … contrasting pairs of similar events, where ... SAME LOCATION