Keith Worsley, Math + Stats, Arnaud Charil, Montreal Neurological Institute, McGill

Detecting connectivity between images: MS lesions, cortical thickness, and the 'bubbles'

task in an fMRI experiment

Keith Worsley, Math + Stats, Arnaud Charil, Montreal Neurological

Institute, McGillPhilippe Schyns, Fraser Smith,

Psychology, GlasgowJonathan Taylor,

Stanford and Université de Montréal

What is ‘bubbles’?

Nature (2005)

Subject is shown one of 40 faces chosen at random …

Happy

Sad

Fearful

Neutral

… but face is only revealed through random ‘bubbles’

First trial: “Sad” expression

Subject is asked the expression: “Neutral”

Response: Incorrect

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sad75 random

bubble centresSmoothed by a

Gaussian ‘bubble’What the

subject sees

Your turn …

Trial 2

Subject response:

“Fearful”

CORRECT

Your turn …

Trial 3

Subject response:

“Happy”

INCORRECT(Fearful)

Your turn …

Trial 4

Subject response:

“Happy”

CORRECT

Your turn …

Trial 5

Subject response:

“Fearful”

CORRECT

Your turn …

Trial 6

Subject response:

“Sad”

CORRECT

Your turn …

Trial 7

Subject response:

“Happy”

CORRECT

Your turn …

Trial 8

Subject response:

“Neutral”

CORRECT

Your turn …

Trial 9

Subject response:

“Happy”

CORRECT

Your turn …

Trial 3000

Subject response:

“Happy”

INCORRECT(Fearful)

0

0.5

1

100

200

300

50100

150

200250

0.65

0.7

0.75

0

0.5

1

Bubbles analysis E.g. Fearful (3000/4=750 trials):

Trial1 + 2 + 3 + 4 + 5 + 6 + 7 + … + 750 = Sum

Correcttrials

Proportion of correct bubbles=(sum correct bubbles)

/(sum all bubbles)

Thresholded atproportion of

correct trials=0.68,scaled to [0,1]

Use thisas a bubblemask

Results

Mask average face

But are these features real or just noise? Need statistics …

Happy Sad Fearful Neutral

0.65

0.7

0.75

-2

0

2

4

0

0.5

1

Statistical analysis Correlate bubbles with response (correct = 1,

incorrect = 0), separately for each expression Equivalent to 2-sample Z-statistic for correct

vs. incorrect bubbles, e.g. Fearful:

Very similar to the proportion of correct bubbles:

Response0 1 1 0 1 1 1 … 1

Trial 1 2 3 4 5 6 7 … 750Z~N(0,1)statistic

0.65

0.7

0.75

-2

0

2

4

Both depend on average correct bubbles, rest is ~ constant

Comparison

Proportion correct bubbles= Average correct bubbles / (average all bubbles * 4)

Z=(Average correct bubbles -average incorrect bubbles)

/ pooled sd

1.64

2.13

2.62

3.11

3.6

4.09

4.58

Results

Thresholded at Z=1.64 (P=0.05)

Multiple comparisons correction? Need random field theory …

Average faceHappy Sad Fearful Neutral

Z~N(0,1)statistic

-4 -3 -2 -1 0 1 2 3 4-20

-10

0

10

20

30

Threshold

Eul

er C

hara

cter

istic

Observed

Expected

Euler Characteristic = #blobs - #holesExcursion set {Z > threshold} for neutral face

Heuristic:At high thresholds t,the holes disappear,

EC ~ 1 or 0, E(EC) ~ P(max Z > t).

• Exact expression for E(EC) for all thresholds,• E(EC) ~ P(max Z > t) is extremely accurate.

EC = 0 0 -7 -11 13 14 9 1 0

The details …

2

S

Tube(S,r)r

3

A B

6

S

TubeΛ(S,r)

r

Λ is small

Λ is big

S S s1

s2 s3

U(s1)

U(s2)U(s3)

Tube Tube

2 ν

R

Tube(R,r)r

N2(0,I)Z1

Z2

Tube(R,r)

t-r t

z

Tube(R,r)

R

z1

z2

z3

R

R

r

Summary

Random field theory results

For searching in D (=2) dimensions, P-value of max Z is (Adler, 1981; W, 1995): P(max Z > z)

~ E( Euler characteristic of thresholded set ) = Resels × Euler characteristic density (+ boundary)

Resels (=Lipschitz-Killing curvature/c) is Image area / (bubble FWHM)2 = 146.2

Euler characteristic density(×c) is (4 log(2))D/2 zD-1 exp(-z2/2) / (2π)(D+1)/2

See forthcoming book Adler, Taylor (2007)

3.92

4.03

4.14

4.25

4.36

4.47

4.58

Results, corrected for search

Thresholded at Z=3.92 (P=0.05)Average face

Happy Sad Fearful Neutral

Z~N(0,1)statistic

0

10000

0

0.5

1

Bubbles task in fMRI scanner Correlate bubbles with BOLD at every voxel:

Calculate Z for each pair (bubble pixel, fMRI voxel) – a 5D “image” of Z statistics …

Trial1 2 3 4 5 6 7 … 3000

fMRI

Discussion: thresholding

Thresholding in advance is vital, since we cannot store all the ~1 billion 5D Z values Resels=(image resels = 146.2) × (fMRI resels = 1057.2) for P=0.05, threshold is Z = 6.22 (approx) The threshold based on Gaussian RFT can be improved

using new non-Gaussian RFT based on saddle-point approximations (Chamandy et al., 2006) Model the bubbles as a smoothed Poisson point

process The improved thresholds are slightly lower, so more

activation is detected Only keep 5D local maxima

Z(pixel, voxel) > Z(pixel, 6 neighbours of voxel) > Z(4 neighbours of pixel, voxel)

Discussion: modeling The random response is Y=1 (correct) or 0 (incorrect), or Y=fMRI The regressors are Xj=bubble mask at pixel j, j=1 … 240x380=91200 (!) Logistic regression or ordinary regression:

logit(E(Y)) or E(Y) = b0+X1b1+…+X91200b91200

But there are only n=3000 observations (trials) … Instead, since regressors are independent, fit them one at a time:

logit(E(Y)) or E(Y) = b0+Xjbj

However the regressors (bubbles) are random with a simple known distribution, so turn the problem around and condition on Y: E(Xj) = c0+Ycj

Equivalent to conditional logistic regression (Cox, 1962) which gives exact inference for b1 conditional on sufficient statistics for b0

Cox also suggested using saddle-point approximations to improve accuracy of inference …

Interactions? logit(E(Y)) or E(Y)=b0+X1b1+…+X91200b91200+X1X2b1,2+ …

MS lesions and cortical thickness

Idea: MS lesions interrupt neuronal signals, causing thinning in down-stream cortex

Data: n = 425 mild MS patients Lesion density, smoothed 10mm Cortical thickness, smoothed 20mm Find connectivity i.e. find voxels in 3D, nodes

in 2D with high correlation(lesion density, cortical thickness)

Look for high negative correlations …

0 10 20 30 40 50 60 70 80

1.5

2

2.5

3

3.5

4

4.5

5

5.5

Average lesion volume

Ave

rag

e co

rtic

al t

hic

kne

ssn=425 subjects, correlation = -0.568

Thresholding? Cross correlation random field

Correlation between 2 fields at 2 different locations, searched over all pairs of locations one in R (D dimensions), one in S (E dimensions) sample size n

MS lesion data: P=0.05, c=0.325Cao & Worsley, Annals of Applied Probability (1999)

Normalization

LD=lesion density, CT=cortical thickness Simple correlation:

Cor( LD, CT )

Subtracting global mean thickness: Cor( LD, CT – avsurf(CT) )

And removing overall lesion effect: Cor( LD – avWM(LD), CT – avsurf(CT) )

0

0.5

1

1.5

2

2.5x 10

5

corr

elat

ion

Same hemisphere

0 50 100 150-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0

0.2

0.4

0.6

0.8

1

distance (mm)

corr

elat

ion

0 50 100 150-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0

0.5

1

1.5

2

2.5

x 105

corr

elat

ion

Different hemisphere

0 50 100 150-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0

0.2

0.4

0.6

0.8

1

distance (mm)

corr

elat

ion

0 50 100 150-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

threshold

thresholdthreshold

threshold

Histogram

‘Conditional’ histogram: scaled to same max at each distance

Scien

ce (2004)

fMRI activation detected by correlation between subjects at the same voxel

The average nonselective time course across all activated regions obtained during the first 10 min of the movie for all five subjects. Red line represents the across subject average time course. There is a striking degree of synchronization among different individuals watching the same movie.

Voxel-by-voxel intersubject correlation between the source subject (ZO) and the target subject (SN). Correlation maps are shown on

unfolded left and right hemispheres (LH and RH, respectively). Color indicates the significance level of the intersubject correlation

in each voxel. Black dotted lines denote borders of retinotopic visual areas V1, V2, V3, VP, V3A, V4/V8, and estimated border of

auditory cortex (A1).The face-, object-, and building-related borders (red, blue, and green rings, respectively) are also superimposed on the map. Note the substantial extent of

intersubject correlations and the extension of the correlations beyond visual and auditory cortices.

What are the subjects watching during high activation? Faces …

… buildings …

… hands

Thresholding? Homologous correlation random field Correlation between 2 equally smooth fields at the same

location, searched over all locations in R (in D dimensions)

P-values are larger than for the usual correlation field (correlation between a field and a scalar) E.g. resels=1000, df=100, threshold=5, usual P=0.051,

homologous P=0.139

Cao & Worsley, Annals of Applied Probability (1999)

Detecting Connectivity between Images: the

'Bubbles' Task in fMRI

Keith Worsley, McGill

Phillipe Schyns, Fraser Smith, Glasgow

Subject is shown one of 40 faces chosen at random …

Happy

Sad

Fearful

Neutral

… but face is only revealed through random ‘bubbles’ E.g. first trial: “Sad” expression:

Subject is asked the expression: “Neutral”

Response: Incorrect=0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sad75 random

bubble centresSmoothed by a

Gaussian ‘bubble’What the

subject sees