Estimating Functional Connectomes:Sparsity’s Strength and Limitations

Gael Varoquaux Ssssssskeptical

Graphical models in cognitive neuroscience

G Varoquaux 2

Functional connectome analysis

Functional regions

Functional connections

Variations in connections

G Varoquaux 3

Functional connectome analysis

Functional regions

Functional connections

Variations in connections

G Varoquaux 3

Functional connectome analysis

Functional regions

Functional connections

Variations in connections

G Varoquaux 3

Outline

1 Estimating connectomes

2 Comparing connectomes

G Varoquaux 4

1 Estimating connectomes

Functional connectomeGraph of interactions between regions

[Varoquaux and Craddock 2013]

G Varoquaux 5

1 Graphical model in cognitive neuroscience

Whish listCausal linksDirected model:IPS = V 2 + MTFEF = IPS + ACC

Unreliable delays (HRF)Few samples

× many signalsHeteroscedastic noise

G Varoquaux 6

1 Graphical model in cognitive neuroscience

Whish listCausal linksDirected model:IPS = V 2 + MTFEF = IPS + ACC

Unreliable delays (HRF)Few samples

× many signalsHeteroscedastic noise

G Varoquaux 6

1 Graphical model in cognitive neuroscience

Whish listCausal linksDirected model:IPS = V 2 + MTFEF = IPS + ACC

Unreliable delays (HRF)Few samples

× many signalsHeteroscedastic noise

Independence structureKnowing IPS, FEF is independent of V2 and MT

G Varoquaux 6

1 From correlations to connectomes

Conditional independence structure?

G Varoquaux 7

1 Probabilistic model for interactionsSimplest data generating process

= multivariate normal:

P(X) ∝√|Σ−1|e−1

2XT Σ−1X

Model parametrized by inverse covariance matrix,K = Σ−1: conditional covariances

Goodness of fit:likelihood of observed covariance Σ in model Σ

L(Σ|K) = log |K| − trace(Σ K)

G Varoquaux 8

1 Graphical structure from correlations

ObservationsCovariance

0

1

2

3

4

Diagonal:signal variance

Direct connectionsInverse covariance

0

1

2

3

4

Diagonal:node innovation

G Varoquaux 9

1 Independence structure (Markov graph)

Zeros in partial correlationsgive conditional independence

Reflects the large-scalebrain interaction structure

Ill-posed problem:multi-collinearity⇒ noisy partial correlations

Independence between nodes makes estimationof partial correlations well-conditionned.

G Varoquaux 10

1 Independence structure (Markov graph)

Zeros in partial correlationsgive conditional independence

Ill-posed problem:multi-collinearity⇒ noisy partial correlations

Independence between nodes makes estimationof partial correlations well-conditionned.

Chicken and egg problem

G Varoquaux 10

1 Independence structure (Markov graph)

Zeros in partial correlationsgive conditional independence

Ill-posed problem:multi-collinearity⇒ noisy partial correlations

Independence between nodes makes estimationof partial correlations well-conditionned.

0

1

2

3

4

0

1

2

3

4

+Joint estimation:

Sparse inverse covariance

G Varoquaux 10

1 Sparse inverse covariance: penalization

[Friedman... 2008, Varoquaux... 2010b, Smith... 2011]

Maximum a posteriori:Fit models with a penaltySparsity ⇒ Lasso-like problem: `1 penalization

K = argminK�0L(Σ|K) +λ `1(K)

Data fit,Likelihood

Penalization,

x2

x1

G Varoquaux 11

1 Sparse inverse covariance: penalization

[Varoquaux... 2010b]

Σ−1 Sparseinverse

Likelihood of new data (cross-validation)Subject data, Σ−1 -57.1

Subject data, sparse inverse 43.0

G Varoquaux 12

1 Limitations of sparsity Sssssskeptical

Theoretical limitation to sparse recoveryNumber of samples for s edges, p nodes:n = O

((s + p) log p

)[Lam and Fan 2009]

High-degree nodes fail [Ravikumar... 2011]

EmpiricallyOptimal graphalmost dense

2.5 3.0 3.5 4.0

−log10λ

Test

-dat

a lik

eliho

od

Sparsity

[Varoquaux... 2012] Very sparse graphsdon’t fit the data

G Varoquaux 13

1 Multi-subject to overcome subject data scarsity

[Varoquaux... 2010b]

Σ−1 Sparseinverse

Sparse groupconcat

Likelihood of new data (cross-validation)Subject data, Σ−1 -57.1

Subject data, sparse inverse 43.0Group concat data, Σ−1 40.6

Group concat data, sparse inverse 41.8

Inter-subject variability

G Varoquaux 14

1 Multi-subject sparsity

[Varoquaux... 2010b]

Common independence structure but differentconnection values

{Ks} = argmin{Ks�0}

∑sL(Σs |Ks) + λ `21({Ks})

Multi-subject data fit,Likelihood

Group-lasso penalization

G Varoquaux 15

1 Multi-subject sparsity

[Varoquaux... 2010b]

Common independence structure but differentconnection values

{Ks} = argmin{Ks�0}

∑sL(Σs |Ks) + λ `21({Ks})

Multi-subject data fit,Likelihood

`1 on the connections ofthe `2 on the subjects

G Varoquaux 15

1 Multi-subject sparse graphs perform better

[Varoquaux... 2010b]

Σ−1 Sparseinverse

Populationprior

Likelihood of new data (cross-validation) sparsitySubject data, Σ−1 -57.1

Subject data, sparse inverse 43.0 60% fullGroup concat data, Σ−1 40.6

Group concat data, sparse inverse 41.8 80% fullGroup sparse model 45.6 20% full

G Varoquaux 16

1 Independence structure of brain activity

Subject-sparseestimate

G Varoquaux 17

1 Independence structure of brain activity

Population-sparse estimate

G Varoquaux 17

1 Large scale organization: communitiesGraph communities

[Eguiluz... 2005]

Non-sparse

Neural communities

G Varoquaux 18

1 Large scale organization: communitiesGraph communities

[Eguiluz... 2005]

Group-sparse

Neural communities= large known functional networks [Varoquaux... 2010b]

G Varoquaux 18

1 Giving up on sparsity?

Sparsity is finickySensitive hyper-parameterSlow and unreliable convergenceUnstable set of selected edges

ShrinkageSoftly push partial correlations to zero

ΣShrunk = (1− λ)ΣMLE + λId

Ledoit-Wolf oracle to set λ[Ledoit and Wolf 2004]

G Varoquaux 19

2 Comparing connectomesFunctional biomarkersPopulation imaging

G Varoquaux 20

2 Failure of univariate approach on correlations

Subject variability spread across correlation matrices

0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25Large lesion

dΣ = Σ2 −Σ1 is not definite positive⇒ not a covariance

Σ does not live in a vector space

G Varoquaux 21

2 Inverse covariance very noisy

Partial correlations are hard to estimate

0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25 Control0 5 10 15 20 25

0

5

10

15

20

25Large lesion

G Varoquaux 22

2 A toy model of differences in connectivityTwo processes with different partial correlations

K1: K1 −K2: Σ1: Σ1 −Σ2:

+ jitter in observed covarianceMSE(K1 −K2): MSE(Σ1 −Σ2):

Non-local effects and non homogeneous noiseG Varoquaux 23

2 Theory: error geometryDisentangle parameters (edge-level connectivities)Connectivity matrices form a manifold

⇒ project to tangent space

θ¹

θ²( )θ¹I -1

( )θ²I -1

Estimation error of covariancesAssymptotics given by Fisher matrix [Rao 1945]

Cramer-Rao bounds

G Varoquaux 24

2 Theory: error geometryDisentangle parameters (edge-level connectivities)Connectivity matrices form a manifold

⇒ project to tangent space

Manifold

[Varoquaux... 2010a]

Estimation error of covariancesAssymptotics given by Fisher matrix [Rao 1945]Defines a metric on a manifold of modelsWith covariances: Lie-algebra structure [Lenglet... 2006]

G Varoquaux 24

2 Reparametrization for uniform error geometryDisentangle parameters (edge-level connectivities)Connectivity matrices form a manifold

⇒ project to tangent space

Controls

Patient

dΣ

Manifold

Tangent

dΣ = Σ−1/2

Ctrl ΣPatientΣ−1/2

Ctrl

[Varoquaux... 2010a]

G Varoquaux 24

2 Reparametrization for uniform error geometry

The simulationsK1 −K2: Σ1 −Σ2: dΣ: MSE(dΣ):

Semi-local effects and homogeneous noise

G Varoquaux 25

2 ResidualsCorrelation matrices: Σ -1.0 0.0 1.0

0 5 10 15 20 25

0

5

10

15

20

25

0 5 10 15 20 25

0

5

10

15

20

25

0 5 10 15 20 25

0

5

10

15

20

25

0 5 10 15 20 25

0

5

10

15

20

25

Residuals: dΣ -1.0 0.0 1.0

0 5 10 15 20 25

0

5

10

15

20

25

Control 0 5 10 15 20 25

0

5

10

15

20

25

Control 0 5 10 15 20 25

0

5

10

15

20

25

Control 0 5 10 15 20 25

0

5

10

15

20

25

Large lesionG Varoquaux 26

2 Post-stroke covariance modifications

p-value: 5·10−2

Bonferroni-correctedG Varoquaux 27

2 Prediction from connectomes

RS-fMRI

Functionalconnectivity

Time series

24

3

1

Diagnosis

ROIs

Connectivity matrixCorrelationPartial correlationsTangent space

Prediction accuracy

Autism[Abraham2016]

[K. Reddy, Poster 3916]

G Varoquaux 28

2 Prediction from connectomes

Time series

2

RS-fMRI

41

Diagnosis

ROIs Functionalconnectivity

3

Connectivity matrixCorrelationPartial correlationsTangent space

Prediction accuracy

Autism[Abraham2016]

[K. Reddy, Poster 3916]

G Varoquaux 28

2 Prediction from connectomes

Time series

2

RS-fMRI

41

Diagnosis

ROIs Functionalconnectivity

3

Connectivity matrixCorrelationPartial correlationsTangent space

Prediction accuracy

Autism[Abraham2016]

[K. Reddy, Poster 3916]G Varoquaux 28

@GaelVaroquaux

Estimation functional connectomes:sparsity and beyond

Zeros in inverse covariance giveconditional independance

⇒ sparsityShrinkage: simpler, faster

(Ledoit-Wolf)

Tangent spacefor comparisons

ControlsPatient

Controls

Patient

Software:http://nilearn.github.io/ ni

References I

V. M. Eguiluz, D. R. Chialvo, G. A. Cecchi, M. Baliki, andA. V. Apkarian. Scale-free brain functional networks.Physical review letters, 94:018102, 2005.

J. Friedman, T. Hastie, and R. Tibshirani. Sparse inversecovariance estimation with the graphical lasso. Biostatistics,9:432, 2008.

C. Lam and J. Fan. Sparsistency and rates of convergence inlarge covariance matrix estimation. Annals of statistics, 37(6B):4254, 2009.

O. Ledoit and M. Wolf. A well-conditioned estimator forlarge-dimensional covariance matrices. J. Multivar. Anal.,88:365, 2004.

References IIC. Lenglet, M. Rousson, R. Deriche, and O. Faugeras.

Statistics on the manifold of multivariate normaldistributions: Theory and application to diffusion tensorMRI processing. Journal of Mathematical Imaging andVision, 25:423, 2006.

C. Rao. Information and accuracy attainable in the estimationof statistical parameters. Bull. Calcutta Math. Soc., 37:81,1945.

P. Ravikumar, M. J. Wainwright, G. Raskutti, B. Yu, ...High-dimensional covariance estimation by minimizing`1-penalized log-determinant divergence. Electronic Journalof Statistics, 5:935–980, 2011.

S. Smith, K. Miller, G. Salimi-Khorshidi, M. Webster,C. Beckmann, T. Nichols, J. Ramsey, and M. Woolrich.Network modelling methods for fMRI. Neuroimage, 54:875,2011.

References IIIG. Varoquaux and R. C. Craddock. Learning and comparing

functional connectomes across subjects. NeuroImage, 80:405, 2013.

G. Varoquaux, F. Baronnet, A. Kleinschmidt, P. Fillard, andB. Thirion. Detection of brain functional-connectivitydifference in post-stroke patients using group-levelcovariance modeling. In MICCAI. 2010a.

G. Varoquaux, A. Gramfort, J. B. Poline, and B. Thirion.Brain covariance selection: better individual functionalconnectivity models using population prior. In NIPS. 2010b.

G. Varoquaux, A. Gramfort, J. B. Poline, and B. Thirion.Markov models for fMRI correlation structure: is brainfunctional connectivity small world, or decomposable intonetworks? Journal of Physiology - Paris, 106:212, 2012.