Population Inference post Model Selection in Neurosciencekuffner/AllenSlides.pdf · Motivating Case Study: Neuron Tuning Model Selection (Lasso) used to assess neuron tuning. Challenge:

Population Inference post Model Selection inNeuroscience

Genevera I. Allen

Dobelman Family Junior Chair,Department of Statistics and Electrical and Computer Engineering, Rice University,

Department of Pediatrics-Neurology, Baylor College of Medicine,Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital.

October 2, 2016

G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, 2016 1 / 18

Functional Neuroimaging & Neural Recordings

Micro-Scale (Neurons):

Calcium-FlorescenceImaging

Electrophysiology

Microscopy

Macro-Scale (Brain):

Functional MRI (fMRI)

EEG / MEG

Electrocorticography


Functional Neuroimaging & Neural RecordingsData:

For each subject i , Xi is p×T for p brain regions (or neurons) and Ttime points.

Subject-level data summarized via Model Selection.


Functional Neuroimaging & Neural Recordings

Objective: Population Inference.


1 Motivating Case Studies

2 PIMS

3 Empirical Studies


Motivating Case Study: Neuron Tuning

Calcium Florescence Imaging:



Study Design:

WT vs. MECP2 mice (two genotypes).

Motor Cortex imaged once a week for 8 weeks.

Mice ran on treadmill at 5 different speeds during imaging.

Neuron Tuning:

Neurons who fire more during certain stimuli (tuned to stimuli).

Objective:

Compare the proportion and strength of tuned neurons between twogenotypes longitudinally.



Model Selection (Lasso) used to assess neuron tuning.

Challenge: Population inference on neuron tuning across genotypes.


Motivating Case Study: Functional Connectivity

How does the brain communicate (connect) at a systems level?

Functional Connectivity: Estimated relationships between brain regions(usually from fMRI data).






Inference for Functional Connectivity: Existing Procedures

Step 0: Standard fMRI pre-processing & Parcellation.

Step 1: Estimate brain networks for each subject.

Step 2: Summarize network via network metrics for each subject.

Step 3: Use standard statistical inferential methods to test populationeffects.

Brain Connectivity Toolbox



Step 0: Standard fMRI pre-processing & Parcellation.I Parcellation: reduces fMRI volume to regions of interest (ROIs).

F Anatomical or Functionally derived Atlases.








I Correlation Networks.

I Markov Networks (partial correlations).

I Causal / Directed Networks.

I Dynamic Networks.









I T-tests, linear regression models, permutation null distributions,multiple testing, etc.


A Simulation Test

Simulation: Markov Networks - Inference on edges - Two-GroupPopulation.


A Simulation Test



A Simulation Test




2 PIMS

3 Empirical Studies


PIMS: Population Inference post Model Selection



PIMS

A two-level problem with:

Subject Level: Selection procedure used to estimate subject-levelparameters.

Population Level: Inference conducted on population-level parameters.

Similarities (and differences) from:

Post Selection Inference.

Classical Hierarchical Models.



Failure to account for subject-level model selection:

Biased estimates of β̂ (population-level parameters).I Support tests.I Parameter tests.

Under-estimated variance of β̂.

Results in:

Uncontrolled Type I Error.

Loss of Statistical Power.


Our Solution

General Selection & Inference Procedure: R3

* Resampling: Resample (bootstrap) subject-level data and performselection on bootstrapped samples.

* Randomized Model Selection: Use randomized model selection onresampled subject-level data.

* Random Effects Models: Conduct population inference via GLMMswith subject-level random effects.


Our Solution

R3 Algorithm Summary:

1 Take S bootstrap samples for each subject. For each sample, X∗si ,

perform randomized model selection:

θ̂∗si ← Randomized Model Selection(X∗si )

2 Fit GLMM:

g(E(θ̂(∗s)i |D,Z)) = Di ,sβ + Zi ,sα for i = 1, . . . n, s = 1, . . .S .

I D denotes the fixed population effects of interest.I Z denotes the random subject effects.I g() GLM link function.

3 Population Inference: H0 : β = 0 v.s. HA : β 6= 0.



2 PIMS

3 Empirical Studies


Simulation Studies



Simulation Studies

Simulation: Markov Networks - Inference on Subnetwork Density -Continuous Covariate (i.e. symptom severity) in Population.


Simulation Studies

Simulation: Markov Networks - Inference on Subnetwork Density -Continuous Covariate (i.e. symptom severity) in Population.


Case Study: Color-Sequence Synesthesia



Brain regions that process color have more connections to those that process

numbers and letters in synesthetes than controls.G. I. Allen (Rice & BCM) PIMS in Neuroscience October 2, 2016 13 / 18


Are there different node clustering patterns between the two groups?


Summary

Population Inference post Model Selection (PIMS)

New type of PSI problem.

Arises often in neuroscience.

Failure to account for model selection leads to uncontrolled Type Ierror and poor statistical power.

General selection and inference procedure: R3.

Future Directions

Develop selection and inference procedures for specific types of PIMSproblems.

Theory: Valid inference with PIMS problems.


Software

MoNet: Markov Network Toolbox (Matlab) for Functional Connectivity

Available from http://www.stat.rice.edu/∼gallen/ orhttps://bitbucket.org/gastats/monet/overview.


Acknowledgments

Manajari Narayan, PhD, Rice Electrical Engineering (Now atStanford).

Michael Weylandt, PhD Candidate, Rice Statistics.

Steffie Tomson, BCM.

David Eagleman, BCM.

Huda Zoghbi, BCM.

Stelios Smirnakis, BCM.

Andreas Tolias, BCM.


References

M. Narayan and G. I. Allen, “Mixed Effects Models for Resampled Network StatisticsImproves Statistical Power to Find Differences in Multi-Subject FunctionalConnectivity”, Frontiers in Neuroscience, 10:108, 2016.

M. Narayan, G. I. Allen & S. Tomson, “Two Sample Inference for Populations ofGraphical Models with Applications to Functional Connectivity”, (Submitted)arXiv:1502.03853, 2015.

S. Tomson, M. Narayan, G. I. Allen, & D. Eagleman, “Neural Networks of Synesthesia”,Journal of Neuroscience, 33:35, 14098-14106, 2013.

M. Narayan & G. I. Allen, “Randomized Approach to Differential Inference inMulti-Subject Functional Connectivity”, In IEEE International Workshop on PatternRecognition in Neuroimaging, 2013.

M. Narayan & G. I. Allen, “Population Inference for Node Level Differences inFunctional Connectivity”, In IEEE International Workshop on Pattern Recognition inNeuroimaging, 2015.

S. Tomson, M. Schreiner, M. Narayan, T. Rosser, N. Enrique, A. J. Silva, G. I. Allen, S.Y. Bookheimer, and C. Bearden, “Resting state functional MRI reveals abnormal networkconnectivity in Neurofibromatosis 1”, Human Brain Mapping, 36:11, 4566-4581, 2015.


Thank You!


Documents

Population Inference post Model Selection in Neurosciencekuffner/AllenSlides.pdf · Motivating Case Study: Neuron Tuning Model Selection (Lasso) used to assess neuron tuning. Challenge: