H T T P : / /WORKS . BEP RES S .C OM /JEFFR EY S M ORRI S S... · MODELING STRATEGIES FOR COMPLEX FUNCTIONAL DATA H T T P : / /WORKS . BEP RES S .C OM /JEFFR EY_ S_ M ORRI S 4 Many

H T T P : / / W O R K S . B E P R E S S . C O M / J E F F R E Y _ S_ M O R R I S 1

OUTLINEFocus: Develop functional regression methods for complex, high-dimensional functional data

Problems: Many standard methods either only model part of the data (feature extraction), or ignore structure in the data (elementwise modeling), which can be inefficient.

Many existing functional regression methods not suitable for complex, high-dimensional functional data characterized by complex interfunctional and intrafunctional structure.

Approach: Develop set of functional regression methods that:

Can model the entire data set, scale up to large data sets (N and T)

Flexibly account for complex interfunctional and intrafunctional structure

Produce rigorous (multiplicity-adjusted) inferential summaries

Outline: Overview of principles underlying functional regression (from review article)

Summarize a general modeling strategy for Bayesian functional response regression

Applications: Proteomic images from Addiction study

Longitudinally observed MRI images on spherical domain from Glaucoma study

Event-related potential (ERP) data from Smoking Cessation study

Whole organ histological and genomic maps (WOHGM): spatially correlated genomics


FUNCTIONAL DATA ANALYSISFunctional Data: data for which ideal units of observation are functions, and data

involves functions sampled on some observational grid

General Idea: data observed on some structured space and this structure

should be taken into account in estimation and inference.

Replication: Combining information across functions to draw conclusions about

the population from which they were sampled

Regularization: Borrow strength from observations within a function, exploiting

their expected structural regularity to gain efficiency and interpretability

Much work in FDA focuses on simple, smooth functions sampled on low

dimensional, 1d Euclidean domain

Scope of FDA is much broader than this: includes complex functions with local

features, higher dim and/or non-Eucl. Domain/manifold, high dim grid


MODELING STRATEGIES FOR COMPLEX FUNCTIONAL DATA


Many commonly used analysis approaches do not do justice to these rich data

Feature extraction: Compute summaries from function, only analyze those

Examples: peak/spot detection, means within ROI, freq/time intervals for ERP, genes for genomics

Benefit: dimensional reduction, incorporates biological knowledge

Drawback: may miss signals by oversimplifying data

Elementwise modeling: Fit separate models to each voxel/pixel/functional location (t)

Examples: mass univariate techniques, electrode-by-electrode models, probe- or m/z-level models

Benefit: easy to do, models all of the data (“unbiased”)

Drawback: computationally intensive, multiple testing, ignores correlation (inefficient).

Functional Regression: Fit model to entire data set, but account for correlation over (t)

Benefit: models all of data, accounts for correlation (typically through basis functions)

Drawback: difficult to build joint model capturing all complex structure, computationally intensive

FUNCTIONAL REGRESSIONFunctional regression is regression analyses involving functional data, with the function treated as

predictor, response, or both.

3 types: Functional predictor, Functional response, Function-on-function

Morris (2015) review: Annual Review of Statistics and Its Application

Nearly all existing methods use a variation of the same two-pronged strategy:

Represent functional quantities in model through basis functions (splines/kernels/wavelets/PCs) in

order to capture intrafunctional correlation (across t)

Regularize functional quantities using truncation, L2 penalization, or L1 penalization/selection of

basis coefficients

Key difference among functional regression methods: Choice of basis and regularization approach

Replication and regularization are applied in different ways and for different purposes for different

functional regression settings.


Functional linear model:

Goals in functional predictor regression:

1. Prediction

2. Evaluation of key features in weight function B(t)

Methods differ in choice of basis functions for X(t) and B(t) and regularization strategy

Replication: regression to uncover the relationship between Y and X(t)

Regularization: denoising of X(t) and regularization of B(t) to impose smoothness

Feedback between replication and regularization:

Regularization of X(t) stabilizes regression (reduce collinearity and measurement error)

Sparse functional data: PACE uses replication to inform regularization;

Yao, et al. (2005a) uses it with same basis for B(t)

Goldsmith, et al. (2011) utilize to represent X(t) while using splines for B(t)


FUNCTIONAL PREDICTOR REGRESSION

iaiai EdttBtXBY )()(0

FUNCTIONAL RESPONSE REGRESSION

Functional Response Regression:

Goals: 1. Test for significant predictors

2. Identify regions of function associated with predictor

Regularization: 1. Regularization of Ba(t)

2. Account for intrafunctional correlation in errors (borrow strength)

Replication: 1. Regression analysis

2. Account for interfunctional correlation (nested/spatial/longitudinal)Reweights samples in the regression and affects inference

Feedback between regularization and replication:

1. Use of basis functions for intrafunctional correlation improves estimates/inference

2. Accounting for interfunctional correlation makes a difference in estimation/inference

Lesson: unified FDA modeling has statistical advantages over multi-step methods that involve smoothing + pointwise regression or pointwise regression + smoothing


Yi(t) = XiaBaa=1

p

å (t)+Ei(t)

FUNCTIONAL MIXED MODELS

Add levels of random effect functions to account for interfunctional correlation (intra-subject)

This model is from Morris and Carroll (2006)

Others have worked with similar models (Guo 2002, Scheipl, Staicu, Greven 2014)

See Morris (2015) review article for comparison/contrast of these models

Gaussian and robust (Zhu, Brown, Morris 2011) versions of model developed

Extensions for spatial/temporal interfunctional correlation (Zhang et al. 2015, Zhu et al. 2015)

Semiparametric version of this model developed (fa(x,t), Lee, et al. 2015)

Most current literature deals with simple, smooth functions on sparse 1d Euclidean domain.

Our approaches have been developed with complex, high-dimensional functional data in mind –flexible enough for complex functions with many features, and scaling up to large data sets


Yi(t) = XiaBa(t)+ ZihlUhl (t)+Ei(t)l=1

q

åh=1

H

åa=1

p

å

BAYESIAN FUNCTIONAL MIXED MODELSModeling Procedure: (Basis transform modeling approach)

1. Represent functions using lossless/near-lossless truncated basis (wavelets, PC, spline, Fourier, …)

2. Compute basis coefficients for observed data (DWT, FFT, matrix multiplication)

3. Fit MCMC to basis space version of FMM, using suitable priors to induce proper penalization in

basis space for regularization (flexible choices can mimic truncation, L2, L1 or other penalties)

4. Transform MCMC samples of basis space parameters to data space.

Benefits of modeling in basis space: accounts for correlation across t

Allows borrowing of strength from nearby t to improve estimation of Ba(t)

Accounts for intrafunctional correlation across t in errors to provide improved inference.


NitYtYT

k

kiki ,,1 )()(

*

1

**

BASIS SPACE MODEL: GAUSSIAN FMM

Model for each basis coefficient k:

Even assuming independence in basis space (Eik~N(0, sk)) induces intrafunctional

covariance in original data space

Priors on B*ak lead regularization of fixed effect functions Ba(t)

Gaussian prior (L2), Laplace prior (L1), variable selection prior (spike-slab) or general

sparsity priors (Horseshoe, GDP, NG, BEN)

Can also apply joint thresholding (Morris, et al. 2011 AOAS) to mimic truncation process


cov{Ei(t1),Ei(t2 )} = jk (t1)k=1

T*

å jk (t2 )sk

Yik* = XiaBak

* + ZihlUhlk* +Eik

*

l=1

q

åh=1

H

åa=1

p

å

BASIS SPACE MODEL: ROBUST FMM

Zhu, Brown, Morris (2011 JASA); Model for each basis coefficient k:

Robust functional regression achieved through heavier-tailed likelihoods and random effects:

g1 exponential leads to Laplace likelihood (median functional regression), g2 Gamma conjugate

Same model can be used for random effects

Notes:

Leads to robust regression with estimation/inference on Ba(t) robust to global/local outliers

Simulations demonstrate construction can obtain good regression function estimates for even Cauchy data, yet trades off relatively little efficiency for Gaussian data

Paired with basis space modeling, construction automatically downweights local outliers (in frequency or time domain), while retaining information from other non-outlying curve regions.


Yik* = XiaBak

* + ZihlUhlk* +Eik

*

l=1

q

åh=1

H

åa=1

p

å

)|(~ )|(~ ),0(~ 21

* kkkikikikik vggλNE

BAYESIAN INFERENCECredible bands for Ba(t):

Pointwise:

Joint:

Posterior Probability Maps:

Pointwise:

SimBaS (Meyer 2015 Biom):

bFDR(d) (Morris 2008 Biom):

Global test (GBPV): Reject Ba(t)=0 if pSimBaS(t)<0.05 for any t

Posterior Probability Maps:Strongly flag t: pSimBaS(t)<0.05 (Experimentwise Error rate)

Weakly flag t: pbFDR(d)(t)<0.05 (FDR-d)

ppw(t)<0.05 (pointwise)

Note: can be computed for any Ba(t) or any combination (contrast or aggregate)

Functional Discriminant Analysis: Can model {Y(t)|group} and get Pr{group|Y(t)}

Zhu, Brown, Morris (2012 Biometrics)


pSimBaS (t) = min a : 0Ï Ja (t){ }

Ja (t) : Pr B(t) Î Ja (t)"t |Y (t){ } =1-a

pbFDR(d )(t) = Pr B(t) < ¶ |Y (t){ }

)(0:min)( tItppw

Ia (t) : Pr B(t) Î Ia (t) |Y (t){ } =1-a

Veera

Baladandayuthapani


Proteomics Data Application

Howard

Gutstein

Andrew

Dowsey

PROTEOMICS IMAGE DATA (2DGE)


Lower pH Higher pHHigher

mass

Lower

mass

Protein Spots (100’s-

1000’s/gel)

From Morris, et al. (2011 AOAS), Morris (2012 SII).

Mice trained to access cocaine, given access C (0hr)/SA (2hr)/LA (12hr)

33 gels from 21 mice on CeA region of brain

Goal: Find proteins over/underexpressed by cocaine exposure

PROTEOMICS IMAGE DATA

Model:

Construct overall mean, case-control images:

Mean Image: M(t1,t2)=1/3{B0(t1,t2) + B1(t1,t2) + B2(t1,t2)}

Drug-Control : C(t1,t2)= ½ {B1(t1,t2)+ B2(t1,t2)} - B0(t1,t2)

Goal: Find gel regions for which C(t1,t2) is “significant”

(significant evidence of at least 1.5-fold case/control ratio)


imageserror residual

21

imageeffect random Mouse

21

21

1

indicatorsmouse

2

0

imagemean group

21

indicatorsgroup

imagesresponse

212 ),(),(),(),(log ttEttUZttBXttY i

l

l

l

il

j

j

jiji

100 200 300 400 500 600 700 800

100

200

300

400

500

600

-1

0

1

2

3

4

5

MODEL-BASED MEAN GEL : M(T1,T2)


100 200 300 400 500 600 700 800

100

200

300

400

500

600-1.5

-1

-0.5

0

0.5

CASE-CONTROL EFFECT IMAGE : C(T1,T2)


100 200 300 400 500 600 700 800

100

200

300

400

500

600

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1-PBFDR(T1,T2), WITH SPOTS


100 200 300 400 500 600 700 800

100

200

300

400

500

600

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1



390 395 400 405 410 415 420

465

470

475

480

485

490

495

500 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1-PBFDR(T1,T2), REGION 1


390 395 400 405 410 415 420

465

470

475

480

485

490

495

500 -2

-1.5

-1

-0.5

0

0.5

1

1.5

2

AVERAGE GEL M(T1,T2), REGION 1


100 200 300 400 500 600 700 800

100

200

300

400

500

600

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1



120 130 140 150 160 170 180 190 200

390

400

410

420

430

440

450

460 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1-PBFDR(T1,T2), REGION 2


120 130 140 150 160 170 180 190 200

390

400

410

420

430

440

450

460 -2

-1

0

1

2

3

4

5

AVERAGE GEL M(T1,T2), REGION 2


Veera

Baladandayuthapani

Wonyul Lee

Massimo FazioCrawford Downs


Scleral Strain Application

GLAUCOMA DATAFrom Lee, et al. (2015)

Glaucoma: optic nerve damage from IOP, etiology unknown

Researchers at UAB have developed custom device to induce fixed IOP and measure scleral displacement around eye (each pixel: strain tensor)

Compute first eigenvector of strain tensor (maximum principal strain, MPS)

Goal: Assess MPS around sclera, and effect of age on MPS

Data: 19 subjects (ages 20-90), both eyes, 9 IOPs (7mm-45mm Hg); Each IOP/eye: MPS sampled on 14.4k grid on sclera -- Yi(θ,φ)

Multi-level, longitudinal images on spherical domain


GLAUCOMA DATA: MODEL SELECTION

Age: linear or nonparametric effect?

Can write f(agei,θ,Φ) in functional mixed model framework using Demmler-

Reinsch spline parameterization in random effect design matrix Z

IOP: linear, nonparametric, or hyperbolic effect? Interaction of IOP x age effects?

Model selection procedure developed in R (BIC-based)

Age: nonparametric effect

IOP: well fit by hyperbola

No interaction of age x IOP


GLAUCOMA DATA: MODEL COMPONENTSRandom Effect Functions: account for intrasubject correlation

Correlation between left and right eyes

Longitudinal correlation across IOP for each eye

Random hyperbolas per eye captures longitudinal correlation

Basis Functions: Wavelets, PC, or wavelet-regularized PCReproducibility and IMSE measures supported wavelets as best fit

Final Model:

H T T P : / / W O R K S . B E P R E S S . C O M / J E F F R E Y _ S_ M O R R I S

Yi(q,f) = B0 (q,f)+ IOPiB2(q,f)+ IOPi-1B3(q,f)+ f (agei,q,f)

+ ZilsubjectU1l

l=1

19

å (q,f)+ ZileyeIOPiU2l

l=1

38

å (q,f)+ ZileyeIOPi

-1U3l

l=1

38

å (q,f)+Ei(q,f)

30

RESULTS: MPS VS. AGE BY IOP


RESULTS: AUC VS. AGE


RESULTS: AUC VS. AGE BY REGION


RESULTS: DAUC/D{AGE} VS. AGE


Hongxiao ZhuWonyul Lee

Francesco Versace Paul Cinciripini


Smoking Cessation ERP Data

SMOKING CESSATION ERP DATA (BASELINE)From Zhu, et al. (2015 in prep)

Smoking cessation study: 180 subjects from randomized trial; each view series of 24 images of each

of 4 types (Nicotine, Pleasant, Unpleasant, Neutral) – focus on baseline, pre-treatment session

After EEG (129 electrodes), extract time series (900ms) for each image, process, and average to

obtain ERP by subject/image type

Processing: High/low pass filtering, artifact removal, eye blink correction, average referencing

ERP: Yia(s,t) for i=1,…,180; a=1,…,4; s=1, …, 129, and t=1,…,225

Goal: Find regions of (s,t) with differential response across stimulus types

Once best model found, will model pre/post treatment data from clinical trial and assess differences

associated with cessation success.


Yia(s, t) = Ba(s, t)+Ui(s, t)+Eia(s, t)

FUNCTIONAL MATERN MODELS FOR ERP DATABayesian wavelet-based functional mixed model has been used for electrode-

by-electrode analysis of ERP data (Davidson 2009)

Limitation: Does not account for electrode-to-electrode correlation

We have extended Bayesian FMM to capture inter-electrode correlation via

functional Matern processes (Zhu, et al. in prep)

Independent Matern processes for each basis coefficient leads to

nonseparable, nonstationary process allowing different level of spatial

correlation at different times.

More parsimonious separable Matern model can also be used which

assumes common Matern parameter across basis coefficients

Correlated NEG prior used to induce spatial smoothness in Ba(s,t)

We have developed these models in both the Gaussian FMM and Laplace

robust FMM settings (insensitive to outliers)

We also developed model selection procedures using posterior predictive

likelihood (Fit MCMC on training data, compute LPPL on validation data)


SIMULATION RESULTS: NO OUTLIERS

Model IMSE IWidth (%cov) SimBaS BFDR (δ=0.6) LPPL

FDR(.2) Sens(1.2) FDR(.5) Sens(1.2)

GFMM 0.46 0.69 (77%) 0.03 0.88 0.16 0.83 -10.4

RFMM 0.56 0.64 (72%) 0.05 0.82 0.14 0.62 -22.1

GFMMc 0.14 1.22 (97%) 0.02 0.93 0.08 0.89 -4.6

RFMMc 0.16 1.14 (99%) 0.02 0.88 0.09 0.89 -4.7


SIMULATION RESULTS: HEAVY TAILED (OUTLIERS)

Model IMSE IWidth (%cov) SimBaS BFDR (δ=0.6) LPPL

FDR(.2) Sens(1.2) FDR(.5) Sens(1.2)

WFMM 0.55 1.12 (80%) 0.01 0.87 0.08 0.88 -13.1

RFMM 0.50 0.70 (72%) 0.07 0.94 0.09 0.94 -26.0

WFMMc 0.22 1.88 (99%) 0.00 0.90 0.03 0.89 -4.6

RFMMc 0.13 0.98 (99%) 0.00 1.00 0.01 1.00 -3.4


MODEL SELECTION

Model selection done using predictive likelihood (train 140, validate 40)

Strongly favored models with inter-electrode correlation (and non-separability)

Some regions favored Gaussian model, others Robust model (outliers)


RESULTS


RESULTS


RESULTS -100MS TO 0MS: PRE-IMAGE


RESULTS 0MS-100MS: AFTER IMAGE


RESULTS: 100MS-140MS: P1 CIGARETTE EFFECT 1


RESULTS 216MS-232MS: PLEASANT EFFECT


RESULTS 232MS-300MS: EMOTIONAL EFFECT 1


RESULTS: 440MS-600MS: EMOTIONAL EFFECT 4


RESULTS: 660MS-800MS: LATE CIGARETTE EFFECT


Veera

BaladandayuthapaniLin Zhang

Hongxiao Zhu

Bogdan CzerniakTad Majewski

Keith Baggerly


Bladder Genomic Map Data

BLADDER GENOMIC MAP DATA


BLADDER GENOMIC MAP DATA

Zhang, et al. (2015 JASA-TM, in press)

Goal: Find genomic regions with “early” copy number gains/losses that are

monotonic with histology (NLGHGIC)

Early events: potential drivers of bladder cancer

Model:

Haar wavelet bases (piecewise constant)

Spatially correlated errors: functional conditional autoregressive (fCAR)

CAR models per basis coefficient CAR at each t

Allows different spatial correlation across genomic locations

Inference: Find t: B1(t)<B2(t)<B3(t) or B1(t)>B2(t)>B3(t) and significant

aberration from normal (δ>0.2)


log2{Ys(t) /Y0(t)} = XsaBa(t)+Es(t)a=1

3

å

RESULTS AND SIMULATION RESULTS


OTHER APPLICATIONS AND METHODS

Functional regression approaches can be applied broadly to many types of complex functional data

Proteomics: LC/MS (Liao et al. 2014 ASMS conference), image MS (Zhang, et al. 2015 JASA-TM)

Sonic images: ultrasound (Lancia, et al. 2015 JAcouSA), sonar (Martinez et al. 2013 JASA-ACS)

Neuroimaging: fMRI, cortical surface thickness, DTI, MAP-MRI, shape data from MRI

Methodological innovations (recent or nearly complete)

Functional CAR models for functions on a lattice (Zhang, et al. 2015 JASA-TM)

Robust and Gaussian Functional Matern models (Zhu, et al. 2015 to be submitted)

Function-on-function regression models (Meyer, et al. 2015 Biom)

Semiparametric FMM for longitudinal functional data (Lee, et al. 2015 to be submitted)

Functional graphical models for time-varying functional connectivity (Zhang, et al. in prep)

Goal: suite of flexible, scalable, automated Bayesian methods for functional/image response regression

Scale up to whole-genome or group-level fMRI data, yield rigorous multiplicity-adjusted inference


Lin Zhang

Francesco

Versace

Paul

CinciripiniH T T P : / / W O R K S . B E P R E S S . C O M / J E F F R E Y _ S_ M O R R I S 62

Acknowledgements

Veera

Baladandayuthapani

Massimo

Fazio

Crawford

Downs

Howard

Gutstein

Hongxiao ZhuWonyul Lee Keith Baggerly

Andrew

Dowsey

Bogdan

Czerniak

KEY REFERENCES1. Lee W, Baladandayuthapani V, Fazio M, Downs C, and Morris JS (2014): Semiparametric Functional Mixed Models for

Longitudinal Functional Data with Application to Glaucoma. Technical Report.

2. Morris JS (2015). Functional Regression. Annual Review of Statistics and Its Applications,to appear.

3. Morris JS and Carroll RJ (2006). Wavelet-Based Functional Mixed Models. Journal of the Royal Statistical Society, Series B , 68(2): 179-199.

4. Morris JS, Baladandauthapani V, Herrick RC, Sanna PP, and Gutstein HG (2011). Automated analysis of quantitative image data using isomorphic functional mixed models, with application to proteomic data. Annals of Applied Statistics, 5(2A), 894-923.

5. Zhu H, Brown PJ, and Morris JS (2011): Robust, Adaptive Functional Regression in Functional Mixed Model Framework. JASA, 106(495): 1167-1179.

6. Zhu H, Brown PJ, and Morris JS (2012): Robust Classification of Functional and Image Data Using Functional Mixed Models. Biometrics, 68(4): 1260-1268.

7. Morris JS (2012) : Statistical Methods for Proteomic Biomarker Discovery using Feature Extraction or Functional Data Analysis Approaches. Statistics and its Interface,.

8. Lee W and Morris JS (2014): Functional Mixed Models for Whole-Genome Methylaton Analysis. TR.

9. Zhang L, Baladandayuthapani V, Baggerly KA, Majewski T, Czerniak BA, and Morris JS (2015): Functional CAR Models for Large Spatially Correlated Functional Data. JASA.

10. Morris JS, Arroyo C, Coull B, Ryan LM, Herrick R, and Gortmaker SL (2006). Using wavelet-based functional mixed models to characterize population heterogeneity in accelerometer profiles: A case study. JASA 101: 1352-64.

11. Morris JS, Brown PJ, Herrick RC, Baggerly KA, and Coombes KR (2008). Bayesian Analysis of Mass Spectrometry Data using Wavelet Based Functional Mixed Models. Biometrics, 12, 479-489.

12. Martinez JG, Bohn KM, Carroll RJ and Morris JS (2013). A study of Mexican free-tailed bat syllables: Multi-domain modeling of nonstationary time series with high frequency content using Bayesian functional mixed models. JASA.

13. Meyer M, Coull BA, Versace F, and Morris JS (2015): Bayesian Function-on-Function Regression for Multi-Level Functional Data. Biometrics.

63H T T P : / / W O R K S . B E P R E S S . C O M / J E F F R E Y _ S_ M O R R I S

Documents

H T T P : / /WORKS . BEP RES S .C OM /JEFFR EY S M ORRI S S... · MODELING STRATEGIES FOR COMPLEX FUNCTIONAL DATA H T T P : / /WORKS . BEP RES S .C OM /JEFFR EY_ S_ M ORRI S 4 Many