Advances in Longitudinal Methods in the Social and ...€¦ · Nonlinear Mixed- Effects Mixture...

Advances in Longitudinal Methods Conference J. Harring 1

Advances in Longitudinal Methods in the Social and Behavioral Sciences

Finite Mixtures of Nonlinear Mixed-Effects Models

Jeff Harring Department of Measurement, Statistics and Evaluation The Center for Integrated Latent Variable Research University of Maryland, College Park harring@umd.edu

Overview

� Mixture modeling

– univariate and multivariate applications

� Characteristics of repeated measures learning data

� Nonlinear mixed-effects models – model description and analysis

� Nonlinear mixed-effects mixture (NLMM) model – model description

Overview

� Learning data example revisited

– analytic decision points

� Issues, challenges & considerations

Finite Mixture Models

� Karl Pearson (1894)

� Primary purposes… – model the density of complex distributions – model population heterogeneity

� Modeling heterogeneity � mixture of distributions from the same parametric family

� Inferential goals

Applications of Finite Mixture Models

� Univariate mixtures

� Regression mixtures

� Regression mixture modeling involves estimating separate regression coefficients and error for each latent class

0 1i k k i iky x e� ��

� Multivariate normal mixtures

– MVN mixtures � cluster analysis

( | , ) ( | )K

i k k i kk

f f��

� �y � � y �

1 22 11( | ) (2 ) exp ( ) ( )2

pk i k k i k k i kf � �� y � � y � � y �

( | , ) ( | )K

i k k i kk

f f��

� �y � � y �

1 22 11( | ) (2 ) exp ( ) ( )2

pk i k k i k k i kf � �� y � � y � � y �

( | , ) ( | )K

i k k i kk

f f��

� �y � � y �

1 22 11( | ) (2 ) exp ( ) ( )2

pk i k k i k k i kf � �� y � � y � � y �

( | , ) ( | )K

i k k i kk

f f��

� �y � � y �

1 22 11( | ) (2 ) exp ( ) ( )2

pk i k k i k k i kf � �� y � � y � � y �

� Growth mixture modeling

y �� e

ki iki

��

� ��

� �

� � � ��

1 22 11( | ) (2 ) exp ( ) ( )2

pk i k k i k k i kf � �� y � � y � � y �

� �� k k k k k

��

� Handle nonlinear data with a nonlinear function

� Modeling potential population heterogeneity

Intrinsically Nonlinear Functions Nonlinear Mixed- Effects Model

Modeling Population Heterogeneity Finite Mixture Model

Intrinsically Nonlinear Functions Nonlinear Mixed- Effects Model

Modeling Population Heterogeneity Finite Mixture Model �

Nonlinear Mixed- Effects Mixture

Nonlinear Learning Data – Speech Errors

� Burchinal & Appelbaum (1991)

– Repeated measures data are recorded speech errors on a standard passage of text from an instrument of language proficiency

– A sample of 43 young

children ranging in ages from 2 to 8 years

Nonlinear Learning Data – Speech Errors

� Burchinal & Appelbaum (1991)

– A maximum number of 6 repeated measures were taken

– Ages at time of testings

differed for each child – incomplete cases were

observed

– independent rating of overall speech intelligibility used as a covariate

Model for Nonlinear Learning Data

� The nonlinear mixed-effects (NLME) model is well- suited to handle both intrinsically nonlinear functions as well as key data and design characteristics – continuous response – compellingly strong individual difference in trajectories – substantial variability in time-response across subjects – distinct measurement occasions for each subject – interesting nonlinear change

1( , , , )ij i pi ij ijy f x e� ��

1, , ; 1, ,ij n i m� ��

NLME Model for Burchinal & Appelbaum Data

� Following Davidian & Giltinan (2003) : two-stage hierarchy

� A subject-specific decelerating, decreasing exponential

function is proposed � Stage 1: Individual-Level Model

1 2exp ( 3)i iij ij ijy x e� ��

– 1i� : the average number of speech errors at age 3 – 2i� : rate parameter that governs functional decline

� Stage 2: Population-Level Model

��

� unconditional

z bz b

� ��

��

� �� conditional

� Distributional assumptions

( , )i Nb 0 � ( , ( ))e 0 � i iN� cov( , )i i

� �e b 0 cov( , )e e 0i i�

� � cov( , )b b 0i i�

� �

var( )cov( )

cov( , ) var( )i

ii i i

bb b b

� ��

� �b

� Distributional assumptions

( , )i Nb 0 � ( , ( ))e 0 � i iN� cov( , )i i

� �e b 0 cov( , )e e 0i i�

� � cov( , )b b 0i i�

� � 2

ii n�� I

Inference NLME Model : Maximum Likelihood

� The marginal distribution of y i

( ) ( , ( )) () |y y b y bb bbi i i i iii i ph p d p d� ��

� Let � �, , ( )vech ��

� Parameter estimation is carried out by maximizing the log-likelihood AGHQ (Pinheiro & Bates),GHQ (Davidian & Gallant), Linearization (Lindstrom & Bates), GTS (Davidian & Giltinan),Bayesian…

( ) ln ( | )

ln ( )

� � y

� �

Analysis

� The unconditional model was fitted using SAS PROC

NLMIXED (Gaussian Quadrature – 30 points)

ˆ .ˆˆ .��

� � � ��

18 48�

ˆ��

� � �� 0.1

77.025 0.05

2� �� 9.73

2ln 1227.0 1249.6 L BIC� � �

Analysis

� The conditional model was fitted with mean-centered

covariate – SAS PROC NLMIXED (GH - 30 points)

ˆ��

� � � ��

18.22�

0.97 1

ˆ��

��

2.76�

ˆ��

� � �� 0.2

76.849 0.04

2� �� 9.73

2ln 1217.2 1246.3 L BIC� � �

Potentially Clustered Data

� In subject-specific models, like the NLME model, regression

parameters are allowed to vary across individuals resulting in differing within-subject profiles

� ( )iE �b 0 & ( )e 0iE � ��assumption all subjects were

sampled from a single populations with common parameters

� Finite mixture models relax the single population distributional assumption of the random effects and the conditional distribution for the data to allow for parameter differences across unobserved populations

Verbeke & Lesaffre, 1996, Verbeke & Molenberghs, 2000 Muthén & Shedden, 1999; Muthén, 2001, 2003, 2004

Hall & Wang, 2005

NLMM Model

� The exponential NLME model can be extended to a NLMM

model for K latent classes where in class k (k = 1, 2,…, K )

1 2exp ( 3)y ei i i ij ix� �� ( , )e 0 �i kN�

� � � z bi k k i i� � � ( , )b 0 i kN�

1 2k k� �and � k� is the probability of belonging to class k 1k

k��

� 0 1k��

NLMM Model – Getting Started

� Fit a series of models suppressing random effects : k � 0

& � �k � (LCGM – Nagin (1999))

� Method of deriving starting values for the mean structure of

the NLMM model

� Use NLME output to suggest starting values for and �

NLMM Model – How Many Latent Classes?

� Several statistics have been proposed and recommended in

practice: AIC, BIC, SBIC, CLC, LMR-LRT (Lo, Mendell & Rubin, 2001), BLRT, Multivariate skewness and kurtosis indices

� Compute and plot BIC or SBIC values against number of

classes

……………… No Within-Class Variation LCGM

____________ Within-Class

Variation NLMM

� Decision based on…

– theoretical defensibility – model fit – parsimony – class separation and class

incidence

� Expertise of the substantive researcher

� Do the classes represent substantively meaningful groups?

NLMM Model – Analysis of Learning Data

� Parameter estimates for the two-class model

NLME Estimates

Class Means

ˆk�

Class-Specific Covariate Estimates

ˆ k�

.18 22

� ��

13 240 33

0 66�

��

� ��

.ˆ. 2

19 450 67

1 07�

� ��

11 12. .ˆ ˆ 00 04 10� � ��

ˆ .ˆ

ˆ��

��

0.082 76

21 22. .ˆ ˆ2 72 0 25� ��

NLMM Model – Analysis of Learning Data

� Parameter estimates for the two-class model NLME Estimate

Covariance Matrix

Class-Specific Covariance

Matrix ˆ ˆk �

NLME Estimate Residual Variance

Class-Specific Residual Variance

2 2ˆ ˆk ��

�� 0.15 0.05

� �

� ��

53 652 35 0 62

NLMM Model – Assessing Model Fit?

� The quality of the mixture based on the precision of the

classification – classification is based on estimated posterior probabilities

� For 2K � , average posterior probabilities can be computed.

A K K� matrix should have high diagonal and low off-diagonal values indicating good classification quality – Average Latent Class Probabilities for Most Likely Latent

Class Membership (Row) by Latent Class (Column) 1 2 1 0.854 0.146 2 0.204 0.796

NLMM Model – Assessing Model Fit?

� Entropy – a summary measure of the classification based

on individuals’ estimated posterior probabilities can be computed with values close to 1 indicating near perfect classification

( ln )ˆ ˆ1

ik iki k

KE m K

� ��

��

2 0.73KE � �

Graphical Summaries

� Plot predicted random coefficients on a contour plot or

surface plot

Graphical Summaries

� Plot predicted random coefficients on a contour plot or

surface plot

Good Separation Modest Separation

Graphical Summaries

Two Latent Classes: Mean Trajectories

Graphical Summaries

Fitted Curves for 4 Individuals Fitted Curves for 4 Individuals in Class 1 in Class 2

Practical Issues…

� Estimation

� Computing standard errors

Log-likelihood

� Let 1 1( , , )K� � �

� ��

� Let ( , ( ), ( ))k k kvech vech� ��

� Let ( , )� � �� all model parameters, then the log-likelihood

� ��

( ) ln ( | )

ln ( )

� � y

k k iki

k k ii k

��

� �

where ( ) ( | ) ( )y y b b bk i k i i k i ih p p d� �

Estimation

� If the nonlinear regression coefficients are fixed across

individuals : Mplus (however – does not handle unique measurement occasions)

21 exp ( 3)ij ij ijiy x e��

� Directly maximize the log-likelihood using gradient methods

like N-R or Quasi-Newton

� Use EM (Expectation – Maximization) algorithm treating class membership as missing data

Standard Errors

� Direct maximization uses the diagonal elements of the

Hessian matrix at convergence for a model-based estimate of the standard errors.

� EM algorithm – no standard errors are computed as a by-product of the algorithm – At convergence, use a direct maximization step to

produce SE

More Practical Issues and Future Considerations…

� Evidence of local extrema

� Covariates predicting individual coefficients & class

membership

� Does the method of handling the intractable integration influence the number of latent classes?

� Normality of the random effects distribution: non-parametric alternatives?

Advances in Longitudinal Methods in the Social and Behavioral Sciences

Finite Mixtures of Nonlinear Mixed-Effects Models

Jeff Harring Department of Measurement, Statistics and Evaluation The Center for Integrated Latent Variable Research University of Maryland, College Park harring@umd.edu

Advances in Longitudinal Methods in the Social and ...€¦ · Nonlinear Mixed- Effects Mixture...

Documents

Nonlinear Hyperbolic Equations, Spectral Theory, and Wavelet Transformations: A Volume of Advances in Partial Differential Equations

Linear and Nonlinear Nanophotonic Devices Based on Silicon ... · and huge knowledge base. In this paper we give an overview of the advances in passive linear and nonlinear functions

Advances in Combat Amputee Care The VA Systems-Based Approach to Longitudinal Care · 2018-05-10 · Advances in Combat Amputee Care The VA Systems-Based Approach to Longitudinal

Advances in Fuzzy Systems — Applications and Theory ... Mining/Nonlinear...World Scientific Advances in Fuzzy Systems — Applications and Theory – Vol. 24 Zhenyuan Wang University

Recent Advances in Nonlinear Response Structural Optimization Using … · Recent Advances in Nonlinear Response Structural Optimization Using the Equivalent Static Loads Method Oct

NONLINEAR DYNAMICS OF LONGITUDINAL GROUND VEHICLE … · 2015-12-31 · of longitudinal ground vehicle traction. Speciﬁcally, the performance of rubber-tired automobiles under straight-ahead

Nonlinear Dynamics of Longitudinal Vehicle Braking VSDIA 04, Budapest, Hungary, November 8-10, 2004

Neural Advances in Processing Nonlinear Dynamic Signalsvenus.unive.it/~fasano/papers/2019_Book...Neural Advances in Processing Nonlinear Dynamic Signals Smart Innovation, Systems and

Editorial Advances in Nonlinear Complexity Analysis for ...downloads.hindawi.com/journals/aaa/2013/585716.pdf · Advances in Nonlinear Complexity Analysis for Partial Differential

Advances in Modelling and Analysis B - AMSE...This method represents a tool for solving nonlinear differential equations such as non-linear optimal control issues [44-47]. Nonlinear

Advances in Iterative Methods for Nonlinear Equations · family. Finally, a new uni-parametric family of multi-point iterations is constructed in Banach spaces to solve nonlinear

7 Generalized and Nonlinear Models for Univariate Responsedavidian/st732/notes/chap7.pdf7 Generalized and Nonlinear Models for Univariate Response 7.1 Introduction The models for longitudinal

A novel combined control based on an adaptive … · Advances in adaptive nonlinear control have shifted the focus toward the robust adaptive control problem for nonlinear systems

MODERN NONLINEAR OPTICS - download.e-bookshelf.de · MODERN NONLINEAR OPTICS Part 2 Second Edition ADVANCES IN CHEMICAL PHYSICS VOLUME 119 Edited by Myron W. Evans Series Editors

Recent Advances in Nonlinear Response Structural ...€¦ · 1 Recent Advances in Nonlinear Response Structural Optimization Using the Equivalent Static Loads Method Oct. 28, 2014

Editorial Nonlinear Dynamics in Applied Sciences Systems ...downloads.hindawi.com/journals/aaa/2014/782657.pdfEditorial Nonlinear Dynamics in Applied Sciences Systems: Advances and

Advances in longitudinal data analysis: An historical ... · longitudinal data at a time when computing was in its infancy. This explains why, in those days, the key issue perceived

Fluid Mechanics and Heat Transfer Advances in Nonlinear Dynamics Modeling

Advances in The Nonlinear Optics Group (NLOG) at Soreq …

Nonlinear Programming - Heidelberg University · 2019-09-16 · Nonlinear Programming Ekaterina A. Kostina IWR School 2018 “Advances in Mathematical Optimization” October 8-12,