Testing Multilevel Theories Through Multilevel Structural Equation Modeling
Christopher R. Beasley
Midwest Eco 2013
Digital Copy of Slides
crbeasley.info
Structural Equation Modeling
Social
Desirability
Value
Congruence
Demands-
Abilities
Interpersonal
Similarity
Needs-
Supplies
Satisfaction
Commitment
OH TenureP-E Fit
Nesting
• Dependence
– Time
– Complex Sampling
Study Design
• Random sample by house
• Oxford House residents
– 95% of houses agreed to assist
• nj = 82
– 48% individual response rate
• ni = 296
Why Multilevel
• Disaggregation of data (Byrne, 2012)
– Biased estimates and standard errors
• Aggregated data (Byrne, 2012)
– Lack of individual variance may exaggerate group
effects
MSEM
• Extension of Multi-Group SEM
– Covariance matrices at within and between level
instead of for different groups
• Assumptions
– General Linear Model assumptions
• Linearity
• Normality
• Homoscedasticity
• Independence
MSEM Alternatives
• Segregated Approach (Yuan & Bentler, 2007)
– More established
– Modification indices
• Partially Saturated Approach (Ryu & West, 2009)
– MLR
– Random coefficients
– 2-1-1, 2-1-2, 2-2-2, 1-1-2, 1-2-1, 1-1-1
– Better power for level 2
– Greater n at L1, so more influence on model fit statistics (Hox, 2002)
Software Packages
• Mplus
• LISREL
• EQS
• GLLAMM
Process (Stapleton, 2013)
1. Consider single-level model
2. Baseline models for w/i & b/t levels while saturating the other level
3. Theoretical w/i model with b/t saturated for model fit
4. Theoretical b/t model with w/i saturated for model fit
5. Combined w/i & b/t theoretical model for parameters
6. Evaluate random coefficients
1. Consider Single Model
• Consider single-level model
– Null model for descriptives
• Define ICC < 0.02 as within unless very large cluster sizes, then possibly lower threshold
WITHIN ARE OHFITVC;
Between-Group Variance
ANALYSIS: TYPE IS TWOLEVEL RANDOM;
MODEL:
%WITHIN%
OHFITVC;
%BETWEEN%
OHFITVC;
Observed VariablesNull
ICC
Value Congruence 0.03
Commitment 0.11
Citizenship Behavior 0.04
1.1 Multilevel Reliability
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
VCw@1;
OHGEF3R OHGEF7R OHGEF15 (WR1-
WR3);
VCw BY OHGEF3R* (WL1)
OHGEF7R (WL2)
OHGEF15 (WL3);
%BETWEEN%
VCb@1;
OHGEF3R OHGEF7R OHGEF15 (BR1-
BR3);
VCb BY OHGEF3R* (BL1)
OHGEF7R (BL2)
OHGEF15 (BL3);
MODEL CONSTRAINT: NEW(NUMW
DENOMW OMEGAW NUMB DENOMB OMEGAB);
NUMW = (WL1+WL2+WL3)**2;
DENOMW = NUMW+(WR1+WR2+WR3);
OMEGAW = NUMW/DENOMW;
NUMB = (BL1+BL2+BL3)**2;
DENOMB = NUMB+(BR1+BR2+BR3);
OMEGAB = NUMB/DENOMB;
WR1 > 0; BR1 > 0;
WR2 > 0; BR2 > 0;
WR3 > 0; BR3 > 0;
Geldhof, Preacher, & Zyhur (in press)
1.1 Measurement Reliability
Measures ωw ωb
Satisfaction 0.77 0.92
Demands-Abilities Fit 0.78 0.97
Needs-Supplies Fit 0.79 0.92
Interpersonal Similarity 0.79 0.98
Social Desirability 0.88 0.96
Value Congruence 0.90 0.93
Tenure 0.91 0.93
Commitment 0.91 0.99
1.2 Descriptive Statistics
DEFINE: SDESIR=SDESIR/3;
Observed Variables n Min Max Mean SD SE
Social Desirability 292 0 13 6.67 3.23 0.19
Tenure 291 1 5 1.78 0.81 0.05
Commitment 292 2 7 5.26 1.10 0.06
Satisfaction 293 2 7 6.14 1.01 0.06
Interpersonal Similarity 291 1 5 3.42 0.96 0.06
Value Congruence 291 1 5 3.88 0.76 0.04
Demands-Abilities 293 1 5 3.96 0.80 0.05
Needs-Supplies Fit 293 1 5 4.01 0.70 0.04
2. Baseline Models
Baseline w/i Model
MODEL:
%WITHIN%
OHFITVC with OHCit@0 OHCOM@0;
OHCit with OHCOM@0;
%BETWEEN%
OHFITVC with OHCit OHCOM;
OHCit with OHCOM;
Baseline b/t Model
MODEL:
%WITHIN%
OHFITVC with OHCit OHCOM;
OHCit with OHCOM;
%BETWEEN%
OHFITVC with OHCit@0 OHCOM@0;
OHCit with OHCOM@0;
Commitment
3. Theoretical w/i Model
Value
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
3. Theoretical w/i Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
OHCOM on OHCit;
%BETWEEN%
OHCOM with OHFITVC OHCit;
OHFITVC with OHCIT;
3. Theoretical w/i Model
Chi-Square Test of Model Fit
Value 38.019*
Degrees of Freedom 1
P-Value 0.0000
Scaling Correction Factor
RMSEA
Estimate 0.359
CFI/TLI
CFI 0.895
TLI 0.373
SRMR
Value for Within 0.169
Value for Between 0.010
Commitment
4. Theoretical b/t Model
Value
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
4. Theoretical b/t Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM with OHFITVC OHCit;
OHFITVC with OHCIT;
%BETWEEN%
OHCOM OHFITVC OHCit;
4. Theoretical b/t Model
Chi-Square Test of Model Fit
Value 5.192*
Degrees of Freedom 3
P-Value 0.1583
Scaling Correction Factor
RMSEA
Estimate 0.050
CFI/TLI
CFI 0.994
TLI 0.988
SRMR
Value for Within 0.022
Value for Between 0.693
Commitment
3. Theoretical Combined Model
Value
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
5. Combined Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
OHCOM on OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;
5. Combined Model Two-Tailed
Estimate S.E. Est./S.E. P-Value
Within Level
OHCOM ON
OHFITVC 0.711 0.079 9.025 0.000
OHCIT 0.338 0.055 6.168 0.000
Between Level
Means
OHCOM 5.250 0.076 68.744 0.000
OHCIT 5.870 0.058 101.094 0.000
OHFITVC 3.887 0.046 83.593 0.000
Commitment
6. Random Coefficients
Value
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
CommitmentValue
Congruence
Citizenship
Behavior
Random
Slope
6. Random Coefficients
LISTWISE=ON; (or Monte Carlo integration)
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
s1 | OHCOM ON OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;
6. Random Coefficients
LISTWISE=ON;
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
s1 | OHCOM ON OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;
s1 on OHFITVC;
6. Random CoefficientsLoglikelihood
H0 Value -1074.838
H0 Scaling Correction Factor 0.9723
for MLR
Information Criteria
Akaike (AIC) 2175.675
Bayesian (BIC) 2223.249
Sample-Size Adjusted BIC 2182.024
6. Random CoefficientsTwo-Tailed
Estimate S.E. Est./S.E. P-Value
Between Level
S1 ON
OHFITVC 0.327 0.112 2.905 0.004
Intercepts
S1 -1.099 0.454 -2.421 0.015
Unresolved Issues
• Sample & Power Estimation
• Model Fit Approaches
• Model Fit Statistics
• MLR Performance in Multilevel Model
• 3+ Levels
• Balancing of Cluster Sizes
Unresolved Issues - Saturation
• “In the MODEL command, the following variable
is a y-variable (endogenous) on the BETWEEN level and an x-variable (exogenous) on the WITHIN level.
• This variable will be treated as a y-variable on both levels: OHGEFIS”
• Any discrepancy is treated as a y-variable on both levels
Other Models
• Longitudinal Multilevel Models
• Latent Models
• Multilevel EFA
• Multilevel CFA
• 3+ Levels
Other Topics
• Convergence Problems
• Power Analysis & Sample Size
• Alternative Estimators (MLR default)
– MUML, Bayes
• Random Starting Seeds
• Interval Estimates
– Bayes
– Monte Carlo
Resources
• Barbara Byrne
Structural Equation Modeling with Mplus
• Hancock & Mueller Structural Equation Modeling: A Second Course
• Little, Bovarid, & Card Modeling Contextual Effects in Long. Studies
• Kris Preacher http://www.quantpsy.org/pubs.htm
• Steve Miller “Things Statistical” http://personalityandemotion.com/