1

General Structural Equations

Week 1 Class #5

2

Today:Today:• More on estimation• More on block tests• Out of range solutions: what they mean & how to

deal with them• Higher order latent variable models

– When a latent variable has other latent variables as indicators

• Using the PRELIS program to generate covariance matrices for LISREL

• A quick look at SAS-CALIS• Item parcels (a controversy?)• An extended discussion of an example set• If time permits: re-expressing latent variable

structural equation models in matrix terms

33

First computer assignmentFirst computer assignment

Due TuesdayDue TuesdayShort answer responses (submit some of Short answer responses (submit some of the programs – if AMOS, diagrams or the programs – if AMOS, diagrams or diskette)diskette)REQUIRED for credit, letter with grade REQUIRED for credit, letter with grade participantsparticipantsChoice of AMOS or SIMPLIS. If using Choice of AMOS or SIMPLIS. If using SIMPLIS, a system (.dsf) file has already SIMPLIS, a system (.dsf) file has already been created.been created.

44

• PRELIS demonstrationPRELIS demonstration

Review of SIMPLIS programs Review of SIMPLIS programs (Relig&SexMor problem)(Relig&SexMor problem)

55

Estimation (notes)Estimation (notes)

Parameter estimates are obtained Parameter estimates are obtained through iterative methodsthrough iterative methods

Start with “start values”Start with “start values”• Could be user “guesses” (early versions Could be user “guesses” (early versions

of LISREL)of LISREL)• Could use some other single-step Could use some other single-step

estimation method (eg 2SLS)estimation method (eg 2SLS) Use start values to calculate Use start values to calculate

reproduced covariance matrixreproduced covariance matrix

66

Estimation (notes)Estimation (notes) Start with “start values”Start with “start values”

• Could be user “guesses” (early versions of LISREL)Could be user “guesses” (early versions of LISREL)• Could use some other single-step estimation method (eg 2SLS)Could use some other single-step estimation method (eg 2SLS)

Use start values to calculate reproduced covariance matrixUse start values to calculate reproduced covariance matrix Calculate first order derivatives for each free parameter Calculate first order derivatives for each free parameter

• Will tell us for any given parameter whether next iteration Will tell us for any given parameter whether next iteration value should be higher or lower – e.g., positive derivative value should be higher or lower – e.g., positive derivative means value is too highmeans value is too high

Optionally, calculate second order derivatives Optionally, calculate second order derivatives • Computationally intensive (usually)Computationally intensive (usually)• Trade off between extra effort at re-calculation (sometimes, Trade off between extra effort at re-calculation (sometimes,

matrix is merely updated with an approximation and only fully matrix is merely updated with an approximation and only fully re-calculated every X iterations) and precisionre-calculated every X iterations) and precision

• Sometimes, programs unable to calculate matrix of 2Sometimes, programs unable to calculate matrix of 2ndnd order order derivatives with given start values and use “Steepest Descent” derivatives with given start values and use “Steepest Descent” methods (esp. initially)methods (esp. initially)

77

Estimation (notes)Estimation (notes)

Convergence declared when estimates of fit Convergence declared when estimates of fit function become sufficiently similar from one function become sufficiently similar from one iteration to next and/or parameter estimates iteration to next and/or parameter estimates don’t differ by more than “x” (convergence don’t differ by more than “x” (convergence criterion)criterion)

Occasionally, a model will not convergeOccasionally, a model will not converge• Check the model to make sure there are no implausible Check the model to make sure there are no implausible

paths, model is identifiedpaths, model is identified• Check to make sure there aren’t any –ve error variances Check to make sure there aren’t any –ve error variances

[how to deal with these will be discussed later][how to deal with these will be discussed later]• If otherwise OK except for non-convergence, ask for If otherwise OK except for non-convergence, ask for

more iterations (most software will allow this)more iterations (most software will allow this)

8

“Problems”

The negative error variance.

In theory, it is impossible for a variance to be negative

But, SEM models can be estimated where the fit function minimum occurs when one of the parameters is “improperly” negative.

9

“Problems”

The negative error variance.Main reasons:1. Improperly specified model (e.g., missing a

parameter that would improve the fit considerably)

Frequently occurs in models with LVs with 2 indicators

2. Sampling distribution (the “real” parameter is positive in the population, but in our sample, it ends up being –ve)

More likely to happen in smaller samples

10

“Problems”

The negative error variance.

How the software responds to it:1. Allow the parameter to go negative, perhaps

providing a warning (“matrix not positive-definite”)

2. Allow a finite number of further iterations, and if the parameter doesn’t become positive, stop the iteration process and generate a warning message (LISREL default)

3. Impose an inequality constraint

11

“Problems”

The negative error variance.How the investigator should respond to it:

1. Check the model carefully. Are there any inappropriate paths? Major (likely) paths that are missing? In 2-indicator LV models, can a 3rd indicator be found?

2. Check the significance of the –ve parameter (if necessary, re-run the model over-riding the defaults so that the model is allowed to reach convergence)

3. Constrain the variance parameter to zero or some small value.

1212

Higher order modelsHigher order models

22ndnd order models: where the indicators order models: where the indicators for a latent variable are themselves for a latent variable are themselves latent variableslatent variables– many of the same principles apply: many of the same principles apply:

treat 1treat 1stst level latent variables as level latent variables as indicators of 2indicators of 2ndnd level latent variables level latent variables will need reference indicator, for examplewill need reference indicator, for example first-level latent variables must be first-level latent variables must be

sufficiently correlated (or model will not sufficiently correlated (or model will not work)work)

13

1

1 1 1

1

1 1 1

1

1 1 1

1

14

1

1 1 1

1

1 1 1

1

1 1 1

Higher order models: start with more modest ambitions and test a “correlated LV”

If correlations among LVs are low, it may not be reasonable or even possible to estimate a higher-order model

Only two indicators? Same issues as with lower-level LVs.

15

11

1

1

11

1

1

11

1

1

1

1

Is this model adequate?

16

11

1

1

11

1

1

11

1

1

1

1

Or is something like this required?

17

Block tests

L4

1

1 1 1

L5

1

111

L11

1

1

1

L21

1

1

1

L31

1

1

1

1

1 SAME ISSUES WHETHER L1-L3 are LVs, single indicator exogenous var’s or dummy variables

18

Block TestsMatrix:

L1 L2 L3 exogenous

L4 b1 b2 b3 (Model 1)L5 b4 b5 b6

Test of whether L1 has effect on endog. variables: Model 2, as above but b1=0 & b4=0Model 3, b1 through b6 = 0Model 4, b1≠0, b4 ≠0, {b2 b3 b5 b6 all = 0}t

Test of equation with L4 dependent:Model 5, as above but b1=0, b2=0, b3=0

Compare models using chi-square difference (df = difference in # of degrees of freedom between models)

19

Tests involving dummy (exogenous) variable contrasts

Example:

Variable = religionCategories: Protestant D1=1

Fundamentalist Prot D2=1Catholic D3=1Muslim D4=1Atheist/agnostic = reference

(D1=D2=D3=D4=0)

20

Tests involving dummy (exogenous) variable contrasts

Example:Variable = religionCategories: Protestant D1=1

Fundamentalist Prot D2=1Catholic D3=1Muslim D4=1Atheist/agnostic = reference

(D1=D2=D3=D4=0)Important: dummy variables are not (usually) orthogonal. Make sure to allow for covariances

among them [exception: orthogonally coded or effects coded in balanced designs]Coefficients:

Protestant b1 Tests Protestant vs. AtheistFund. Prot b2 Tests Fund. Prot. Vs. AtheistCatholic b3 Tests Cath. Vs. atheistMuslim b4 Tests Muslim vs. atheist

Other tests:Protestant vs. Catholic? Run a new model with b1=b3, compare chi-sq.Protestants AND fund. Prot. TOGETHER vs. Catholic?

Model 1: b1=b2Model 2: b1=b2=b3

Muslim vs. all others? Model 1: b1=b2=b3=0Model 2: b1=b2=b3=b4=0

Atheists vs. all others? Model 1: b1=b2=b3=b4Model 2: b1=b2=b3=b4=0

21

Item parcels

X1

1

1

X2

1

X3

1

X4

1

X5

1

X6

1

X7

1

X8

1

X1 X2 X3 X4 X5 X6 X7 X8

Versus:

Add scores of X1+X2+X3+X4+X5+X6+X7+X8

to get an “item parcel”

X9=

X9

1

1

Assume 0 error variance or estimate from reliability coeff.

22

Item parcels

Less extreme:

X1

1

1

X2

1

X3

1

X4

1

X5

1

X6

1

X7

1

X8

1

X9

1

X1+X2+X3

1

1

X3+X4+X5

1

X6+X7+X8

1

23

Reasons for using item parcels

• With single indicator models, will get fits that are very close to perfect (bad reason!)

• Individual indicators may be non-normally distributed (e.g., 4-category, 5-category attitude scale items tend to be kurtotic); summing indicators will often help

• Individual indicators may be “extreme” (e.g, dichotomies, tricotomies)

• The model may be “monstrous” (dozens of indicators per construct = hundreds of variables in model) with a lot of somewhat redundant information (alternative would be to randomly select indicators, but why throw away data?)

24

Reasons for NOT using parcels

• One of the big advantages of LV SEM models is discarded (at least in extreme cases where items reduced to a single item)

• Hypothesized pattern of inter-relationships among indicators may be incorrect (suggestion: test by running model on items to be parceled, if possible)

• Even if items internally consistent, assumes internal consistency will imply consistency with respect to other variables in the model

25

LV Structural Equation Models in Matrix terms

Thus far, our work has involved “scalar” equations.

• one equation at a time

•Specify a model (e.g, with software) by writing these equations out, one line per equation

26

Matrix formWe can represent the previous 2 equations in

matrix form:

Matrix Form

(single, double subscript)

27

There are other matrices in this model

Variance-covariance matrix of error terms (e’s)

28

(other matrices, continued)

Variance covariance matrix of exogenous (manifest) variables

29

Two scalar equations re-written

scalar

Matrix

Contents of matrices

30

More generic form (combines all exogenous variables into single matrix)

More generic:

Where E1 Ξ X1, E2 Ξ X2 and E3 Ξ X3

31

More generic form:

All exogenous variables part of a single variance-covariance matrix

32

Reproduced covariances

(the formula in matrix terms)

Θ above – elements of which are called θ[theta] is not the same as θ in Σ(θ). Latter refers to all parameters in a model. Theta above refers to elements in the variance-covariance matrix of errors/exogenous variables.

33

A simple model:

B

Continued……..

34

Reproduced covariances

(observed variable model without latent variables)

35

36

(proof of inverse: quick aside)

37

38

Measurement (“factor”) model

39

40

Alternative notation systems for coefficients:

41

42

43