1

Use of SEM programs to precisely measure scale reliability

Yutaka Kano and Yukari AzumaOsaka University

IMPS2001, July 15-19,2001Osaka, Japan

2Reliability measure for

Reliability with possibly correlated errors

α coefficient

jiji

ii

ii

iiii

uuCov

ufX

,

2

2

),(

'

)(

)(1

1 XV

XV

p

p i

p

iiXX

1

3

An example

α 0.69 0.74 0.78ρ’ 0.69 0.64 0.60

4

From the example

Coefficient alpha can be distorted seriously by error correlations e.g. Green-Hershberger (2000), Raykov (20

01)

In the case, ρ’ has to be used to correctly figure out the reliability

jiji

ii

ii

uuCov,

2

2

),(

'

5

ProblemHow can one identify error correlations? The factor model allowing for (fully) correlat

ed errors is not identifiable, because it contains too many parameters

A trivial solution would be

It does not work because)ˆ(),(̂ ijijji suuCov

0),(̂ ji

ji uuCov

6

LM approach

Start from the factor model with no error correlationPerform the LM test for releasing a zero covariance between errors using a SEM programEQS can perform it most easily and most accurately

7

Real data analysis

A questionnaire on perception on physical exercisen=653, p=15, one-factor modelThe data were collected by Dr. Oka (Waseda U.)

8

Result_1

Best fitted model, with 7 correlated errors χ2=250.375(df=83) (n=653) GFI=0.950, CFI=0.952, RMSEA=0.056

9

Result_2

Estimates of reliability α = 0.90 ρ’ = 0.90 by ρ’ = 0.87 by LM test

Note that

)ˆ(),(̂ ijijji suuCov

996.1),(

005.0)ˆ(

es7covarianc

^

ji

jiijij

uuCov

s

10

Search for variables

Even though one factor model is fitted well, inclusion of a variable with small true variance can reduce reliability

There is no convenient way to select variables for the composite scale to have maximum reliability

11

Mathematically…

.)(, where

,yreliabilit increases droppingthen

,

ed.uncorrelat and 1)( with Let

2

22

ijj

ijj

j

i

iiiii

uVBA

XBA

BBAAAB

IfuXVufX

It is complicatedIt will become more complicated if error correlations are allowed

12

New program

A new program is being developed which gives a list of reliability estimates for each f

actor; gives a list of predicted reliability estimates

when one variable is removed

Error correlations are allowed

13

Flowchart

Decide composite scale items

DATA

Factor analysis

Well fit?

Free some error covariances

to get good fit

Print reliability

No

Yes

End

14

Example, continued

ρ’ = 0.87 with 15 variables

15Scale developer

16If V13 is removed, then…

17

Results For one-factor model with uncorrelated errors, the variable with the smallest factor loading is least favorable. If there is a variable whose deletion improves

reliability, then this is the variable.

For one-factor model with correlated errors, the variable with the smallest factor loading is not always least favorable. While deletion of the variable does not

improve reliability, there may be other variables to be deleted to improve reliability.

The example here is the case.

18

Summary

Correlated errors invalidate the coefficient alpha and traditional one-factor based reliability.LM test is useful to find error correlations.Magnitude of factor loadings does not necessarily provide accurate information on indicator selection when correlated errors exist.The forthcoming Web-based program will help reliability analysis.