Upload
sarao26
View
228
Download
0
Embed Size (px)
Citation preview
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
1/23
8/02/2011
1
An Introduction to Confirmatory Factor
Analysis (CFA) and Structural
Equation Modeling (SEM)Gavin T L Brown, PhD
Presentation to Research Development Office, Continuing Professional DevelopmentProgramme, HKIEd, 10 February 2011
What is a CFA or SEM model?
A theoretically informed simplification of
the com lexities of realit created to testor generate hypotheses about how
various constructs are related
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
2/23
8/02/2011
2
Theory We have theories that explain the way things are (not just
descriptions)
eory an a a are n er- w ne
We see phenomena and seek to explain them with theories
We have theories and seek to test them with phenomena
Theories Knowledge
but theories that do not explain phenomena are certainly false[Knowledge--Popper]
CFA/SEM is situated in hypothetico-deductive orabductive approaches to meaning
Models
Everything is connected to everything in the real world
Its messy and hard to make sense of
in a model we select for theoretical reasons the important
connections that we THINK explain most of what is going on
in the phenomenon of interest
It is not the real thing, but a simplification
The arrangement of the connections between and among
variables of interest constitute testable expressions ofour theories about how things go together
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
3/23
8/02/2011
3
Prediction, Causation, Association CFA/SEM models assume linear (i.e., correlations and
regressions) relationships (paths) exist among constructs.
or examp e:
(AB) [2 things are correlated]
(AB) C [2 correlated things jointly influence a 3rd
thing]
(A + B) C [2 things separately and/or jointly influence a 3rd
thing]
A B C [1 thing influences a 2nd which influences a 3rd] And so on. [moderation, mediation, complex inter-
relationships]
CFA/SEM Involves Mathematical Testing of
Models
A sophisticated correlational-causal mathematical testingof a model against a data set
ow c ose are t ey oes t e mo e t t e ata
Models are rejected if they do NOT have close fit to the data
the data cant be wrongits the reality we are trying to model
Models are NOT accepted if they have close fit to the data
They are NOT YET DISCONFIRMEDPopper
Multiple models can fit equally well the same data
Fit could be attributable to chance factors in the data we collected
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
4/23
8/02/2011
4
CFA/SEM: Extending Latent Trait Theory Observed manifest behaviours
e.g., test scores, attitude item responses, observed frequenciesof behaviours, etc.
Are shaped and influenced by invisible (LATENT) sharedcauses. For example, Answers to items [manifest observed] on a test are caused (in
part) by INTELLIGENCE [latent unobserved] traits
Student responses to Browns Conceptions of Assessmentinventory are shaped in part by the hypothesised beliefs that:
ASSESSMENT IS FOR IMPROVEMENT; ASSESSMENT IS IRRRELEVANT;
ASSESSMENT HAS AFFECTIVE/SOCIAL BENEFITS
ASSESSMENT REFLECTS EXTERNAL CAUSES
Latent Trait Theory Multiple manifest indicators are required to have stable
estimation of the latent traits existence, strength, anddirection
ence, ac or ana ys s expec s o ems per ac or Hence, test scores rely on 5 to 30 test questions
WHY?
CHANCE.ERROR.DEFICIENCIES IN STIMULI Observed behaviour is not perfectly controlled or reflective of our
TRUE intelligence, attitude, etc. I chose B but I meant A; I chose response 3 but I meant 4
I want 3.4 but I had to choose 3 or 4
Hence, all values are ESTIMATES A range of most likely values exists Multiple indicators reduces error/chance effects
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
5/23
8/02/2011
5
Interpreting Output Values in CFA/SEM
CFA and SEM modelsuse conce ts of ecausation and prediction linear regression
Changes in XXX cause alinear change (increase ordecrease) in YYY
Formula: Y= m*X + b
Yv
ariabl
X
bintercept
m=slope [standardisedbeta = a proportion ofstandard deviation]
b=intercept [starting pointof equation; represents allthe unknown stuff]
var a e
Interpretations:1. For every 1 SD change in X, youwill get m*SD change in Y.2. This relationship explainsx% ofvariance in Y
Looking Under the Hood: Components of
CFA and SEM models
Variables
Manifest [observed behaviours,,
Latent [unobserved, explanatory, ovals]
Residual [unobserved, unexplained, ovals]
Manifest variables are predicted by both Latent traits andresiduals
Goal to have large proportion of variance in manifest explainedby latent rather than residual disturbances
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
6/23
8/02/2011
6
Looking Under the Hood: Components of
CFA and SEM models Paths
Fixed: equations require SEED values to solve; 1 is the.
predicted manifest variables with a fixed value. All other valuesare estimated relative to the seed value.
Free: All other paths are allowed to be estimated freely based
on the data provided to the model; they may be stronger thanthe fixed path, but better to make the strongest path in a
factor the fixed path.
Zero: Paths not required by the model are forced to be non-existent. This contrasts to EFA where all paths have somefreely estimated value.
Example of Path Values
EFA indicated Grades wasthe strongest value
Grades e12
1
1
Thus, seed value on path
Residual terms exist and
have seed value of 1 becausethey are equal to each other
Note: manifest variablesONLY have paths from the
Well-being
Evaluative
Ticks e13
Praise e14
Stickers e15
Answers e16
1
1
1
conceptua tra t Zero between each other
If 2 or more factors, itemsshould have ZERO paths toother factors
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
7/23
8/02/2011
7
Estimation Maximum likelihood
The parameter values in the data set (a sample) are the most likelyvalues in the population (not present, but to which we wish togeneralise)
Hence, procedure attempts to maximise the input values (means,standard deviations, covariances) when estimating the solution
Hence, it matters that the sample reflects the population and issufficiently large that parameters are likely to apply to population
N500; if 100) and large
number of manifest variables
o s a poor es , no w s an ng ve emen o ec ons y
some researchers
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
8/23
8/02/2011
8
Evaluating Results: Which Fit indices &
What Values?
Goodness of Fit Badness of fit
Decision p of2/df CFI
RMSEA SRMR*gamma a
Good >.05 >.95 .90
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
9/23
8/02/2011
9
Distinguishing CFA from SEM CFA = measurement model of a construct
CFA models can have multiple dimensions and complex
An achievement score can be hierarchical
total consists of surface AND deep cognitive processes
An attitude or opinion can be multi-correlated
Total consists of correlations between 3 or more related dimensions
SEM = structural model of paths between constructs
Attitudes towards X influence performance on Y
Attitude towards X is related to attitude towards Y
Example: CFA + SEM(Brown & Hirschfeld, 2008)
CFA: Measurement Model-4 correlated factorsNote. Accurate measurement models are also needed for
reading score, year, sex, & ethnicity
Structural model:multiple predictorsof performance
Note.If measurements of each construct are NOT robust, do NOTuse them for anything!!!
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
10/23
8/02/2011
10
Linear Models are Recursive(Brown et al., 2009)
CFA/SEM assume models are recursive
origins
ave a eg nn ng an an en w c are no e same
NOT circular
endings
How to Test Reciprocal Models?
Make it longitudinal
Time 1 Time 2
1 1 1 2 2 2
Use 2 different methods of measuring construct A
AM1BCA
M2
These approaches honour the reciprocal effects in theorywithout invalidating the linear regression equations
beyond todays talk
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
11/23
8/02/2011
11
Interpreting a Model Statistical significance of paths
The weights & directions of each path
The proportion of variance explained (the effect size)
Evaluating Results
Statistically significant paths
The strength of the path should exceed what might occur by
option to remove such paths or indicate as ns
If p>.05 pathnot stat sig
Note. Fixedpaths have no
probability.
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
12/23
8/02/2011
12
Evaluating Results Variance explained (SMC)
Equivalent to R2
.19MQQ44e42 .44
f2 =.19/.81=.23 (medium)
e ec s ze -
Small: .02 to .14
Medium: .15 to .34
Large: >.35
(Cohen, 1992)
.08
Evaluation
.34MQQ23e37
.29MQQ8e38
.17MQQ25e39
.23MQQ5e40
.MQQ63e41
.58
-.54
.41
.48.37
-.28
Note. SMC = Beta squaredBalanced not explained is in theresidual (goal small residuals, sotarget >.50)
Developing a Model
Evidence from theory
Evidence from Exploratory Factor Analysis
Evidence from Regression Analysis
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
13/23
8/02/2011
13
The Role of Theory in Designing Models to
Test My research questions:
Do conceptions of assessment influence performance?
eoret ca ramewor :
Icek Ajzen: Reasoned or Planned Behaviour
Beliefs & Intentions influence Behaviour & Outcomes
Beliefs are inter-correlated
Outcomes
Criterion of effectiveness
EFA to CFAStatement 1 2 3 4 5 6 7
29.Assessmentfostersstudents'character. 0.556 0.023 0.11 0.154 0.097 0.047 0.072
22.Assessmentcultivatesstudents'positiveattitudestowardslife. 0.685 0.049 0.02 0.074 0.065 0.059 0.008
20.Assessmentisusedtoprovokestudentstobeinterestedinlearning. 0.591 0.04 0.084 0.066 0.059 0.02 0.048
14.Assessmenthelpsstudentssucceedinauthentic/realworldexperiences. 0.446 0.085 0.105 0.216 0.092 0.14 0.124
13.Assessmentensuresstudentspayattentionduringclass. 0.533 0.066 0.131 0.012 0.007 0.22 0.224
34.Assessmentmeasuresstudents'higherorderthinkingskills. 0.509 0.167 0.007 0.03 0.176 0.11 0.077
Note. Non-zero values on otherfactors, but all weak.
27.Assessmentallowsdifferentstudentstogetdifferentinstruction. 0.487 0.017 0.102 0.128 0.011 0.15 0.213
24.Assessmentstimulatesstudentstothink. 0.678 0.061 0.074 0.008 0.001 0.12 0.105
49.Assessmentforcesteacherstoteachinawayagainsttheirbeliefs. 0.083 0.458 0.03 0.121 0.071 0.19 0.106
31.Assessmentinterfereswithteaching. 0.102 0.54 0.08 0.06 0.086 0.13 0.066
10. Assessmenthaslittleimpactonteaching. 0.134 0.384 0.19 0.034 0.062 0.01 0.067
26.Assessmentisanimpreciseprocess. 0.004 0.629 0.034 0.008 0.021 0.057 0.09423.Assessmentresultsarefiled&ignored. 0.017 0.646 0.01 0.057 0.02 0.022 0.056
45.Teachersconductassessmentsbutmakelittleuseoftheresults. 0.019 0.493 0.045 0.003 0.193 0.008 0.012
EFA steps
1. Run MLE, oblimin allowing eigenvalues>1.00
NB. This is the SPSSpattern matrix of
2. Remove items with cross-loadings >.303. Remove items with no loading >.304. Remove items which did not logically fit their factor5. Remove items that seem literally repetitive in content6. Remove factors that are repetitive in meaning to earlier factorsRESULT
Items kept fit conceptually and have strong unique loadings on 1 factor
regress ons
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
14/23
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
15/23
8/02/2011
15
Additional Causes of Broken Models To be examined in the next session
negative error variances
covar ance ma r ces a are no pos ve e n e
Recommended solutions to be discussed as well
Testing Multiple Models
analyst job is to identify which model fits best and makessense in terms of what we already know and believe
Instrument: Teachers Conceptions of Feedback
Theoretically expected 10 factors
Data: independent samples from Louisiana and NewZealand
comparison of 2 groups, re-analysis of NZ sample
Results: multiple structures and many possible validmodels could fit; better model found in a series of studies
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
16/23
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
17/23
8/02/2011
17
What is Confirmation in CFA? Most studies follow this process
An inventory is developed using theory
The validit of the uestionnaire ma be ex lored
EFA identifies a plausible model within a data set
CFA tests the fit of the EFA model to the data
CFA refines the EFA model with the same data
This process is better considered Restrictive analysis not CFA
True confirmation comes when an existing model is
No EFA needed
Just run the model, does it fit?
If NOT, then EFA must begin again
True Confirmatory Study
TCoA: 9 factors in 4 factor
New Sample: Cyprus primary &secondary teachers
Tested:
CFA NZ Model (original &
simplified);
EFA Cyprus Model;
joint hierarchical model
Result: Model D fits bothgroups satisfactorily Brown & Michaelides, 2011
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
18/23
8/02/2011
18
Developing a structural model (SEM) Identify possible structural paths between important
variables in measurement models
Regression analysis
If theory suggests causal relations use regressions
If no idea, look at correlations
Note. In SEM, a correlation and a regression will have the
.theoretically if there is cause or temporal precedent
Why Use SEM instead of Multiple
Regressions?
Limitations of multiple regressions
only 1 construct can be predicted at a time; its not
The joint correlations among predictor constructs is not takeninto account
The paths from origin to terminus cannot be accounted for
Thus, SEM is better able to test for statistical significance
of regressions
Provided N is large enough
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
19/23
8/02/2011
19
Example: SCoA to Definitions of Assessment(Brown, Irving, et al., 2009)
Hypothesis
beliefs about the nature and purpose of assessment predict the
Multiple regression analysis
2 latent traits were predicted by 8 latent traits in 2 separateanalyses; only 4 were statistically significant
Interactive-Informal assessment practices (R2=.02):
Class Environment, = .12, p = .01,
Assessment is ignored (Ignore), = .10, p = .06. The Teacher-Controlled assessment practices (R2=.08):
Teacher Improves Student Learning, = .14, p = .02, and
Personal Enjoyment, = -.14, p = .003.
Example: SCoA to Definitions of Assessment
SEM
Beta values much
regression values
Proportionvarianceexplained much
higher thanregression
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
20/23
8/02/2011
20
CFA/SEM: Belief
to Belief(Brown, 2009)
CFA: A change in each latent trait predicts a large SEM: Only statistically
change in responses for each contributing variable.Range = .38 to .88; proportion variance explained = 2
, hence 13% to 77%. Relatively low proportion ofunexplained. This is required for good measurement in
CFA.Conclusion: Latent Traits predict responses on
Observed Variables.
significant paths kept inmodel.
Consider This Model
Theory Self-regulation involves increasing adaptive beliefs & practices
and decreasin malada tive ones
Inventory development Multiple studies, multiple versions, multiple samples
Include measure of academic performance N=520; #manifest variables=46; 9 factors; 3 measurement
models; 2 models are hierarchical.
Fit: 2= 2146.58 d=970 2/d=2.21 =.13gamma hat=.91; RMSEA=.048; SRMR=.064; SMC=.20
What beliefs are adaptive or maladaptive to performancein mathematics? Does it matter?
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
21/23
8/02/2011
21
Summary
Theories are used to devise models that attempt toexplain how changes occur in various constructs and in
CFA/SEM mathematical equations are based on linearregressions to identify the strength of relationships
among Latent, Manifest, and Unexplained variables
CFA/SEM models are used to establish validity ofmeasurements and answer substantive questions
CFA/SEM are powerful because of simultaneousproperties and tighter specification of model
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
22/23
8/02/2011
22
Summary Same techniques used to validate measurement models
and explore relations between constructs
equ res arge an sop s ca e ma ema ca ormu ae
Is powerful to test and generate hypotheses
Logically depends on the notion of causation andprediction
Can be done relatively easily with modern software but nd
2 weeks
References: Studies used Brown, G. T. L., Harris, L. R., & Harnett, J. (2010, July). Teachers conceptions of feedback:
Results from a national sample of New Zealand teachers. Paper presented at theInternational Test Commission biannual conference, Hong Kong.
Brown, G. T. L., Harris, L. R., OQuinn, C., & Lane, K. E. (2011, April). New Zealand andous ana rac cng eac ers conce ons o ee ac : m ac o ssessmen o earnng
versus Assessment for Learning policies?Paper accepted for presentation to theClassroom Assessment SIG at the annual meeting of the American EducationalResearch Association, New Orleans, LA.
Brown, G. T. L., & Hirschfeld, G. H. F. (2008). Students conceptions of assessment:Links to outcomes.Assessment in Education: Principles, Policy and Practice, 15(1), 3-17.
Brown, G. T. L., Irving, S. E., Peterson, E. R., & Hirschfeld, G. H. F. (2009). Use ofinteractive-informal assessment practices: New Zealand secondary studentsconceptions of assessment. Learning & Instruction, 19(2), 97-111.
Brown G. T. L. & Michaelides M. 2011 . Ecolo ical rationalit in teachers. . . . .conceptions of assessment across samples from Cyprus and New Zealand. EuropeanJournal of Psychology of Education. doi:10.1007/s10212-010-0052-3
Brown, G. T. L., Peterson, E. R., & Irving, S. E. (2009). Self-regulatory beliefs aboutassessment predict mathematics achievement. In D. M. McInerney, G. T. L. Brown, &G. A. D. Liem (Eds.) Student perspectives on assessment: What students can tell us aboutassessment for learning(pp. 159-186). Charlotte, NC: Information Age Publishing.
8/2/2019 2011 an Introduction to Confirmatory Factor Analysis CFA
23/23
8/02/2011
References: Authorities Ajzen, I. (2005).Attitudes, personality and behavior(2nd ed.). New York:
Open University Press.
Byrne, B. M. (2001). Structural Equation Modeling with AMOS: BasicConcepts, Applications, and Programming. Mahwah, NJ: LEA.
Fan, X., & Sivo, S. A. (2007). Sensitivity of fit indices to modelmisspecification and model types.Multivariate Behavioral Research,42(3), 509529.
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules:Comment on hypothesis-testing approaches to setting cutoff valuesfor fit indexes and dangers in overgeneralizing Hu and Bentler's
(1999) findings. Structural Equation Modeling, 11(3), 320-341. Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more
ever too much? The number of indicators per factor in confirmatoryfactor analysis.Multivariate Behavioral Research, 33(2), 181-220.
Basic Readings on CFA/AMOS . (2007).AMOS. Taipei, Taiwan:.
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratoryfactor analysis: Four recommendations for getting the most from
. , ,Available online: http://www.pareonline.net/pdf/v10n17.pdf.
Klem, L. (2000). Structural equation modeling. In L. G. Grimm & P. R.Yarnold (Eds.), Reading and Understanding More Multivariate Statistics
(pp. 227-260). Washington, DC: APA. Kline, P. (1994).An easy guide to factor analysis. London: Routledge.
Kim, J.-O., & Mueller, C. W. (1978). Factor Analysis: Statistical methodsand practical issues (Vol. 14). Thousand Oaks, CA: Sage
Thompson, B. (2000). Ten commandments of structural equationmodeling. In L. G. Grimm & P. R. Yarnold (Eds.), Reading andUnderstanding More Multivariate Statistics (pp. 261-283). Washington,DC: APA.