Embed Size (px)
DESCRIPTION
Multilevel Multitrait Multimethod model. Lluís Coromina (Universitat de Girona) Barcelona, 06/06/2005. Background. Measurement data quality in social networks analysis. Assess reliability and validity in egocentered social networks. Complete networks Egocentered networks. Index. - PowerPoint PPT Presentation
Citation preview
Multilevel Multitrait Multimethod model.
Lluís Coromina (Universitat de Girona)
Barcelona, 06/06/2005
• Measurement data quality in social networks analysis.
• Assess reliability and validity in egocentered social networks.
• Complete networks
• Egocentered networks
Background
Index
• Reliability and Validity
• MTMM Model
• Data
• Multilevel analysis
• Results and interpretation
Reliability and Validity
MTMM ModelConfirmatory Factor Analysis (CFA) specification of the MTMM model.Yij = mij Mj + tij Ti + eij (1)
where:• Yij : response or measured variable “i” measured by method “j”.• Ti : unobserved variable of interest (trait). Related to validity.• Mj : variation in scores due to the method. Related to invalidity. • mij and tij : factor loadings on the method and trait factors.• eij : random error, which is related to lack of reliability.
Reliability and Validity
Figure 1 : Path diagram for the MTMM model for trait (Ti) and method (Mj).
MTMM model
mij
Yij
Ti
Mj
tij
eij
mij
Yij
Ti
Mj
tij
eij
Traits
T1 Frequency of contact
T2 Feeling of closeness
T3 Feeling of importance
T4 Frequency of the alter upsetting to ego
Methods
M1 Face-to-face interviewing
M2 Telephone interviewing
MTMM model
Y11 Y21 Y31 Y41 Y22Y12 Y32 Y42
M1 M2
T1 T2 T3 T4
e11
e21 e31e41 e12
e22 e32
e42
Figure 2: Path diagram of a CFA MTMM model for two methods and four traits.
MTMM model
MTMM model
Var (Yij) = mij2Var (Mj) + tij
2Var (Ti) + Var (eij) (2)
Validity and Reliability for CFA MTMM model:
Reliability coefficient = (3)
Validity coefficient = (4)
)(
)()( 22
ij
iijjij
YVar
TVartMVarm
)()(
)(22
2
iijjij
iij
TVartMVarm
TVart
Kogovšek, et al., 2002: Estimating the reliability and validity of personal support measures: full information ML estimation with planned incomplete data. Social Networks, 24, 1-20.
T1 Frequency of contact T3 Feeling of importance
T2 Feeling of closeness T4 Frequency of the alter upsetting to ego
G N First interview Second interview
314 1371 M1 Face-to-face M2 Telephone
Table 1: The design of the study
Data
Representative sample of inhabitants of Ljubljana
Multilevel analysisTwo-level MTMM model.
The highest level: group level = egos = gThe lowest level: individual level = alters = k
Multilevel MTMM model
The mean centred individual scores for group “g” and individual “k”
can be decomposed into:
Within group component (5)Between group component (6)
where:• is the total average over all alters and egos.• is the average of all alters of the gth ego. • Ygk is the score on the name interpreter (questions) of the kth alter chosen by the gth ego.• G is the total number of egos. • n is the number of alters within each ego.• N=nG is the total number of alters.
Y
gY
YYY gkgkT
YYY ggB
ggkWgk YYY
Multilevel MTMM model
Sample covariance matrices:
Multilevel MTMM model
GN
YYYY ggkggk
nG
)')((SW=
1
)')((
G
YYYYn gg
G
SB=
1
)')((
N
YYYY gkgk
nG
ST = SB + SW =
(7) (8)
(9)
Population covariance matrices: T = B + W (10)
Yij = mBijMBj + tBijTBi + eBij + mwijMwj + twijTwi + ewij (11)
YBij YWij
Härnqvist MethodSeparate analysis for SB and SW
Group measureSw is the ML estimator of ΣW
SB is the ML estimator of ΣW+cΣB (12)
Multilevel MTMM model
Model estimated by Maximum Likelihood (ML).
ΣB
Σw
c
ewij
twij mwij
Ywij
Mwj Twi
mbij
Ywij
Mbj Tbi
Ybij
Mwj Twi
ebij
SW SB
ewij
twij mwij
tbij
Figure 3: Multilevel CFA MTMM Model.
Multilevel MTMM model
Interpretation:
We can obtain 2 reliabilities and 2 validities for each trait-method combination.
To analyse each component separately:
Yij = mBijMBj + tBijTBi + eBij + mwijMwj + twijTwi + ewij (11)
YBij YWij
Decompose the variance:Var (Yij) = mij
2wVar (MjW) + mij
2BVar (MjB) +
tij2
wVar (TiW) + tij2
BVar (TiB) + (13)
Var (eijw) + Var (eijB)
Multilevel MTMM model
Analysis:
Multilevel MTMM model
Analysis 1: traditional analysis on ST. ML estimation.
Analysis 2: traditional analysis on SW. ML estimation.
Analysis 3: traditional analysis on SB, which is a biased estimate of ΣB. ML
estimation. Analyses 2 and 3 together constitute the recommendation of Härnqvist (1978).
Analysis 4: multilevel analysis, to fit ΣW and ΣB simultaneously. ML estimation.
Table 1: Goodness of fit statistics.
Results and interpretation
Analysis
1 (ST) ML
2 (SW) ML
3 (SB) ML
4 (ΣT and ΣW) ML
Initial χ2 statistic 190.934 112.104 81.807 155.099 d.f. () 15 15 15 27 RMSEA 0.093 0.079 0.119 0.083 Respecifications
var(M2T)=0
var(M2W)=0
var(M2B)=0
ti2b=1 var(M1B)=0 var(M2W) = 0 var(e41B) =0
χ2 statistic 192.852 112.295 82.377 215.939 d.f. () 16 16 16 34 RMSEA 0.090 0.076 0.115 0.088
Table 2: Decomposition into 6 variance components. Analysis 4.trait variance within T1 T2 T3 T4
M1 0.79 0.56 0.67 0.47
M2 0.76 0.55 0.64 0.45
method variance within*
M1 0.03 0.03 0.03 0.03
M2 0.00 0.00 0.00 0.00
error variance within
M1 0.16 0.16 0.18 0.22
M2 0.14 0.13 0.13 0.17
trait variance between
M1 0.18 0.07 0.11 0.13
M2 0.18 0.07 0.11 0.13
method variance between*
M1 0.00 0.00 0.00 0.00
M2 0.01 0.01 0.01 0.01
error variance between*
M1 0.02 0.03 0.03 0.00
M2 0.04 0.02 0.02 0.06
* Boldfaced for small non-significant variances constrained to zero.
Results and interpretation
Table 3: Decomposition into 6 variance components*trait variance within T1 T2 T3 T4
M1 67,2% 66,0% 66,3% 55,2%
M2 67,5% 70,2% 70,6% 55,0%
method variance within*
M1 2,6% 3,6% 3,1% 3,7%
M2 0,0% 0,0% 0,0% 0,0%
error variance within
M1 13,3% 19,2% 17,4% 26,0%
M2 12,6% 17,4% 14,7% 20,6%
trait variance between
M1 15,3% 8,1% 10,5% 15,1%
M2 16,0% 9,0% 11,6% 15,6%
method variance between*
M1 0,0% 0,0% 0,0% 0,0%
M2 0,8% 1,2% 1,0% 1,1%
error variance between*
M1 1,5% 3,1% 2,7% 0,0%
M2 3,2% 2,2% 2,1% 7,7%
* Boldfaced for small
non-significant variances constrained to zero.
Results and interpretation
Within level Between level Overall level
T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4
Reliability coef
M1 0,92 0,89 0,90 0,84 0,95 0,85 0,89 1,00 0,92 0,88 0,89 0,86
M2 0,92 0,90 0,91 0,85 0,92 0,91 0,93 0,83 0,92 0,90 0,91 0,85
Validity coef
M1 0,98 0,97 0,98 0,97 1,00 1,00 1,00 1,00 0,98 0,98 0,98 0,98
M2 1,00 1,00 1,00 1,00 0,98 0,94 0,96 0,97 1,00 0,99 0,99 0,99
Table 4: Multilevel reliabilities and validities*
Results and interpretation
* Boldfaced for small non-significant variances constrained to zero.
Table 5: Within part. Comparison of analyses 2 (SW) and 4 (multilevel).
* Boldfaced for small non-significant variances constrained to zero.
Results and interpretation
Analysis 2 Analysis 4 T1 T2 T3 T4 T1 T2 T3 T4 Reliability coefficients M1 0.92 0.89 0.90 0.84 0.92 0.89 0.90 0.84 M2 0.92 0.90 0.91 0.85 0.92 0.90 0.91 0.85 Validity coefficients* M1 0.98 0.97 0.98 0.97 0.98 0.97 0.98 0.97 M2 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Trait correlations T1 1.00 1.00 T2 0.57 1.00 0.57 1.00 T3 0.58 0.99 1.00 0.58 0.99 1.00 T4 0.41 0.26 0.31 1.00 0.41 0.25 0.31 1.00
Table 6: Between part. Comparison of analyses 3 (SB) and 4 (multilevel).
* Boldfaced for small non-significant variances constrained to zero.
Results and interpretation
Analysis 3 Analysis 4 T1 T2 T3 T4 T1 T2 T3 T4 Reliability coefficients* M1 0.92 0.85 0.86 0.93 0.95 0.85 0.89 1.00 M2 0.92 0.87 0.91 0.84 0.92 0.91 0.93 0.83 Validity coefficients* M1 0.98 0.97 0.97 0.97 1.00 1.00 1.00 1.00 M2 1.00 1.00 1.00 1.00 0.98 0.94 0.96 0.97 Trait correlations T1 1.00 1.00 T2 0.23 1.00 -0.18 1.00 T3 0.35 0.98 1.00 0.14 0.97 1.00 T4 0.27 -0.03 0.07 1.00 0.18 -0.34 -0.16 1.00
Table 7: Overall analysis. Comparison of analyses 1 (ST) and 4 (multilevel).
* Boldfaced for small non-significant variances constrained to zero.
Results and interpretation
Analysis 1 Analysis 4 T1 T2 T3 T4 T1 T2 T3 T4 Reliability coefficients M1 0.93 0.88 0.91 0.86 0.92 0.88 0.89 0.86 M2 0.93 0.91 0.93 0.86 0.92 0.90 0.91 0.85 Validity coefficients* M1 0.98 0.97 0.98 0.97 0.98 0.98 0.98 0.98 M2 1.00 1.00 1.00 1.00 1.00 0.99 0.99 0.99 Trait correlations T1 1.00 1.00 T2 0.46 1.00 0.46 1.00 T3 0.50 0.99 1.00 0.50 0.99 1.00 T4 0.36 0.15 0.22 1.00 0.36 0.15 0.22 1.00
T1 T2 T3 T4
tij2
wVar(Tiw)/ [tij2
wVar(Tiw) + tij2
BVar(TiB)] 0.83 0.89 0.87 0.85
0.80 0.88 0.85 0.76
Table 8: Percentages of variance at within level form M1 and M2
Results and interpretation
T1 T2 T3 T4
Var(eijw)/ Var(Yij) 0.13 0.19 0.17 0.26
0.13 0.17 0.15 0.21
Contribution:
• To consider egocentered networks as hierarchical data.
• To specify a multilevel MTMM.
• Interpretation from measurement theory of different % of variance.
Results and interpretation
For further information and contact:
http://www.udg.es/fcee/professors/llcoromina
