Upload
catherine-heath
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
2
ContentContent
• Introduction
• Scale development methods
• Scale evaluation methods
• Statistical analysis of scale data
3
§1 Introduction
1 Definition
Measurement instrument that are collections of items combined
into a composite score, and to reveal levels of a certain state,
behavior or attitude of the research objects are referred to as scale.
4
Example24-1 How to evaluate the treatment of pain? Or how can we measure different degree of pain?
6
Ways to obtain the variable value:
1) Measurement ----to get the
quantitative data
2) questionnaire or inquiring---- to
get the quantitative or
qualitative data
7
Structure design of a scale is similar to a questionnaire, but they are different:
questionnaire ----different independent contents can be included in
scale----describe one characteristic of the object, and the items are associated with each other
8
2 Scales are used in the situations of:
measure variable that can not assess directly. Eg:
some physical, psychological and social
characteristics.
1) Variables can not be measured directly
2) Nonobjective conceptions and attitudes
3) Complicated behaviors or psychological states
9
3 merits and demerits of the scale
measurement
merits : Strong objectivity, standardized process,
comparable and easy to implement
demerits : large variation with different
individual, high quality are request in scale
development
10
1 Structure of a scale
Scale----domain(subscale) ---- facet
---- item
§2 Methods of scale development
11
2 principle of scale development
An ideal scale: exactly measure the aim
characteristics, and obtain reliable data
( 1 ) principle of suitability
( 2 ) principle of validity
( 3 ) principle of feasibility
12
3 Approaches of scale development1) Make sure the research aim and measurement content
nominal group
focus group
2) Define dimensions and facets
13
3) Establish item pool and select items
4) Design operable items
Make sure the format of items and response
14
Table 24-1 response scale analysis of importance quantifiers
quantifiers Average score
quantifiers Average score
Very unimportant 0.90 important 5.96
unimportant 1.10 More important 7.14
More unimportant 2.40 Quite important 8.04
important 4.41 Very important 8.59
Some important 4.72 highly important 8.69
17
4 quantitative analysis and selection of items
Similar to the indicator selection in
comprehensive evaluation, a good item should
have good importance, sensitivity, independency,
and acceptability,and be representative, ascertain.
18
Methods of item selection:
( 1 ) subjective evaluation method--importance
( 2 ) discrete tendency method—sensitivity
( 3 ) correlation coefficient method–
representative and independency
( 4 ) principal components analysis and
factor
analysis method-- representative
19
( 5 ) clustering analysis method--
representative
( 6 ) stepwise selection method based on
importance
( 7 ) stepwise regression -- importance
( 8 ) stepwise discriminant method –
discriminative ability
20
Example 24-2 Analyze the quality of
12 items of 3 facets (ache, energy and
sleeping) in physical domain of 206 cases
of general people and hypertension
patients
21
Table 24-2 results of item selectionitem Correlation
coefficient CV Factor loadings Stepwise regression
Stepwise discrimin
ant
(%) F1 F2 F3
F11 -0.166 * 40.90 0.222 0.013 -0.478
F12 -0.226 ** 42.51 0.762 -0.046 -0.143
F13 -0.240 ** 44.25 0.812 -0.069 -0.175
F14 -0.089 40.58 0.706 0.020 -0.265
F21 0.243 ** 42.06 -0.112 0.233 0.801
F22 -0.181 * 39.09 0.657 -0.255 -0.197
F23 0.227 ** 32.24 -0.126 0.224 0.812
F24 -0.276 ** 41.15 0.741 -0.293 -0.016
F31 0.212 ** 32.14 0.025 0.830 0.297
F32 -0.317 ** 51.74 0.512 -0.665 -0.017
F33 0.237 ** 35.24 -0.037 0.824 0.308
F34 -0.316 ** 55.73 0.480 -0.669 0.112
22
5 Notice
1) Quantity of items 30-50
2) wording of items should be explicit and
material
3) Objective index and subjective index
24
1 qualitative evaluation of scale
experts colloquia or experts consultation
Improve the structure of the scale, wording of
items, item selection, and so on.
25
2 reliability analysis
Reliability is used to evaluate the accuracy,
stability and consistency of a scale.
( 1 ) test-retest reliability ≥: 0.7。( 2 ) split-half reliability : ( 24-1
)( 3 ) Cronbach’s alpha coefficient : ( 24-2 )K-number of items , -the score variance of the ith
item , -the total score variance of a scale
2
1
rR
r
2
2(1 )
1i
T
Sk
k S
2iS 2
TS
26
Example 24-4 A doctor measured Quality of life
with 50 people using WHOQOL-100 , and a retest
was done one week later, results were showed in
table 24-4.
27
…
表 24-4 50 名正常人生存质量调查得分
Q1 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14 F15 F16 F17 F18 F19 F20 F21 F22 F23 F24 T1 T2
80 10 11 9 14 12 14 9 6 13 10 4 16 12 13 13 13 14 10 14 14 13 11 10 13 278 296
80 7 12 11 11 15 15 13 9 18 12 4 17 13 13 10 10 9 9 11 9 12 11 8 13 272 257
90 6 12 10 13 15 16 9 8 18 12 4 18 11 14 8 11 12 12 13 12 15 11 10 13 283 270
98 8 11 10 17 18 19 12 8 17 11 4 20 13 18 12 12 8 8 11 12 17 11 10 19 306 317
60 12 13 13 11 13 16 9 10 14 9 8 15 10 7 13 11 10 8 11 13 14 10 14 11 275 281
75 14 11 11 8 13 7 12 14 13 13 7 14 8 10 10 7 9 12 8 10 9 6 11 4 241 257
90 8 12 11 15 16 16 12 7 19 11 4 17 12 15 9 13 7 7 11 15 15 5 8 15 280 259
60 6 12 9 12 13 12 9 11 18 11 5 16 12 13 17 11 15 11 14 12 13 8 9 6 275 278
96 5 10 9 12 14 14 12 7 12 12 4 16 11 14 11 11 14 8 12 13 15 10 9 8 263 264
70 9 12 10 10 14 13 11 10 12 11 4 18 12 14 11 11 4 9 15 16 12 9 11 9 267 284
50 9 12 10 7 9 9 10 13 16 10 5 12 12 8 10 10 5 8 11 11 10 10 9 6 232 243
75 10 13 11 13 12 16 13 9 18 12 5 17 13 14 13 10 4 10 11 12 12 11 9 16 284 304
60 7 9 11 10 13 14 11 9 13 11 4 17 12 14 10 10 4 11 9 10 13 8 9 18 257 264
70 8 10 9 11 12 12 9 10 14 11 4 16 11 11 11 10 7 9 9 8 8 10 11 11 242 238
80 8 13 12 16 16 16 12 8 14 12 4 16 14 13 13 14 12 11 13 12 14 11 11 16 301 328
Table 24-4 QOL score of 50 persons
28
Results:
① test-retest reliability : r=0.82;
②mean difference =3.87 , t=1.544 , P=0.129 ;
③ Cronbach’s alpha coefficient :
d
24 (6.2551 3.3571 13.8371 24 151.3710(1 ) (1 ) 0.80
23 661.3588 23 661.3588
29
3 validity analysis of a scale
Validity is used to evaluate the veracity,
validity of a scale. The accordance of
practice results and the anticipate results.
30
1) content validity
2) criterion-related validity
3) contract validity
(evaluating:confirmatory factor analysis, CFA)
31
4 responsibility analysis of a scale
Responsibility is the capability of a scale to observe the changes
between different objects or the changes with in an object at
different time. The statistic is effect size:
( 24-3)
: Scale socre
X
X X
S
af ter treatment before treatment
before treatment
Example 24-5
33
1 characteristic of scale data
1) grouping the objects and compare the scores
between groups;
2) Repeated measure data;
3) multi-domain data
36
Statistical inference1) For situation if one time point, more than one
group
one variable analysis: t-test, F test, rank sum test
37
some comprehensive evaluation methods:
Fuzzy discriminant method, O’Brien
synthetic method, rank sum method,
TOPSIS method.
38
2) Compare of longitudinal data
Hotelling T2 test, multi-variable variance analysis,
variance analysis of repeated measure, profile
analysis
40
4 Application of scalespsychology, pedagogy, and sociology----
medicine
( 1 ) psychology and psychiatry ( 2 ) evaluation of disease treatment in clinical research