Upload
walter-turner
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
1
Class 2Measurement Terminology and
Locating Measures
October 6, 2005
Anita L. StewartInstitute for Health & Aging
University of California, San Francisco
2
From Last Week
Some information based on last week’s discussion – On weights and values
– Does cognitive functioning reflect physical health?
3
Implicit and Explicit “Weighting”
Description (describe health states) Valuation (mapping those health states
onto a set of values) Most measures use “implicit” values Utility or preference measures represent
“explicit” values
Kind P. Values and valuation in the measurement of HRQoL, in P Fayerset al (eds) Assessing quality of life in clinical trials, 2nd ed, Oxford, 2005
4
Example of “Implicit” Weighting: Number of Items per Domain in 3 MOS Indexes
MHI32 MHI17 MHI5Anxiety 7 4 1
Depression 13 8 2Positive affect 7 4 2Belonging/loneliness 5 1 02 summary scores:
Distress 22 12 3Well-being 10 5 2
5
Example of “Implicit” Weighting: Percent of Items per Domain in 3 MOS Indexes
MHI32 MHI17 MHI5Anxiety 22 24 20
Depression 41 47 40Positive affect 22 24 40Belonging/loneliness 16 6 02 summary scores:
Distress 69 71 60Well-being 31 30 40
6
Correlations of MOS Long Form Measures with Physical and Mental Health
Physical Mental
Pain intensity -.71 -.35
Role limitations – emotional -.48 -.66
Cognitive functioning .44 .80
Symptoms -.69 -.51
Energy/fatigue .79 .61
Sleep problems -.54 -.63
Current health perceptions .77 .50
Health distress -.70 -.63
Hays RD and Stewart AL, The structure of self-reported health inchronic disease patients, Psychol Assessment, 1990;2:22-30.
7
Overview of Class 2
Measurement terminology– Confusion, variation, and overlap
Types of response scales, contribution to concept definition
Measurement models– Depicting structure of measures
Locating measures
8
Measurement
Assignment of numbers to aspects of objects or events according to a rule
Assignment of numbers to an unobservable characteristic of individuals (construct)
9
Construct, Concept, Variable, Latent Trait, Latent Variable
Terms defined differently by different investigators/authors
Meaning is very similar– considerable overlap among them,
Often used interchangeably
10
Construct/Concept
A variable that is relatively abstract as opposed to concrete and is defined or operationalized in terms of observed indicators
An idea developed or constructed through informed scientific theory. Concepts that are directly or indirectly observable.
An abstraction based on observations of certain behaviors or characteristics (e.g. pain, stress)
11
Latent Trait, Factor
Latent - present but not visible, unobservable Latent trait - unobservable set of characteristics
that can be empirically inferred and estimated through answers to a set of questions
(Medical Care glossary) Factor - a latent variable or theoretical construct
operationalized in terms of associations among indicators in a factor analysis
(Stewart and Ware, 1992 glossary)
12
Latent Variable, Manifest Variable
Latent variable - unobservable latent trait given a label
Manifest variable is the “observed” information, i.e., the answers to items used to infer the latent trait
13
Domain, Dimension, Sub-domain, Component
Domain: a state, attitude, perception, behavior, or other sphere of action or thought related to health …. All entities in a single domain have some property in common ….
(Patrick & Erickson, 1993) Dimension: a distinct component of a
multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)
(Stewart & Ware, 1992, glossary)
14
Example: Dimension, Component, Sub-domain
Mental health (dimension)– Psychological distress (component)
» Depression (sub-domain)» Anxiety
– Psychological well-being» Positive affect» Sense of belonging
15
Components of Domains of MOS Long-form Measures
Role Role limitations due to:Functioning -- physical health
-- emotional problems
Social Social limitations due to healthFunctioning Sexual problems
Family functioningMarital functioning
Pain Pain severityEffects of painDays with pain
16
Multidimensional and Unidimensional
A multidimensional “measure” or “instrument” provides several scores– A “profile” of scores
Unidimensional measure - all items measure only one concept– Only one score is obtained
Dimensionality must be empirically tested– e.g., factor analysis identifies one factor,
or more than one factor
17
Example of Multidimensional Instrument
Patient Satisfaction Questionnaire (PSQ) 55 items, 18 subscales, e.g.
– Access to care– Technical quality– Interpersonal manner– Explanations– Continuity of care
Marshall, GN Psychological Assessment, 5:477-483, 1993
18
Example of Unidimensional Measure
Perceived Stress Scale (PSS) 14 items, subjective experiences of stress
– felt confident could handle life’s problems
– able to control irritations in your life
– difficulties piling up so high, could not overcome them
Single score from all itemsCohen, S, J Health Soc Behav 24:385-396, 1983
19
Instrument, Profile, Battery, Measure, Questionnaire, Survey, Tool, Inventory Instrument - typically refers to a published,
named measure or set of measures Profile - same, but all scores on the same
metric and can be compared (0-100 scales) Battery (of independent measures)
– collection of measures from diverse sources Measure - single- or multi-item scale or
index (umbrella term)
20
Example of a Multidimensional Profile Instrument: SF-36Measures 8 health concepts (36 items): Physical functioning Role limitations due to physical health Role limitations due to emotional problems Social functioning Pain Vitality (energy/fatigue) Mental health Health perceptions
21
Generic vs. Disease-Specific Approaches
Generic– Applicable across many conditions and
diseases (e.g., sleep, health perceptions, fatigue, pain)
Disease-specific– Specific to particular disease or condition
(e.g., arthritis, cancer, depression)» Generic approach (e.g., general dexterity scale for arthritis
study)» Specific (nausea due to chemotherapy, back pain)
22
Generic or Disease Specific bySingle Domain, Profile, or Index
Single domain
Profile Summary index
Generic Pain
Fatigue
SF-36
SIP
PCS
MCS
QWB
Disease specific
Nausea
Back pain
Diabetes impact
23
Questionnaire, Survey, Tool, Inventory
Questionnaire - any of the above formatted into a “questionnaire” or set of measures
Survey - same as questionnaire Tool = Questionnaire = Survey? Inventory?
24
Examples of Names of “Instruments”
SF-36 Short Form Health Survey Beck Depression Inventory Health Assessment Questionnaire Sickness Impact Profile Center for Epidemiological Studies
Depression Scale
25
Example of a Battery
A survey for a particular study containing the following measures/instruments:– SF-36 (8 domains)– Perceived stress scale (1 domain)– Social support survey (4 domains)– Pain measure
Typical in clinical trials, epidemiological studies– Each measure serves particular purpose (outcome,
predictor, mediator, covariate)
26
Measure, Scale, Index, Item
Measure - single- or multi-item scale or index (umbrella term)
Scale - aggregation of items from one concept or domain, scored according to an accepted scaling method
Index - aggregation of 2 or more scales into a summary score
Item - a single question or statement including its response scale
27
Item
Consists of an item stem and a response scale or response choices
Item stem – – The question or statement
Response scale or choices –– Answers presented to a respondent with which to
respond to a (closed-ended) question
28
Example of a Response Scale
How often have you felt tired inthe past week? 1 Never 2 Once or twice 3 A few times 4 Fairly often 5 Very often
Responsescale
Itemstem
29
Response Choice Formats
Verbal descriptors of all levels Verbal descriptors of endpoints only Verbal descriptors of endpoints plus
midpoint Visual analogue scales:
10cm line with endpoints labeled
30
Format: Verbal Descriptors of All Levels
1 - All of the time2 - Most of the time3 - Some of the time4 - A little of the time5 - None of the time
31
Vague, Imprecise Quantifiers
How often?– very often, pretty often, not too often
– Sometimes, often, never How much?
– Too little, about right, too much
– Below average, average, above average
Bradburn NM, Vague quantifiers. Public Opinion Quart 1979, 92-101.
32
Intensity Scales – “How distressed have you been?”
Not at all, slightly, moderately, quite a bit, extremely
Not at all, a little, a fair amount, much, very much
Not at all, a little, fairly, quite, very, extremely
Not at all, a little, somewhat, very much
33
“How Often” Response Scales
1 - Never
2 - Once or twice
3 - A few times
4 - Often
1 – Never
2 - Hardly ever
3 - Some days
4 - Most days
5 - Almost every day
6 - Always, every day
34
“How Often” Response Scales
1 - Never
2 - Almost never
3 - Sometimes
4 - Fairly often
5 - Very often
6 - Always
1 - Once or twice
2 - A few times
3 - Fairly often
4 - Very often
5 - Almost every day
6 - Every day
35
Format: Verbal Descriptors of Endpoints Only
0 1 2 3 4 5 6 7 8 9 10 No Pain pain as bad
as you can imagine
36
Format: Verbal Descriptors of Endpoints with Midpoint Labeled
Very Very
much 1 2 3 4 5 6 7 muchworse No better
change
37
Format: Verbal Descriptors of Endpoints with Midpoint Labeled
-3 -2 -1 0 1 2 3Very No Very much change muchworse better
39
Optimal Number of Response Choices
Are 100 levels better than 5 or 10? 5-points is probably optimal for respondents
– Especially if items will be combined into multi-item scales
For single-item measures– 7 points may be preferred– Allows distribution over 5 points since people
seldom use endpoints Hard to come up with labels for 7-point scales
40
Format: Verbal Descriptors of Endpoints Only
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
No Pain pain as bad
as you can imagine
41
Magic Number 7 + 2
George Miller determined that 7 “chunks” of information is the most that short-term memory can retain– Plus or minus two (5-9 possible)
Seven (+ 2) is also the most points that people can discriminate along a continuum
Principle has held up well in measurement
Miller, GM Psychological Bulletin, 63:81-97, 1956
42
Can You Make a 7-point Intensity Scale? (How physically active were you this week?)
1 - Not at all active
2 - A little bit active
3 - Moderately active
4 - Quite active
5 - Extremely active
1 - Not at all active
2 - A little active
3 - Fairly active
4 - Quite active
5 - Very active
6 - Extremely active
43
Measurement Models
The dimensional structure of a measure in terms of how the items related to the constructs (latent traits, factors, concepts)
Can be depicted as a list or visually Standard visual format for structural
equation modeling (SEM)
45
Measurement Model (List format)
Physical Functioning defined in terms of:– Walking
– Climbing stairs
– Bending
– Reaching
47
Measurement Model (List format)
Psychological distress– Depression
» Sad» Lost interest» Can’t get going
– Anxiety» Restless» Nervous
48
Measurement Model (Visual format)
Psychological Distress
Depression Anxiety
SadLost
interestCan’t
get goingRestless Nervous
50
Content Areas: Four Depression Measures
CES-D HAM-D Beck GDS
Sleep problems X X X X
Concentration X X
Fatigue, energy X X
Appetite loss X X
Depressed mood X X X
Anxiety, restlessness X X
Irritability X X
Sense of failure X X
Social withdrawal X X
Guilt X
Self-criticism X X
51
Content Areas: Four Standard Measures of Physical Functioning
AIMS MOS HAQ SIP
Walking X X X X
Climbing stairs or steps X X X X
Bending, kneeling X X X X
Lifting, carrying X
Getting out of bed X
Bathing X X X
Running errands X
Opening jars X
Vigorous activities X X X
52
MOS SF-36 versus SF-12:Physical Functioning
SF-36 SF-12 Vigorous activities X Moderate activities X X Lift/carry X Climb several flights stairs X X Climb one flight stairs X Bend/kneel/stoop X Walk > 1 mile X Walk several blocks X Walk 1 block X Bathe/dress X
53
MOS SF-36 versus SF-12:Mental Health Items
SF-36 SF-12 Nervous X Down in the dumps X Calm and peaceful X X Downhearted and blue X X Happy person X
54
Sequence of Steps in Developing New Self-Report Measures
Develop concept
Create item pool
Pretest/revise
Field survey
Psychometric analyses
Final measures
55
Locating Specific Measures
Compendia Web Organizations National surveys Large research studies Many other sources
56
Locating Specific Measures
Compendia of measures (reviewed)– Books that compile various measures and
review their characteristics (e.g., McDowell and Newell)
Web sources of Compendia– HaPI-Health and Psychosocial Instruments– Measurement Excellence and Training Resource
Information Center (METRIC)
57
Locating Specific Measures
Organizations– RAND publishes measures, scoring manuals, and
lists citations– University of Michigan, Survey Research Center
National and State Surveys– NHIS, NHANES, HHANES, BRFSS, CHIS, MEPS– Entire surveys can be downloaded– Seldom includes measurement information
58
Locating Specific Measures
Large research studies on similar topic– Health ABC, CARDIA, WHI– EPESE
Quality tools (AHRQ)– Consumer Assessment of Health Plans Survey
(CAHPS)– National Quality Measures Clearinghouse– National Healthcare Quality Report– National Healthcare Disparities Report
59
Locating Specific Measures
National Institutes of Health – NCI has a measurement initiative
Health Information National Trends Survey (HINTS)– Working group compiled cancer screening
questions, identified best ones, conducted extensive pretesting, cognitive interviewing
– Measures are on the NCI web site
60
Locating Specific Measures
Projects/Centers– Toolkit to measure end of life care (TIME)– Stanford Patient Education Research Center– Michigan Diabetes Research and Training Center– Resource Centers for Minority Aging Research
Proprietary measures– SF-36, SF-12 (have to pay for instruments, scoring
manuals)
61
Locating Specific Measures: Bibliographic Searches
Published measurement articles– Medline Searches
– MESH headings or keywords:» health status indicators
» outcome assessment (health care)
» psychometrics (methods)
» questionnaires
62
Locating Specific Measures: Finding Authors of Measures
Published research using measure you are interested in– Unpublished measures often described in
methods– Authors may provide measures