MEASUREMENT IN HEALTH BEHAVIOR
-
Upload
others
-
View
3
-
Download
0
Embed Size (px)
Citation preview
Colleen Konicki Di Iorio
File Attachment
Measurement in Health Behavior
MEASUREMENT IN HEALTH BEHAVIOR
Colleen Konicki Di Iorio
Copyright © 2005 by John Wiley & Sons, Inc. All rights
reserved.
Published by Jossey-Bass A Wiley Imprint 989 Market Street, San
Francisco, CA 94103-1741 www.josseybass.com
No part of this publication may be reproduced, stored in a
retrieval system, or transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, scanning, or
otherwise, except as permitted under Section 107 or 108 of the 1976
United States Copyright Act, without either the prior written
permis- sion of the publisher, or authorization through payment of
the appropriate per-copy fee to the Copyright Clearance Center,
Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax
978-646-8600, or on the Web at www.copyright.com. Requests to the
publisher for permission should be addressed to the Permis- sions
Department, John Wiley & Sons, Inc., 111 River Street, Hoboken,
NJ 07030, 201-748-6011, fax 201- 748-6008, or online at
http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and
author have used their best efforts in preparing this book, they
make no representations or warranties with respect to the accuracy
or completeness of the contents of this book and specifically
disclaim any implied warranties of merchantability or fitness for a
particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and
strategies contained herein may not be suitable for your situation.
You should consult with a professional where appropriate. Neither
the publisher nor author shall be liable for any loss of profit or
any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
Readers should be aware that Internet Web sites offered as
citations and/or sources for further information may have changed
or disappeared between the time this was written and when it is
read.
Jossey-Bass books and products are available through most
bookstores. To contact Jossey-Bass directly call our Customer Care
Department within the U.S. at 800-956-7739, outside the U.S. at
317-572-3986, or fax 317- 572-4002.
Jossey-Bass also publishes its books in a variety of electronic
formats. Some content that appears in print may not be available in
electronic books.
Library of Congress Cataloging-in-Publication Data Di Iorio,
Colleen Konicki, 1947-
Measurement in health behavior : methods for research and education
/ Colleen Konicki Di Iorio.
p. ; cm. Includes bibliographical references and index. ISBN
0-7879-7097-2 (alk. paper) 1. Health behavior--Measurement. 2.
Health
behavior--Research--Methodology. [DNLM: 1. Health Behavior. 2. Data
Collection. 3. Reproducibility of
Results. W 85 D575m 2005] I. Title. RA776.9.D54 2005
362.1072--dc22
2005027390
Printed in the United States of America FIRST EDITION
PB Printing 10 9 8 7 6 5 4 3 2 1
01_970972 ffirs.qxd 9/30/05 9:31 AM Page vi
Preface xxi
1 Introduction to Measurement 1
Role of Measurement in Health Education and Health Behavior
Research 1
Brief History of Psychosocial Measurement 3
Conceptualization of Measurement 6
Scales of Measurement 9
Major Concepts of Measurement: Reliability and Validity 16 •
Reliability 16
• Validity 17
Summary 17
2 Types of Measures 18
Self-Report 19 • Interviews 19
• Indexes 31
Observations 33
• Systematic Error 39
Response Sets 42
Social Desirability 42
Positive Skew 45
Strategies to Reduce Errors During Interviews 51
• Situational Factors 51
Measurement Rules 53
4 Survey Development 54
• Objectives 56
• Respondents 59
General Principles of Item Writing 59 • The Do’s of Item Writing
60
• The Don’ts of Item Writing 68
Survey Format 70
Survey Administration 72
Test Construction 76 • State the Purpose of the Test 76
• State the Test Objectives 78
• Review Content 79
• Domain-Sampling Model 80
• True/False Items 90
Average Congruency Percentage 98
6 Theory and Measurement 103
Linking Measurement to Theory-Based Health Practice and Research
104 • Gibbs’s Model 106
• Theoretical and Operational Definitions 109
Conceptualization Issues in Scale Development 110 • Survey Versus
Scale 110
• Single-Item Versus Multiple-Item Scales 111
• Concept Selection 113
02_970972 ftoc.qxd 9/19/05 6:01 PM Page ix
Concept Analysis 114 • Identify Definitions and Uses of the Concept
114
• Identify Critical Attributes of the Concept 116
• Identify Dimensions of the Concept 117
• Identify Similar and Different Concepts 117
• Identify Antecedents and Consequences of the Concept 118
• Write a Model Case 118
• State the Variables 119
Item Development Process 122 • State the Theoretical Definition
123
• Identify Dimensions of the Concept 123
• Create a Content-Domain by Concept-Dimension Matrix 123
• Decide How Many Items to Include for Each Matrix Cell 124
• Write Items for Each Content-Domain by Concept-Dimension Cell
126
Literature Review 126
Qualitative Interviews 126
Other Instruments 127
Summated Rating Scales 128 • Types of Response Options 128
• Number of Response Options 130
• Odd or Even Number of Categories 131
• Meanings of Response Option Adjectives or Adverbs 132
• Use of Adjectives, Adverbs, or Numbers, or Combinations Thereof
132
• Positively and Negatively Worded Items 133
• Scoring 133
Summary 135
Basic Statistical Concepts 137 • Frequency Distribution 137
Measures of Central Tendency 139
Measures of Dispersion 139
• SPSS Commands 140 Recode Negatively Worded Items Using SPSS
144
Compute Total Scale Scores Using SPSS 145
x Contents
Review of Correlation 146 • Interpretation of the Correlation
Coefficient 150
• Correlation Matrix 152
• Correlation Issues 152
Analysis of Variance 157 • Definition and Use 157
• SPSS Commands for a One-Way ANOVA 158
• Interpreting a One-Way ANOVA 158
Summary 162
Classical Test Theory 165 • Observed, True, and Error Scores
165
• Assumptions and Characteristics 167
• Calculation of Variance Due to the True Score 173
• Calculation of the Reliability Index 174
Summary 174
• Stability 178
Factors Associated with Coefficient Alpha 191 • Test Length and
Interitem Correlation 191
• Test Adjustments to Increase Alpha 191
• Standards of Reliability 193
Item Analysis 193
Intraclass Correlation Coefficient 203
Generalizability Theory 206
• Content Validity Index 218
Think Aloud 221
Verbal Probe 222
• Construct Validity 229
• Multitrait-Multimethod Matrix 233
• Validity Issues 235
• Conceptual Basis 241
Steps in Factor Analysis 244 • Basic Requirements for FA 246
• Initial Assessment 247
Test for an Identity Matrix 251
• Tests of Sampling Adequacy 252
• Selection of the Type of Factor Analysis 253
Initial Extraction 255
Communality 257
Disadvantages of Classical Test Theory 277
Item Response Theory Basics 278
Polytomous Models 285
Technical Issues 288
TABLES, FIGURES, AND EXHIBITS
Tables
2.1 The Rosenberg Self-Esteem Scale. 5.1 Sample Table of Test
Specifications. 5.2 Sample Table of Test Specifications Using
Individual Learning Objectives. 5.3 Table of Test Specifications
for an HIV Knowledge Test. 5.4 Item-Objective Congruency Evaluation
Conducted by Three Content
Experts for Ten Items. 5.5 Example of Item Analysis for a
True/False HIV Knowledge Test. 5.6 P Values for Answer Choices on
Three Multiple-Choice Items on an HIV
Knowledge Test. 7.1 Example of a 1 × 1 Matrix. 7.2 Example of an 8
× 1 Matrix. 7.3 Matrix for Outcome Expectations for Healthy Eating.
7.4 Items on the Outcome Expectations for Healthy Eating Scale. 8.1
Frequency and Percentage of Participants’ Benefit Ratings (N =
100). 8.2 Calculation of Variance Using Benefit Ratings of 100
Participants. 8.3 SPSS Printout of Measures of Central Tendency and
Dispersion. 8.4 Calculation of the Correlation Coefficient for the
Association Between
Exercise and Oxygen Consumption.
8.5 Correlation Matrix Showing Relationships Among Benefit Ratings,
Hours of Exercise, VO2, and Weight Change.
8.6 SPSS Printout Using the One-Way ANOVA Descriptive Command. 8.7
SPSS Printout for Test of Homogeneity of Variances. 8.8 SPSS
Printut of ANOVA Summary Table. 8.9 SPSS Printout Showing
Comparisons Among the Groups. 9.1 Josh’s Observed, True, and Error
Scores for Ten Attempts on a Research
Methods Quiz. 9.2 Descriptive Statistics for True Scores, Parallel
Test Scores, and Error Scores
for a Sample of Students. 9.3 Correlations Among True Scores,
Parallel Test Scores, and Error Scores for
a Sample of Students. 10.1 SPSS Printout Showing the Results of
Parallel Test Reliability Assessment. 10.2 SPSS Printout Showing
the Results of Test-Retest Reliability Assessment. 10.3 Correlation
Matrix for the Maternal Involvement Scale. 10.4 Variance-Covariance
Matrix. 10.5 Variance Covariance for the Maternal Involvement
Scale. 10.6 Values of Cronbach’s Alpha for Various Combinations of
Different Num-
ber of Items and Different Average Interitem Correlations. 10.7
Ratings of Two Observers on Ten Videotape Segments. 10.8
Cross-Tabulations for Agreement Among Observers. 10.9 Amount of
Variance and Percentage of Total Variance for the Social
Support
Scale. 11.1 New and Old Terminology for Validity. 11.2 Hypothetical
Scores of Participants on a Readiness Scale and Percentage of
Doses Taken Correctly. 11.3 Correlation Coefficients for
Relationships Between Self-Efficacy and Outcome
Expectations, Social Support, Difficult Life Circumstances, and
Age. 11.4 SPSS Printout of ANOVA for Hypothetical Data Comparing
Differences
Between Two Groups on a Self-Efficacy Scale. 11.5 Expected Pattern
of Relationships for a Multitrait-Multimethod Matrix. 11.6
Multitrait-Multimethod Validity Matrix for QOL and Stigma Scales
Mea-
sured by Self-Report and Interview. 12.1 Categories of Physical
Activities Formed by Conceptual Similarities. 12.2 Correlation
Matrix for Eight Items on the SE Scale. 12.3 SPSS Printout with
Descriptive Statistics for the Twelve-Item SE Scale. 12.4 SPSS
Printout of the Correlation Matrix for the Twelve-Item SE Scale.
12.5 SPSS Printout of the KMO and Bartlett’s Test of Sphericity for
the Twelve-
Item SE Scale. 12.6 SPSS Printout of the Anti-Image Correlation
Matrix for the Twelve-Item SE
Scale.
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvi
12.7 SPSS Printout of the Initial Solution for the Twelve-Item SE
Scale. 12.8 SPSS Printout for the Unrotated Factor for the
Twelve-Item SE Scale. 12.9 SPSS Printout of the Communalities Table
for the Twelve-Item SE Scale.
12.10 SPSS Printout of Goodness-of-Fit Test for the Twelve-Item SE
Scale. 12.11 SPSS Printout of the Unrotated Factor Matrix for Eight
Items on the SE
Scale. 12.12 SPSS Printout of the Rotated Factor (Pattern) Matrix
for Eight Items on the
SE Scale. 12.13 SPSS Printout of the Rotated Factor Matrices and
Factor Correlation Matrix
for the Twelve-Item SE Scale. 12.14 Factor Correlation Matrix.
12.15 SPSS Printout of the Rotated Pattern Matrix for the
Twelve-Item SE Scale
Using the Sort and Suppression Commands Under Options. 13.1 Items
for Two Hypothetical Tests. 13.2 Equations for the One-, Two-, and
Three-Parameter Logistic IRT Models. 13.3 CCC for a Three-Category
Graded Response Model Where a = 1.5,
b1 = –1.0, and b2 = 1.0. 13.4 ICRFs for a Three-Category Item Where
a = 1.5, b1 = –1, and b2 = 1.
Figures
1.1 Conceptualization of the Measurement Process. 1.2 Interval
Scale for Weights of Newborns Using the Mean Weight as the Origin.
1.3 Schema for Levels of Measurement. 2.1 Response Format from a
Worksheet Completed by a First-Grade Student. 2.2 Wong-Baker Faces
Pain Rating Scale. 2.3 General Form of a Visual Analog Scale. 3.1
Response Options. 4.1 Steps in Survey Development. 5.1 Steps in the
Development of a Knowledge Test. 5.2 Diagram of the Domain-Sampling
Method. 6.1 Steps in the Scale Development Process. 6.2 The
Measurement Process in Health Behavior Research. 6.3 Example of
Relational Statements in Social Cognitive Theory. 6.4 Gibbs’s
Relational Model (1972) as Modified by Dulock and Holzemer (1991).
6.5 Steps in Conducting a Concept Analysis. 7.1 Steps in the
Item-Writing Process. 8.1 Graphic Representation of Benefit Ratings
for a Sample of 100 Participants. 8.2 SPSS Data Editor Screen. 8.3
SPSS Dialog Box for Frequency.
Tables, Figures, and Exhibits xvii
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvii
8.4 Relationship Between Number of Hours Exercised and VO2 for 100
Participants. 8.5 Relationship Between Number of Hours Exercised
and Change in Weight for
100 Participants. 8.6 Range and Sign of Correlation Coefficients.
8.7 Possible Causal Relationships Between Two Variables. 8.8
Relationship Between Number of Hours Exercised and VO2 for Those
Who
Exercise Fewer Than 5 Hours per Week. 8.9 Venn Diagram Showing
Variance for Two Separate Variables.
8.10 Venn Diagram Showing the Shared Variance Between Two
Variables. 9.1 Histogram of Josh’s Scores on 100 Repeated Attempts
on a Research Methods
Test. 9.2 Histogram of Josh’s Scores on 100 Repeated Attempts on a
Health Behavior
Theory Test. 9.3 Relationship Among Observed-, True-, and
Error-Score Variance.
10.1 Diagram of the Components in a G Study. 10.2 Venn Diagram for
P ×O ×I Design. 11.1 SPSS Printout of Scatterplot Showing the
Relationship Between Readiness
Scores and Medication Adherence for a Hypothetical Set of Data.
11.2 Hypothesized Relationships Among Variables Within Social
Cognitive Theory. 11.3 Hypothesized Relationships Among Variables
to Be Examined. 12.1 Structure of the Outcome Expectations for
Healthy Eating Scale. 12.2 Reconfiguration of a Correlation Matrix
into a Factor (Structure) Matrix
for Eight Items on the SE Scale. 12.3 Steps in Factor Analysis.
12.4 SPSS Dialog Box for Descriptives Command. 12.5 SPSS Dialog Box
for Extraction Commands. 12.6 SPSS Printout of the Scree Plot for
the Twelve-Item SE Scale. 12.7 Scatterplot of Factor Loadings for
the Unrotated Factor Matrix for the Eight-
Item SE Scale. 12.8 Scatterplot of Factor Loadings for the Rotated
Factor (Pattern) Matrix for the
Eight-Item SE Scale. 12.9 SPSS Dialog Box for Rotation
Commands.
12.10 SPSS-Generated Scatterplot of Rotated Factor Matrix for the
Twelve-Item SE Scale.
12.11 SPSS Dialog Box for the Options Commands. 12.13 Latent
Variable Model for SEM Analysis.
13.1 Example of an Item Characteristic Curve. 13.2 Theoretical ICCs
for One-, Two-, and Three-Parameter Logistic IRT Mod-
els for Four Items with Given Parameter Values.
xviii Tables, Figures, and Exhibits
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xviii
Exhibits
1.1 Transformation of an Ordinal Variable by Addition and by
Multiplication. 1.2 Conversion of Temperature from Fahrenheit to
Celsius Degrees. 2.1 Examples of Response Formats. 2.2 Example of a
Thurstone Scale, Using Hypothetical Items and Weights for an
Epilepsy Attitude Scale. 2.3 Examples of Items for an Index of
Socioeconomic Status. 3.1 Example of a Knowledge Test Demonstrating
Instrument-Related Errors. 4.1 Example of Survey Items on
Hand-Hygiene Practices. 5.1 Materials for Review of Knowledge Items
by Content Experts 6.1 Sample Survey and Scale Items, Demonstrating
Differences. 7.1 Examples of Response Options.
10.1 SPSS Printout Showing the Results of a Split-Half Reliability
Assessment for the Maternal Involvement Scale.
10.2 SPSS Printout for the Alpha Coefficient for the Maternal
Involvement Scale. 10.3 SPSS Printout of the Results of a KR20 for
an HIV Knowledge Test. 10.4 SPSS Printout for Use in Item Analysis
for the Maternal Involvement Scale. 10.5 SPSS Printout with the
Results of Kappa 10.6 SPSS Printout of an Intraclass Correlation
Coefficient. 11.1 Example of Materials for Conducting a Content
Validity Assessment. 11.2 Example of Verbal Probes Used in an
Assessment. 12.1 Items on the Epilepsy Stigma Scale. 13.1 Equations
for the One-, Two-, and Three-Parameter Logistic IRT Models.
Tables, Figures, and Exhibits xix
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xix
To the memory of my parents, Thaddeus M. Konicki and Mary Hogan
Konicki
03_970972 fbetw.qxd 9/19/05 5:56 PM Page xx
PREFACE
This text is written primarily for public health students,
particularly those in health education and health behavior
programs. Students in other health disciplines,
including nursing, medicine, applied health sciences, and
psychology, may also find it useful. This book is intended to be an
introductory text for students and public health professionals who
have a limited background in statistics and mathematics, yet need a
basic understanding of measurement issues. It emphasizes classical
measurement strategies for constructing instruments, for testing
for reliability and validity, and for interpreting results. A major
portion of the text deals with scale development and test- ing, but
chapters on survey development and the construction of knowledge
tests are also included. Likewise, even though classical test
theory is the major theoretical approach, generalizability theory
and item response theory are also introduced.
The approach used stresses the practical application of concepts
and principles. With that in mind, I have used data from research
and programs that I have been involved in during the past ten years
to provide realistic examples from public health that demonstrate
the concepts. This book also includes illustrations of materials
used to gather evaluative data, and instructions for conducting
tests of reliability and validity using SPSS. Samples of SPSS
printouts are provided, along with detailed interpretations of the
results.
The overall objectives of this text are to improve the student’s
(1) knowledge of instrument development and testing using classical
test theory, (2) understanding of reports of reliability and
validity testing found in articles and reports, (3) ability
xxi
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxi
to conduct basic tests for reliability and validity, and (4)
ability to interpret the results of data analysis. To accomplish
these aims, the book is divided into thirteen chapters. The first
three chapters provide a background in measurement theory. In
Chapter One, the definition of measurement is presented, along with
a brief history of mea- surement in the psychological and health
sciences and a discussion of the advan- tages of measurement.
Chapter Two presents a brief description of the different types of
measures that might be used to collect data about health behavior
or factors asso- ciated with it. Chapter Three presents a
discussion of the types of measurement error and strategies to
reduce error.
Chapters Four and Five present overviews of survey development and
knowledge test construction, respectively. Chapter Six begins a
discussion of scale development, which is the focus for the
remainder of the book. This chapter presents important information
on the analysis of concepts. In Chapter Seven, the principles of
item writ- ing and of formatting summated rating scales are
discussed. Chapter Eight presents basic statistical concepts that
are necessary for understanding the material in the fol- lowing
chapters. Chapters Nine and Ten focus on reliability: Chapter Nine
discusses classical test theory, and Chapter Ten examines the
different types of reliability as- sessment. The final three
chapters of the book focus on validity testing. In Chapter Eleven,
the concept of validity is introduced and several strategies for
assessing va- lidity are discussed. In Chapter Twelve, the basic
principles of factor analysis are pre- sented. The final chapter,
Chapter Thirteen, provides an elementary description of item
response theory.
Examples and data analyses described in several of the chapters
rely heavily on my research in HIV prevention and self-management
of chronic illnesses. This work has been supported by several
agencies including the National Institute of Mental Health, the
National Institute of Nursing Research, the National Institute of
Child Health and Human Development, the Centers for Disease Control
and Prevention, the Epilepsy Foundation, and the Emory University
Research Committee.
Acknowledgments
There are many people who have helped bring this book to
completion. First are the students in my classes at the Rollins
School of Public Health, who provided the in- spiration for the
book. I especially want to thank the students in the measurement
classes of spring 2004 and spring 2005. These students used drafts
of the book for their class text and provided invaluable feedback
on the content and the presentation, for which I am grateful. I
also want to thank Amanda Whatley and Andrea Landis, who critiqued
each chapter from a student’s perspective and provided comments
to
xxii Preface
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxii
improve the clarity of the text. Many thanks also to those who read
drafts of the book and provided feedback, especially Stanley A.
Mulaik, Frances McCarty, Ronda Sinkowitz-Cochran, and Amy Lansky. I
am indebted to Stanley Mulaik who provided many helpful comments
throughout the book, in particular on reliability assessment and
factor analysis. His perspective on the use of factor analysis for
instrument de- velopment is clearly evident in Chapter Twelve.
Frances McCarty combed the chap- ters and recommended a number of
changes that helped clarify difficult concepts. She also
contributed to the section on analysis of variance and wrote the
final chapter on item response theory. Ronda Sinkowitz-Cochran and
Amy Lansky reviewed the chap- ters from the practitioner’s
perspective, and their suggestions for revisions improved the link
between theory and practice. I appreciate the support of those who
helped with the compilation of the book, and I especially want to
thank Pamela Kennedy, Regina Daniel, and Sara Di Iorio for their
many hours of assistance with gathering references, formatting
chapters, and other tasks. Not forgotten are the many stu- dents
who made numerous trips to the library for materials.
I am also indebted to my family for their patience and support.
Special thanks go to my husband, Ralph; my children, Matt and Sara;
and my sisters, Susan Arrowood, Doris Konicki, Mary Morin, Carol
Konicki, Nancy Konicki, and Pamela Kennedy.
August 2005 Colleen Konicki Di Iorio Atlanta, Georgia
Preface xxiii
THE AUTHOR
Colleen Konicki Di Iorio is a professor in the Department of
Behavioral Sci- ences and Health Education at Rollins School of
Public Health and holds a sec-
ondary appointment in the Department of Family and Community
Nursing at the Nell Hodgson Woodruff School of Nursing. She earned
her Ph.D. degree in nurs- ing research and theory development from
New York University, her master’s de- gree in nursing from New York
University, and her bachelor’s degree in nursing from the
University of Iowa. She is a registered nurse and a Fellow of the
American Acad- emy of Nursing. She has extensive experience in
health promotion research and in- strument development. Her
research on health behavior has been funded by the Centers for
Disease Control and Prevention, the National Institutes of Health,
and the Epilepsy Foundation, among others. Her work focuses on two
broad areas of health: adherence and self-management, and HIV
prevention. Di Iorio has published widely in the health behavior
literature. She has served as a consultant to and been a member of
panels of professional and community organizations and governmen-
tal agencies. She currently serves as the associate editor of the
Journal of Nursing Measurement. She recently received the
Distinguished Nurse Scholar Award from New York University and the
Distinguished Alumni Award from the University of Iowa.
xxv
Measurement in Health Behavior