Upload
alfred-norris
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Statistical Issues in Data Collection and Study Design For
Community Programs and Research
October 11, 2001
Elizabeth Garrett
Division of Biostatistics
Department of Oncology
Overview
• Goals of Data Collection and Study Design
• Key concepts– Reliability – Validity
• “Latent” Constructs
• Study Designs
• Potential Biases
Goals of Data Collection
Two broad goals*
– evaluation of intervention• controlled
• uncontrolled
– summary of population• demographics
• attitudes
The Goal of Study Design
To devise a model for some complex etiologic or clinical process that
gives valid and precise inference.
Issue Specific to Interventions
• Outcomes tend to be “soft”
• Not always an easily quantified response
• We often use one or more “surrogates” to measure outcomes.
Key Concepts• Reliability: Is the data that you are collecting a reliable or
reproducible measure?• Has to do with how closely your measure correlates with the
underlying construct you want to measure.
• Truth = Observed Data + Error– If you collected the same data tomorrow, would you get the same
answer?– If you ask a related question, will the two questions have
correlated answers?– If two different “raters” (i.e. collectors) evaluate the same
individual, do they get the same data?
Validity
• Validity: Is what you are collecting measuring what you want to measure?– Face validity– Construct validity– Criterion validity– Etc.
Valid
Invalid
Reliable(but still invalid!)
Valid and Reliable
Validity
• Internal Validity:– Valid for the population from which you
sampled
• External Validity– Generalizable to a broader population
Latent Constructs
• Definition: A latent variable is a variable that cannot be directly measured.
• Examples:– Quality of Life– Socio-economic status– Distress– Depression
Latent Constructs
• Need to be measured using multiple variables• Variables, taken together, should “define” the construct• Methods should be decided upon ahead of time and
data collection needs to be considered.• Examples: latent class analysis, factor analysis• Coding is important
– likert scale: “On a scale of 1 to 7…..”– binary: yes/no, present/absent– continuous: age, income
Some Study Design Types• Cross-sectional, No Intervention
– Attributes• quantify community
• “summarize” attitudes, demographics, etc.
• descriptive statistics– means, medians, standard deviations
– “pictures” of the sample: histograms, boxplots
• “hypothesis generating”, and NOT “hypothesis testing”
• simplest conceptually
• Cross-sectional, No Intervention (cont.)– Issues to think about
• sampling– Who?
– When?
– Where?
• Data (this is general to ALL study designs)– format?
– Binary versus continuous versus ordinal versus categorical?
– open-ended?
• Pre-post Design, One group (uncontrolled)– Was intervention successful?– Attributes:
• Compare baseline to follow-up
• simplest when only two time points are collected.
• Convenient that each individual serves as his/her own control
• Hypothesis testing: – Ho: intervention worked
– Ha: intervention did not work
• Some methods: binomial tests, signed rank test, paired t-test, regression methods
– Issues to think about• when should “success” be measured?
– 1 week? 1 month? Both?
– What if effect at 1 month but “washed out” by 6 months?
• How is success measured?
– Yes/no? Continuous change in score?
• Learning effect
– bias of questionnaires
– is this the most appropriate design if there is a potential learning effect?
• “Placebo” effect could play a role.
• Adherence!
– Is the effect of intervention smaller than anticipated because some study participants did not adhere?
• Confounders and effect modifiers!
– Are there some individuals that respond more strongly to the intervention than others in such a way that is predictable (e.g. age? weight? political views?)
• Pre-Post, Two Groups (Controlled)– Does intervention group improve more than the
control group?– Attributes
• similar to pre-post, one group
• can quantify placebo and learning effects (caveat)
• hypothesis testing:– Ho: effect in control group = effect in treatment group
– Ha: effect in control group effect in treatment group
• Some methods: 2 sample t-test, rank sum test, fisher’s exact test, regression methods
– Issues to think about• We have a measure of placebo effect
• blinding or masking? Is it possible?
• Randomization– Balance?
– Stratification necessary?
– Matched?
• ITT versus Treatment received?
• Drop out
• Adherence
Other Study Designs
• Case-Control Studies
• Cohort Studies (aka Prospective Study)
• Ecologic Study
Potential Biases to Keep in Mind• Selection Bias (IV)
– individuals who join the study are not representative of the population in a way that affects the outcome.
• Information Bias (IV)– measures tend to be biased in one direction
• Confounding (IV)– Mixing of effects leads to wrong inference
• Effect Modification (IV)– effect of treatment depends on another factor (e.g. age)