Upload
burt
View
21
Download
1
Tags:
Embed Size (px)
DESCRIPTION
“Maximising Return from Cohorts” Project: Prevention of Attrition Findings. Cara Booker, Ph.D. MRC SPHSU October 12, 2009. Overview. Systematic Review Survey of Retention Among MRC Cohort Studies Preliminary Findings Future Analyses. Systematic Review. Objectives - PowerPoint PPT Presentation
Citation preview
“Maximising Return from Cohorts” Project: Prevention of Attrition Findings
Cara Booker, Ph.D.MRC SPHSUOctober 12, 2009
Overview
• Systematic Review
• Survey of Retention Among MRC Cohort Studies• Preliminary Findings
• Future Analyses
Systematic Review
• Objectives• To determine the effectiveness of
retention strategies in improving response rates in prospective population-based cohort studies.
• To identify possible characteristics (i.e. locality, age, etc.) of studies that may have affected the retention of cohort members.
Definition, Inclusion & Exclusion Criteria
• Population-based cohort study:• “Any well-defined population defined by geographic
boundaries, membership or occupation”. Szklo M. Population-based cohort studies.
Epidemiologic Reviews. 1998;20(1):81-90.
• Exclusion Criteria:• Clinical or non-clinical
trials• Non-population-based
cohort studies • Cohorts with record
linkage as the only method of follow-up
• Studies which only looked at effectiveness of tracking methods
• Inclusion Criteria:• At least one wave of
follow-up data collection in which the participant was personally contacted by the study
• Retention rates were reported
• Some description of the retention methods was available
Review Methods
• Electronic Database Search• 13 databases searched• 5 search terms
• Manual searches of bibliographies
• Internet and website searches for technical reports and internal documents
• Contact with Principal Investigators and other study personnel for access to unpublished manuscripts and grey literature
Flowchart
Potentially relevant prospective population-based cohort studies (n=913)
Excluded: Trials & non-population-based cohort studies (n=12,596)
Papers retrieved for further evaluation (n=290)
Excluded from review (n=265)No evaluation (n=245)
Reviews (n=13) Tracing methods only (n=7)
28 Studies in 32 Papers
Total papers found from search (n=17,210)
Excluded: Recruitment (n=3,701)
Excluded (n=623)No information on strategies (n=577)No information on follow-up (x=46)
Working Papers /Handbooks/Users Guide (n=5)Unpublished papers (n=1)Book chapter (n=1)
Results
• 28 Studies were identified• Countries
• 15 in the US• 5 in the UK• 3 in Canada• 2 in Sweden• 1 in Norway, Spain & Australia
• 11 were randomized trials • Response rates ranges from 34% - 98%
Retention Strategies by Data Collection Method
Evaluated Retention Method
Data Collection Method
PostalQuestionnaire
s
Face-to-FaceInterviews
Telephone Interviews
MixedMethods
Incentives 4 2 2 2
Monetary 1 1 1 2
Non-Monetary 2 1
Mixed 1 1
Reminder Methods 9
Repeat Questionnaires/Visits/Calls
8 1 2
Alternative Methods of Data Collection
5 3 2
Other 2
Length of Questionnaire 1
Postal Method 1
Incentives
• Randomized Trials (n=8)• Incentives associated with increased response rates
• Average increase ranged from 4% to 27%• Type of incentive (i.e. gift, money, cash card) did not appear to
have an effect on response rate• Use of cash cards and percentage of checks cashed varied• Previous round responders had higher rates than non-responders
• Non-randomized studies (n=2)• Comparing monetary to gifts, monetary had increased response• Non-monetary and informational incentives compared against
each other produced no difference in response
• Greater value of incentive great response, less money spent overall
Reminder Letters/Calls
• Randomized Trials (n=1)• Higher response among participants with second
reminder of phone call• Non-randomized studies (n=8)
• Reminder letters increased response• However compared to a second questionnaire there
was lower response from the reminder letters• Overall, use of multiple reminders or methods of data
collection increased response
• Average initial response rate was 61%• The average increase of response rate via
• Reminder letters was 17%• Reminder calls was 11%
Multiple Posting/Calls/Visits
• 8 studies posted multiple questionnaires, 1 had multiple visits and 2 with multiple calls• Posting additional questionnaires increased
response by an average of 15%• Costs increase with subsequent posting
• Multiple visits to schools increased response by 34%
• More call attempts appeared to increase response rates
Alternative Data Collection Method
• 10 studies offered alternative methods• Postal studies that offered telephone interviews increased
response by an average of 5%• Face-to-face interviews which offered alternatives
increased response by 25%• Two studies started in a central location, one started with
clinic visits• Telephone studies that sent postal questionnaires
increased response by 1%• Costs are higher for postal vs. in-school
questionnaires (costs of mail & project coordinator) and telephone interviews are more expensive than post (due to personnel costs, and tracing costs)
Other
• 2 Randomized Trials• Length of questionnaire
• Received higher response with short form of questionnaire (not significant)
• Postal method• Randomized type of envelope and
certified mail vs. UPS– Certified mail had better response– Handwritten envelopes had better response
than other types of envelopes
Study Characteristics
• Different designs, sample sizes, reporting etc make it difficult to address our secondary aims. Broad pattern of:• Lowest response rates in US-based studies• Higher response rates in studies started in the 1980s
and 1990s• In general, participants who were ethnic minorities,
younger and of lower SES had lower response regardless of retention method
• Higher education associated with earlier response• Postal studies with males only had higher response
rates than mixed and female only studies• In mixed gender studies females had higher response
rates in 8 of 11 studies.• Proportion of males increased with increasing contacts
Conclusions
• Incentives were the only strategy that had a clear and positive association with response rates
• Multiple mailings of questionnaires and reminder letters also appear to increase response
• Alternative data collection methods had minimal increases for postal questionnaire studies, but large increases for in-person interview studies
• Use of multiple strategies increases overall response rate
Discussion/Limitations
• Very few cohort studies with analyses of retention methods
• Reporting of attrition is not standardized• Often have some mention in various manuscripts
detailing characteristics of attriters, which may vary by wave of data collection
• More often don’t have technical reports, or detailed manuscript about the strategies used the characteristics of the attriters across the study
• Calculation of response rates is also difficult due to eligibility criteria, definition of denominator and booster samples
Recommendations
• Pilot studies or sub-samples to evaluate retention strategies
• Focus more expensive strategies to non-responders• Most initial responders will do so without the need for
incentives or other more vigorous reminders
• Explore effectiveness and ethical issues associated with internet searches and use of social networking sites in relation to participation not just for tracking
Survey of Retention Among MRC Cohort Studies
• Objectives• To examine what retention methods are
associated with reductions in attrition• To explore what other study design features,
if any, influence minimisation of attrition
• Sample• Drawn from studies which are housed in one
of the 12 MRC Population Health Research Network (PHSRN) units• Thirty-eight studies were identified
Methods of Survey
• Data collection• Conducted between December 2007 and August 2008 • Reminder methods
• 1 Reminder telephone call• 3 Reminder emails
• Questionnaire Contents• General study design issues• Recruitment • Consent issues • Compensation/Incentives/Reimbursement • Tracking of study members• Attrition
“Maximising the return from cohort studies: The prevention of
attrition and efficient analysis” Study Questionnaire
Funded by MRC Population Health Sciences Research Network http://www.sphsu.mrc.ac.uk/sitepage.php?page=cohort_studies
Table 1: Response rates 1993 - 2002
50
55
60
65
70
75
80
85
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002
HSE
BCS
FRS
BSA
NALS
GHS
average
All Eligible 13,304
Excluded 1799
Refused 3001
Enrolled 8,504
Baseline Participants: 8504
1st Follow-up Participants: 7535 (89%)
Deceased 275
Dropped Out 1799
Successfully Traced 8119
Retention, Tracking, Tracing...
• Retention Strategies• Used to increase response at time of data collection
• Tracking/Tracing Strategies• Used to find participants between methods of data
collection
• Ambiguous Strategies• Often used between data collection but often used to
increase loyalty & not specifically to track/trace
General Study Design
• 24 studies responded (75% response rate)• 7 were dropped from analyses • One study had two distinct samples,
increasing analysed studies to 25• 4 settings
• Data presented here is from 18 studies
Data Collection
• How studies collected data• Face-to-face: Over 90% • Post: 70%• Telephone: 40%• Internet: 10%
• Studies collected between four and eight different types of data
0
10
20
30
40
50
60
70
80
90
100
Pe
rce
nt
Method
Figure A3. Method of Data Collection
Retention Strategies
• 6 studies allocated funding for retention
• 10 studies offered incentives• 1 study evaluated use of incentives
• Unconditional gift voucher found to be the most successful
• Retention Methods• 50% of the studies used 2-6 different retention methods• Most commonly used
• Email, newsletters, newspapers/magazines• GPs/CCs, schools• Key Leaders/Gatekeepers, administrative/supervisory
bodies, parent assessments
Tracking Methods
• Tracking/Tracking Methods• Minimum number of methods used was 1 & 14
was maximum• Most used:
• Change of address cards, GP records, routinely registered events, stable address of close relative
• Ambiguous methods• 80% provided findings• 40% provided holiday cards & 20% provided
birthday cards
Strategies: Numbers, Barriers & Successes
• Successful Retention Strategies• Family/friend contact, change of address card and
telephone/directory assistance
• Barriers to retention • Residential mobility, disinterest in issues covered by study,
incorrect address and people too busy to join study.
Figure S1. Number of Retention Strategies Used Per Wave
0
1
2
3
4
5
1 2 3 4 5 6 7 8 16 17
Few
est
Str
ate
gie
s
0
1
2
3
4
1 3 4 5 6 8 9 10 15 16 21 24
Most
Str
ate
gie
s
FewestStrategies Usedin a Wave
Most StrategiesUsed in a Wave
Further Analyses
• Analyze the impact of retention methods and different data collection methods on retention• What retention methods are the most significant in
reducing attrition?• What is the impact on attrition when more invasive data
collection methods are used (i.e. Vene-puncture, tissue collection, etc.)
• Use a multi-level modelling approach• Waves nested within studies
• Wave variables include: retention methods, data collection methods
• Study level variables include: setting, average age of sample at baseline, gender of sample, year study started
Future Analytic Methods
• Multivariate modelling• Combine methods into larger categories
• i.e. Radio + Internet = Media
• Examine the effects of different sample population characteristics on subsequent retention rates• Data collection types• Age• Gender• Study setting
Thank you!