Upload
hpcareernet-llc
View
426
Download
1
Tags:
Embed Size (px)
Citation preview
Planning, Implementation, and Evaluation Using the RE-AIM Framework
HPLive.org Presentation: March 13, 2015
Contributors: Russ E. Glasgow, PhD and Paul A. Estabrooks PhD
Presenter: Samantha M. Harden, PhD, Human Nutrition, Foods and Exercise, Virginia Tech
RE-‐AIM Dimensions Example Studies History of
RE-‐AIM
Outline
Outline
“If we want more evidence-based practice, we need more
practice-based evidence.” Green LW. Am J Pub Health 2006
• Internal validity perspec=ve § The magnitude of effect as the key indicator of readiness for transla=on and adheres to the principles of evidence ra1ng for determining efficacy
• External validity perspec=ve § ACen=on to interven=on features that can be adopted and delivered broadly, have the ability for sustained and consistent implementa1on at a reasonable cost, reach large numbers of people, especially those who can most benefit, and produce replicable and long-‐las1ng effects
Brief History of RE-AIM
Glasgow RE, Vogt TM, Boles SM. Evaluating the Public Health Impact…Am J Public Health, 1999;89:1322-1327
Original RE-AIM
• First published ar=cles in 1999
• Originally intended to increase balance between internal and external validity
• First used to evaluate preven=on and health behavior change programs
• RE-‐AIM Trivia: was going to be called ARIEM
Gaglio B & Glasgow RE. (2012). Evaluation approaches for dissemination and implementation research. In: R. Brownson, G. Colditz, & E. Proctor (Eds.), Dissemination and implementation research in health…1st Edition pp. 327-356). New York: Oxford University Press.
6
www.RE-‐AIM.org
A CHOICE?
– Program A: 16 session physical activity program that produces a 150 minute per week change in moderate intensity physical activity for 8 out of every 10 participants
– Program B: 16 session physical activity
program that produces a 150 minute per week change in moderate physical activity for 2 out of every 10 participants
Which program do you think is better?
Answer: It Depends! � Why does it depend?
– Who delivers? ÷ Program A: Trained master’s level health educators ÷ Program B: Administrative assistants in community health
center – How easy is it to implement?
÷ Program A: Moderately difficult ÷ Program B: Moderately easy
– What resources? ÷ Program A: Group exercise area and counseling rooms ÷ Program B: Email access and participants can do activities at
home or in neighborhood.
Which is Better?
Answer: It Still Depends! – How scalable is it?
÷ Program A: 20 people can participate per class session which includes 90 minute counseling session and 3 one hour classes each week.
÷ Program B: 100 people can participate per session which includes monitoring of physical activity and sending out weekly newsletters.
– What does it cost? ÷ Program A: 33 hours/week for 6 months from health educator
for every 16 successes (20 people per group). ÷ Program B: 8 hours/week for 6 months from administrative
assistant for every 20 successes. – How sustainable are the effects?
÷ Program A: 50% return to baseline activity after 6 months ÷ Program B: 50% return to baseline level after 6 months
Which is Better?
RE-AIM: Goals for translating useful interventions into regular practice
10
• Move from a paradigm that emphasizes: • The magnitude of effect as the key indicator of readiness
for translation and adheres to the principles of evidence rating for determining efficacy
• Move to one that emphasizes: • Attention to intervention features that can be adopted
and delivered broadly, have the ability for sustained and consistent implementation at a reasonable cost, reach large numbers of people, especially those who can most benefit, and produce replicable and long-lasting behavior changes
• Reach • Effectiveness • Adoption • Implementation • Maintenance
The RE-AIM Framework:
Glasgow et al, AJPH, 1999
What is the RE-AIM Framework?
12
Definition: The number, percent of target audience, and representativeness of those who participate.
Data Needed:
Denominator—number of eligible contacted for potential participation
Numerator—number of eligible that participate
Comparative information on target population
E-AIM ELEMENTS:
13
Example: Move More.
Data:
Denominator—Inactive or insufficiently active adults going to the doctor for a physical (n=1518 total; 607 eligible; 218 referred)
Numerator—number of eligible that participate (n=115)
Participation Rate: 115/607=19%
Almeida et al. JSEP 2005
E-AIM ELEMENTS:
14 Almeida et al. JSEP 2005
Enrolled Participants Census Tract Total
Sample (n=115)
Stimulus Control (n=44)
Standard Care
(n=71) Female 60.9% 65.9% 57.7% 51.1%
48.8 (±11.9)
48.6 (±11.3)
48.9 (±12.3)
30.4
White 58.9% 56.8% 60.3% 55.3% Black 22.3% 27.3% 19.1% 19.8% Latino 12.5% 6.8% 16.2% 14.8% Asian 0% 0% 0% 5.6%
E-AIM ELEMENTS:
15
Definition: Change in outcomes and impact on quality of life and any adverse outcomes
Data needed:
Primary Outcome
Quality of life
Potential negative outcomes
R -AIM ELEMENTS:
16
Example: Family Connections Data:
Primary Outcome: Significant reductions in BMI z-score.
Quality of life: Improvement in quality of life with lower weight status
Potential negative outcomes: No evidence of heightened eating disordered symptoms
Estabrooks et al. AJPM 2009 Shoup et al. QLR 2008
R -AIM ELEMENTS:
Definition: Number, percent and representativeness of settings and educators who participate.
Data needed:
Denominator—number of eligible sites contacted for potential participation
Numerator—number of eligible sites that participate
Comparative information on target population of sites
RE- IM ELEMENTS:
Example:
Data:
Denominator—105 counties in Kansas eligible to participate
Numerator—48 agreed; 48/105=46%
Representativeness—Less active agent, less likely to deliver; Smaller population counties, more likely to deliver
Estabrooks, Bradshaw, Fox, et al. , AJHP, 2004 Estabrooks, Bradshaw, Dzewaltowski , & Smith-Ray, ABM, 2008
RE- IM ELEMENTS:
19
Definition: Extent to which a program or policy is delivered consistently, and the time and costs of the program.
Data needed:
Information on program components and essential elements
Information on resource use
RE-A M ELEMENTS:
Example:
Data:
Variability in delivery of program components based upon local tailoring of the program.
On average, 80% of program components were delivered as intended
2.5 hours of delivery agent time per participant compared to 36 hrs per participant in control
RE-A M ELEMENTS:
21
Definition:
Individual/member target: Long-term effects and attrition.
Setting/educator: Extent of discontinuation, modification, or sustainability of program.
Data needed:
Primary outcome assessment 12 months post intervention
Documented sustained delivery
RE-AI ELEMENTS:
22
Data:
Decreased BMI z-scores sustained 12 months after intervention complete
Data:
RE-AI ELEMENTS:
Individual level factors balancing internal and external validity:
� Shift from focus on the numerator to the denominator � Generalizability to target population � Avoid contributing to disparities � Common comparison for decision making including
unintended consequences � Robustness when combined with adoption: what
works best for whom, and under what conditions
Summary: Key RE-AIM Issues to Improve Translational Research
Setting level factors balancing internal and external validity:
� Will the intervention fit in a typical practice setting? � Generalizability to who will deliver the program � Initial start-up and ongoing costs � Understanding structure and who makes adoption
decisions (and how they are made) � Characteristics of the intervention, setting, culture,
and organization that help or hurt implementation
Summary: Key RE-AIM Issues to Improve Translational Research
RE-AIM: Goals for translating useful interventions into regular practice
Develop and translate research and practice-based interventions that can:
be adopted and delivered broadly,
have the ability for sustained and consistent implementation at a reasonable cost
reach large numbers of people especially those who can most benefit,
produce replicable and long-lasting behavior changes
One more example!
Integrated Research-Practice Developed versus Pipeline Model in Physical Activity Programming: A comparative analysis. Samantha M. Harden, Sallie Beth Johnson, Fabio A. Almeida, Paul A. Estabrooks In Preparation
Adoption-Based Randomized Control Trial
Integrated Research-Practice Model Efficacy to Effectiveness to Demonstration to
Dissemination Model
Fit Extension Active Living Everyday
Untitled
Implementation
0
100
200
300
400
500
600
Fit Ex ALED
Cost assessed as implementation hours
• Degree delivered as intended • ALED ~90% • Fit Ex ~ 80%
• Adaptation • ALED None reported • Fit Ex Numerous small changes in feedback timing and
structure
Effectiveness
� Reach ¡ Fit Ex-75 participants per program
¡ ALED 15 participants per program
¡ Both underrepresented by men
� Maintenance- Delivered for 3 years post initial evaluation
Descriptive Information