Using RE-AIM as a tool for Program Evaluation

From Research to Practice

What is RE-AIM

• RE-AIM is an acronym that consists of five elements, or dimensions, that relate health behavior interventions:– Reach the target population– Efficacy or effectiveness– Adoption by target settings or institutions – Implementation - consistency of delivery of

intervention – Maintenance of intervention effects in individuals

and populations over time

• Commonly used for evaluating efficacy trials (Phase III Research)

Defining Elements

Reach — The absolute number, proportion, and representativeness of individuals who participate in a given program.

Representativeness refers to whether participants have characteristics that reflect the target population's characteristics. For example, if your intent is to increase physical activity in sedentary people between the ages of 35 and 70, you wouldn't test your program on triathletes.

Defining Elements

Efficacy/Effectiveness — The impact of an intervention on important outcomes. This includes potential negative effects, quality of life, and costs.

Defining Elements

Adoption — The absolute number, proportion, and representativeness of settings and staff who are willing to offer a program.

Defining Elements

Implementation — At the setting level, implementation refers to how closely staff members follow the program that the developers provide. This includes consistency of delivery as intended and the time and cost of the program.

Defining Elements

Maintenance — The extent to which a program or policy becomes part of the routine organizational practices and policies. Within the RE-AIM framework, maintenance also applies at the individual level.

At the individual level, maintenance refers to the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.

How do elements relate to planning?

• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Reach: HOW DO I REACH THE TARGETED

POPULATION FOR INTERVENTION?– Efficacy or effectiveness: HOW DO I KNOW

THAT MY INTERVENTION IS EFFECTIVE?– Adoption HOW DO I DEVELOP

ORGANIZATIONAL SUPPORT TO DELIVERY THE INTERVENTION?

How do elements relate to planning?

• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Implementation: HOW DO I ENSURE THE

INTERVENTION IS DELIVERED PROPRLY?– Maintenance: HOW DO I INCOPROATE THE

INTERVENTION SO IT IS DELIVERED OVER THE LONG TERM?

How is RE-AIM different from other evaluation approaches?

RE-AIM draws upon previous work in several areas including:

Diffusion of innovations, multi-level models, and Precede-Proceed.

The primary ways that it is different is that it (a) is intended specifically to facilitate translation of research to practice, (b) it places equal emphasis on internal and external validity issues and emphasizes representativeness, and (c) it provides specific and standard ways of measuring key factors involved in evaluating potential for public health impact and widespread application.

Is RE-AIM used to design programs, or just to evaluate them?

• It is both. Although used more commonly at present to report results or compare interventions, it is also intended as a planning tool.

An Example

• Physical Activity Promotion in Primary Care: Bridging the Gap Between Research and Practice– Eakin, Brown, Marshall et al. (2004).

Translating and Disseminating Evidence-based Falls Prevention Programs into Community

Li et al. (2008). American Journal of Public Health

An Example of Application

Primary Aim:

(a) To translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults; and

(b) Using the RE-AIM framework (Glasgow et al., 1999), to pilot implement the program with a primary focus on reach, uptake (adoption), and implementation.

Secondary Aim:• To evaluate the effectiveness of the program

with respect to improvements in physical performance measures germane to falling.

Tertiary Aim:• To evaluate program maintenance with

respect to the extent to which older adults would continue to practice Tai Chi beyond the period of the initial evaluation.

Translation

Translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults

• Identify training objectives and elements• Identify end users and dissemination partners• Develop a dissemination package• Expert evaluation • Pilot testing

Program Evaluation

Reach: A total of individuals/providers responded to the program promotion

Effectiveness: defined as change in physical performance outcome measures taken at baseline and again at 12 weeks termination.

Adoption (or uptake): defined as the proportion of local community (senior activity) providers that agreed to participate and implement the program.

Implementation: defined as the extent to which providers' implemented a Tai Chi class to participating older adults and the ability to conduct the various elements of program protocols, including the use of implementation plan, a 2-times weekly program schedule, distribution of program supplements (i.e., videotape and a guidebook), a class attendance rate of 75% or better over the 12-week class period.

Maintenance: defined at both the service provider level and participant level. At the provider’s level, it was defined as providers’ willingness to consider the program to be part of routine organizational provisions (assuming adequate financial resources). At the participant level, it was defined as the extent to which improvements in participants’ physical performance were sustained, and their continued practice of Tai Chi, 8-weeks after completion of the class.

REACH: 87% (by study criteria); 45% by client attendance

Dissemination Outcome

ADOPTION: Six senior activity centers from five communities: 100% adoption

IMPLEMENTATION: 75% completed; >85% class attendance; average 32 min. of home practice

Effectiveness: Improved physical performance and quality-of life measures (a)Functional Reach; (b) Up and Go; (c) Chair Stands, (d) 50-foot speed walk, and SF-12.

Maintenances: Five centers continued; 87% participants continued

Dissemination Outcome

ImpactAdopted by the State of Oregon: being implemented in four counties; three more starting this year

Conclusion-Tai Chi

The evidence-based Tai Chi program is practical to disseminate and can be effectively implemented and maintained in community settings