Upload
todd-garrett
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
RE-EXAMINING THE ROLE OF PROFESSIONAL DEVELOPMENT AND TRAINING EVALUATION THROUGH AN IMPLEMENTATION SCIENCE LENS
MICHELLE GRAEF & ROBIN LEAKE
NHSTES June 15, 2011
Presentation Overview
Children’s Bureau’s Child Welfare Implementation Centers
What do we mean by “implementation”?
Overview of one Implementation Science framework (NIRN)
Implications of framework for training and training evaluation
Implementation Centers and National Resource Centers
Improve child welfare administration and practice Meet Federal performance standards and
implement effective programs Pursue sustainable and positive systems change Improve outcomes for children, youth & families
Funded by the Children’s Bureau in 2008 to provide States and Territories, Tribes, and Tribal Consortia individualized training and technical assistance (T/TA) to:
States and Tribes are sometimes without the resources necessary to implement comprehensive strategic plans
National Resource Centers have limited resources to provide intensive, long-term technical assistance
Implementation Centers provide in-depth and long-term technical assistance and peer networking opportunities to States and Tribes
Implementation Centers: Filling the Gap
Geographic Assignments
What do we mean by “implementation”?
The process of moving an idea from concept to reality
“a specified set of activities designed to put into practice an activity or program of known dimensions” (NIRN)
Involves both: project management (managing the tasks,
resources, budget, process –technical) change management (managing the people
– adaptive)
In other words…
Science of Implementation
There is a body of research on effective implementation, mostly from research on evidenced-based practices
Child welfare systems have been slow to explore the science, often using “paper implementation”
The Implementation Centers are using several models to guide their work: National Implementation Research Network (NIRN) Phases of Change (John Kotter) ADKAR (Awareness, Desire, Knowledge, Ability,
Reinforcement) - Prosci
Selection
Training
Coaching
Performance Assessment
Systems Intervention
LEADERSHIP
Decision Support Data
System
Integrated &
Compensatory
CO
MPE
TEN
CY
Drive
rs
ORG
AN
IZATIO
NAL
Drivers
Facilitative Administration
© Fixsen & Blase, 2008
National Implementation Research Network: Implementation Drivers
Stages of ImplementationCore Activities
Adapted from Implementation Research: A Synthesis of the Literature (2008) The National Implementation Research Network
Stages of ImplementationCore Activities (cont’)
Adapted from Implementation Research: A Synthesis of the Literature (2008) The National Implementation Research Network
So does any of this really happen in child welfare agencies? ICs have developed a common measure
of our implementation process, based on NIRN drivers but includes others
Collecting data across all projects: summarize implementation drivers that
are salient, describe implementation activities
happening over life of projects, across multiple stages.
Intervention vs. Implementation Outcomes Important to differentiate between
intervention-level activity/outcomes and implementation-level activity/outcomes (Proctor et al, 2010).
Examples of implementation outcomes: Acceptability (user satisfaction) Adoption/uptake (utilization or intent to try) Feasibility (suitability for use, practical) Fidelity (adherence to program model) Penetration (spread across settings) Sustainability (level of institutionalization)
Implications for Role of Training
What is the role of training in the NIRN framework? How is this different/not different from the way training is usually conceptualized by CW agencies?
Training alone, no matter how well done, does not lead to successful implementation
Fixsen, Naoom, Blase, Friedman, Wallace, 2005
Implications for Framework
How does overall professional development fit within the NIRN framework?
Why is training depicted separately from coaching in the triangle? Is there a better way to model this?
Implications for Role of Training Evaluation
What is the role of training evaluation in this framework?
What are the boundaries between training evaluation and implementation evaluation? Is there a distinction? Should there be?
What is the implication for the work that we do?
Learn More
Implementation Centers Website
Northeast & Caribbean Implementation Center (NCIC)
http://ncic.muskie.usm.maine.edu
Atlantic Coast Child Welfare Implementation Center (ACCWIC)
www.accwic.org
Midwest Child Welfare Implementation Center (MCWIC)
www.mcwic.org
Mountains & Plains Child Welfare Implementation Center (MPCWIC)
www.mpcwic.org
Western & Pacific Child Welfare Implementation Center (WPIC)
www.wpicenter.org