The Science and Practice of The Science and Practice of Implementation : Are We Implementation : Are We

headed Down the Right Path?headed Down the Right Path?

10th Biennial Regenstrief Conference 10th Biennial Regenstrief Conference Emerging Perspectives on Transformational Emerging Perspectives on Transformational

Change in Healthcare SystemsChange in Healthcare Systems  OctoberOctober  2 2 –– October October  4, 2007.4, 2007.

Jeff Alexander, Ph.D.Department of Health Management and Policy

University of Michigan School of Public Health

Goals

Problems with Quality Improvement Research- why it’s not contributing to systems change

State of the art- Implementation Research- good, bad, and ugly

Modest proposals for advancing the science of implementation and the usefulness of QI research

State of the Art – QI Research

Single organizational samples Opportunistic not systematic Short duration studies No replication of studies No explicit consideration of context No explicit consideration of

implementation

Bottom line:

We don’t know what works, when it works, or where it works

BEYOND THE LINEAR MODEL

Basic Research

Clinical Trial

(Efficacy)

TreatmentDevelopment

EffectivenessTrial

TreatmentDeployment

Problems with the Linear Model of Implementation

Little on causal pathways & nested interconnected structures and activities

Little influence of OT & OB in QI studies

RCT thinking: control context away

RCTs as the Gold Standard?

Great for testing efficacy of molar interventions

Not so great for assessing: Process related phenomena Complex interactions among program

components Contextual effects implementation

BackgroundBackground

Health care provided in organizational Health care provided in organizational contextcontext

behavior of clinicians influenced by the behavior of clinicians influenced by the organizations in which they workorganizations in which they work

recognition of the interconnections among recognition of the interconnections among components of organizations (clinical components of organizations (clinical teams function within hospitals, interact teams function within hospitals, interact with other clinical teams, support systems with other clinical teams, support systems - embeddedness- embeddedness

Implementation-State of the Art

Emerging lists of “best practices” Continued assumption, encouraged

by funding streams, of linear development of interventions

Anecdotal information on implementation

Some efforts to produce models, theories to test implementation

Implementation: the influence of content, context, and process

Implementation

Content Process•Opinion leaders, change champion

•Systemic processes (e.g., supervisory practices, quality improvement)

•Organizational learning

•Triability

•Innovation type

•Evidence interpretation and packaging

Internal:

•Organizational culture

•Organizational structure

•Practice setting characteristics

•Local stakeholders (e.g., attitudes and behaviors)

•Resources

External:

•Networks

•regulation

•Economic (e.g., reimbursement)

•Competition

Context

Management Support:Management

communicates a rationale and priority

Klein and Sora Model

Implementation Effectiveness:Consistency and

quality of innovation use

Implementation Effectiveness:Consistency and

quality of innovation use

Innovation-Values Fit:The perceived fit

between the end-user's values and the

innovation

Champion(s):Champion(s) promotes

the innovation with targeted org members and/or management

Financial Resource Availability:

Resources are made available to support

implementation policies and practices

Implementation Climate:

The innovation is perceived as an

organizational priority by targeted end users

Implementation Policies and Practices:

Formal organizational actions ensure user

skills, create incentives and/or identify and

address barriers to use

Why Context Matters

Context may affect implementation directly

Context may moderate the relationship between an innovation and outcomes of interest

Context may establish the external validity of both implementation and QI research

Problems with Implementation Problems with Implementation Context Measurement and Context Measurement and

AnalysesAnalyses assigning the same group value to all assigning the same group value to all

members of a groupmembers of a group aggregating individual outcomes to aggregating individual outcomes to

the group levelthe group level Separate analyses of organizational Separate analyses of organizational

and individual phenomenaand individual phenomena

Advantages of Multilevel Advantages of Multilevel DesignsDesigns

statistically efficient estimates of statistically efficient estimates of regression coefficients regression coefficients

Use of clustering information Use of clustering information provides correct standard errors, provides correct standard errors, confidence intervals and significance confidence intervals and significance teststests

Allows for uneven assessments and Allows for uneven assessments and different program tenures (for different program tenures (for longitudinal studies)longitudinal studies)

Advantages of MLDAdvantages of MLD

Measurement at any of the levels of Measurement at any of the levels of a hierarchy enables examination of a hierarchy enables examination of whether differences in average whether differences in average outcomes between organizations are outcomes between organizations are explained by factors such as explained by factors such as organizational practices/structuresorganizational practices/structures, , or other characteristics of individual or other characteristics of individual patients or providerspatients or providers

0

0.5

1

1.5

2

2.5

3

entry 1 year 5 years

ADL

impa

irmen

t

low participation

high participation

Potential applications of Potential applications of MLDMLD

Effects of organizational infrastructure Effects of organizational infrastructure on implementation in micro teamson implementation in micro teams

Effects of org. culture on individual Effects of org. culture on individual provider attitudes and behavior (e.g. provider attitudes and behavior (e.g. physician use of clinical guidelines) physician use of clinical guidelines)

Translational researchTranslational research Effects of micro-team structure and Effects of micro-team structure and

process on patient outcomesprocess on patient outcomes

Issues with Multilevel Issues with Multilevel AnalysisAnalysis

Data requirementsData requirements Statistical powerStatistical power Analysis and interpretation issuesAnalysis and interpretation issues

Multi-method DesignsMulti-method Designs

Quantitative-QualitativeQuantitative-Qualitative RCT-case studyRCT-case study Process study-outcome studyProcess study-outcome study

Sustainability- long term studiesSustainability- long term studies

Life Cycle of Quality Improvement

Time

90

80

70

60

50

40

30

20

10 0

Adoption Rate

Implementation

Diffusion

Plateau

Institutionalization

Science of Complexity

Assumptions regarding interactions among components or “agents” of the system Heterogeneity- agents differ in important

characteristics (e.g. preferences) Dynamic-agents change, how system

changes are non linear, chaotic Feedback- change often results from

feedback that agents receive from their own behavior

Complexity Science

Organization- agents organized into groups or hierarchies that influence how system evolves over time

Emergent behavior- what results from the actions and interaction of individual agents

Engaged Scholarship

Theory

Solution

Model

Reality Proble

m F

orm

ulatio

n Theory Building

Resea

rch

DesignProblem

Solving

Describe Problem/Issue- visit & study it- map & diagnose itFormulate the Question- from users’ perspective?Criterion - Relevance

Answers & Arguments- plausible alternatives-- clarify context- identify key variations- cross levels of abstractionCriterion - Validity

Obtain the Evidence- case/field/experimental study- unit selection & sampling- measurement & observation- data analysisCriterion - Truth

Application & Implementation- knowledge for what? who?- for science & profession - apply findings to problem- develop implementation planCriterion - Impact

Capacity Building for Implementation Research

Data Funding Multi-disciplinary teams Make implementation part of the

intervention Bring in users of intervention/innovation Long term studies Basic research on implementation

Other questions

What aspects of care are modular and what aspects are inherently interdependent?

To what extent can one element of a system be altered without consequences to other elements?

Should intervention content and context dictate implementation process?