Upload
norman-norman
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
1
Evidence-Based Public HealthTPHA 2013
Paul Campbell Erwin, MD, DrPHProfessor and Department Head
Department of Public Health, University of Tennessee
2
Learner Objectives
1. Identify and describe the domains of EBPH;2. Understand the applicability of EBPH to
improving the efficiency, effectiveness, and accountabilities of public health practice;
3. Describe how EBPH concepts can provide the basis for stating the value of local public health to stakeholders.
3
4
Why do we do the things we do?
How do we decide what to do?
5
How do we decide what to do?
6
Why do we do the things at WORK the way that we do?
How do we decide what to do?
Among competing priorities, how do we decide what to do, and why?
7
8
9
Evidence-Based Public Health the process of integrating science-
based interventions with community preferences to improve population health.
Dr. Ross Brownson
Acknowledgment: To Dr. Ross Brownson and team, for all EBPH Framework slides and general approach
10
Retool
Discontinue
Disseminate widely
What is “Evidence”?
11
What is “Evidence”? Scientific literature in systematic reviews Scientific literature in one or more
journal articles Public health surveillance data Program evaluations Qualitative data
– Community members– Other stakeholders
Media/marketing data Word of mouth Personal experience
Objective
SubjectiveLike beauty, it’s in the eye of the beholder…
12
Are we talking only of scientific evidence?
13
Where am I?
You’re 30 yards above the
ground in a balloon
You must be a researcher
Yes. How did you know?
Because what you told me is
absolutely correct but completely
useless
You must be a practitioner
Yes, how did you know? Because you don’t know
where you are, you don’t know where you’re
going, and now you’re blaming meThe problem
14
15
How much Evidence is enough?
“The best is the enemy of the good”
Voltaire
The effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials…. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.Smith and Pell, BMJ, 2004
16
How are decisions generally made in public health settings?
17
How are decisions generally made in public health settings?
History/inertia Anecdote Pressure from policy makers or administrators Media driven Resources/funding availability (C-E) Peer reviewed literature/systematic reviews Expert opinions (e.g., academics, community
members)
OR Combined methods, based in sound science The challenge is in how to make the best use of
multiple sources of information & limited resources. 18
Some Key Characteristics of EBPH1. Making decisions based on the best
available peer-reviewed evidence (both quantitative and qualitative research);
2. Using data and information systems systematically;
3. Applying program planning frameworks (that often have a foundation in behavioral science theory);
19
Some Key Characteristics of EBPH
20
4. Engaging the community* in assessment and decision making;
5. Conducting sound evaluation; and
6. Disseminating what is learned to key stakeholders and decision makers.
* Community of fellow staff/employees, of workgroups, of neighborhoods, etc.
21
Small Group Work
What was the most recent NEW thing you did at work?
Why did you do it? What was the Evidence-Base for
decision-making? Did it make a difference?
22
EBPH is NOT Rocket Science **
What’s the problem? How big is it? What’s the cause? What do I do about it? Once I do something, how do I know
I’ve made a difference?
** But rocket science depends on the use of evidence
23
EBPH begins with The Question
24
What’s the issue?
25
EBPH begins with The QuestionFor example…..
What are the most important health problems in our community?
Given the problems of Obesity, Chronic Disease, Child Health and Health Disparities, and Substance Use, what should we focus on? And once we decide, what should we do?
What’s the best way to reduce tobacco use in our community? In our health department?
How do we reduce no-show rates in Family Planning?
26
What’s the issue?
Community Assessment
• Decide what to assess• Decide the best methods to collect the
data to answer your questions• Develop a work plan that identifies
tasks to accomplish, partner roles and responsibilities, and a time frame for completion
• Organize information as it is collected so that it can be shared with all partners, community stakeholders, and community members.
27
Community Assessment
• Present information gathered and summarized from the community assessment back to partners• Coalition/partnership• Community forum• Meetings with community groups
• Move to Action Planning – what do we do to create the desired changes?
28
Community HealthIssues
Identify and discuss
problems and causal factors
Performance Improvement
Approach to Community Health Assessment, 2012-13
Phase 2 Focused
Assessment
Phase 1 Secondary Data
Assessment
Target yourCommunity
health Intervention
programs
Target your community
health intervention
programs
30
What’s the issue?
How big is it?
Quantifying the issue
• Disease/condition vs. Death
• Incidence/Prevalence vs. Mortality
• Problem of small numbers
• Types of rates
31
Quantifying the issue
• Incidence = new cases; a measure of
risk
• Prevalence = existing cases; a
measure of the overall burden of
disease
• P = I x D
32
Downloaded from: StudentConsult (on 4 September 2008 03:58 PM)© 2005 Elsevier
In a steady state: P = I x D
Downloaded from: StudentConsult (on 4 September 2008 03:58 PM)© 2005 Elsevier
Downloaded from: StudentConsult (on 4 September 2008 03:58 PM)© 2005 Elsevier
Downloaded from: StudentConsult (on 4 September 2008 03:58 PM)© 2005 Elsevier
Quantifying the issue
A program to address cardiovascular disease was put in place twenty years ago. An evaluation of the program showed that the prevalence of cardiovascular disease remained the same after twenty years. The program director was convinced this was a good program, but the evaluator retorted – “20 years and no improvement in prevalence?” How can you help this program director out – i.e., how could you explain that the program has indeed been successful despite a stable prevalence rate?
37
P = I x DTime 1: __ = 10 x 550 = 10 x 5Time 2:__ = 5 x 1050 = 10 x 5
38
39
Where do we get good data on Disease/Condition
Population-based
Vital Statistics• Birth and death
Reportable diseases
Registries• Birth defects• Cancer• Immunizations• Trauma
Representative Samples
National Health Interview Survey (NHIS)
National Health and Nutrition Examination Survey (NHANES)
Behavioral Risk Factor Surveillance System (BRFSS)
Youth Risk Behavior Survey (YRBS)
Hospital/facility-based
TennCareTHA
40
Quantifying the issue
The problem of small numbers: most challenging when expressed as rates, when the multiplier is large and the actual numerators and denominators are relatively small
41
Quantifying the issue
The problem: in 1994 a call from central office to alert me that the female African-American mortality rate for diabetes was higher in Claiborne County than for any county in the state, including Shelby County.
http://hit.state.tn.us/index.shtml
42
43
What’s the issue?
How big is it?
How important is it?
Developing a concise statement of the issue through “Data Synthesis”
Data Synthesis in MAPP
What are we synthesizing?
http://healthyknox.org/home.html
How can we create a sustainable network of partnerships that effectively contributes to improved community health?
STRATEGICISSUES
www.healthyknox.org
How can we achieve equitable health outcomes for all community members?
STRATEGICISSUES
www.healthyknox.org
How can we position health as a consideration in community policy and planning decisions?
STRATEGICISSUES
www.healthyknox.org
51
What’s the issue?
How big is it?
How important is it?
What is the cause?What should we do about it?
Determining what is known using scientific literature
Do immunizations work? Does family planning work? Does WIC work? Does restaurant inspection work? Do County Health Councils work?
52
Evidence-Based Public Health
http://www.thecommunityguide.org/index.html
http://prcstl.wustl.edu/training/Pages/Evidence-Based-Public-Health-Course.aspx
Evidence-Based Public Health
For which of the following is there strong or sufficient evidence that the intervention is effective (“recommended”) in obesity prevention and control?:Provider remindersSchool-based programsWorksite programs
http://www.thecommunityguide.org/obesity/index.html
54
Ok…Programs…but what about evidence for how we operate?
Fostering more effective public health by identifying administrative evidence-based practices. Brownson RC, Allen P, Duggan K, Stamatakis KA, Erwin PC. Am J Prev Med 43(3):309 –319, 2012
55
Administrative Evidence-Based Practices
56
Domain Evidence-Based Practice
Workforce development Training
Access to technical assistance
Leadership Skills and background of leaders
Values and expectations of leaders
Participatory decision-making
Organizational climate and culture Access and free flow of information
Support of innovation and new methods
Learning orientation
Relationships and partnerships Interorganizational relationships
Vision and mission of partnerships
Financial Allocation and expenditure of resources
58
What’s the issue?
How big is it?
How important is it?
What is the cause?What should we do about it?
What are we going to do about it?
Developing & Prioritizing Options
1. Multi-voting Technique2. Strategy Grids3. Nominal Group Technique4. The Hanlon Method5. Prioritization Matrix
59
Developing & Prioritizing Options
60
61
Developing & Prioritizing Options
64
What’s the issue?
How big is it?
How important is it?
What is the cause?What should we do about it?
What are we going to do about it?
How are we going to do it, when, with what results?
Developing an Action Plan and Implementing Interventions
Assessment should guide the development of programs
Community members should participate in the process
Program should be developed with key partners or as part of coalitions (builds infrastructure for change)
Participants should develop a comprehensive intervention strategy- but may implement one step at a time
Evaluation should emphasize feedback and program improvement
The community capacity for health promoting changes should be increased – skill transfer between community & academic/practice partners (two way)
65
Program development and implementation
Source: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
67
What’s the issue?
How big is it?
How important is it?
What is the cause?What should we do about it?
What are we going to do about it?
How are we going to do it, when, with what results?
How will we know if we have made a difference?
Why Evaluate?
68
What funders want:– Prove that their money is making a
big difference in outcomes of their choice.
What grantees/people who receive funds want: – Show that they should be given more
money to continue and expand the good work they are doing.
The Evaluation Trap
69
If evaluation is conducted for these reasons only, the result, over time, may be to undermine both political support for funding and program quality.
Why? These motivations can lead to: – looking for wrong outcomes – looking for wrong effect sizes (how big a
difference your program makes)
Logic models document your theory of change: how your program works
70
SpecifiedInterventionComponents
AndActivities
SpecifiedInterventionComponents
AndActivities
ChosenDeterminants
(Risk & ProtectiveFactors)
ChosenDeterminants
(Risk & ProtectiveFactors)
Outcomes
Interventions• Inputs• Activities
OutcomesProximalDeterminants• Behaviors• Outputs
71
Early detection of breast cancer
Outcomes
Breast cancer screening by mammography
Determinants
Provider reminder and recall systems
Activities
Evaluation Strategy 1: Start with Outcomes
We are used to thinking about activities first.
That’s OK! Build the model from left to right.
But you still need to think through why your activities “make a difference.”
72
Evaluation Strategy 2: Start Building the Model with Activities
73
SpecifiedActivitiesSpecifiedActivities
Immediate or short-term outcomes
(Exactly the sameas proximaldeterminants)
SpecifiedOutcomesSpecifiedOutcomes
Evaluation Strategy 2: Start Building the Model with Activities
Evaluation Words of Wisdom Don’t worry about the labels
– Determinants are often called outputs– Outcomes are sometimes called impacts– Sometimes logic models have separate columns
for inputs, activities, outputs, short-term outcomes, long-term outcomes, and impacts (whew!).
Focus instead on the causal mechanisms.– If you think through the logic of why your program
works, you can fill in any format the funder/evaluator requires of you.
– ….and actually benefit from the experience!
74
75
Small Group Work
What was the most recent NEW thing you did at work?
Why did you do it? What was the Evidence-Base for
decision-making? Did it make a difference? –
create a logic model with inputs, activities (outputs), and outcomes to evaluate
Contextual Factors(Individual Student Characteristics – Race, Special Needs, Language, achievement; Family Characteristics – Income, parental education, structure;School Characteristics- Average Test Scores, Mobility, %FRL, Attendance; Community – % Poverty, % Affluent, % Idle youth)
TCAP Staff
Administrative Buy-in/Support
Organizational Structure
INPUTS ACTIVITIES OUTPUTS OUTCOMES
Example of a Complex Logic Model: Turning the Corner on Achievement Program
June 1, 2006 Center for Adolescent Health, JHU
Relationship with Families
School-Level
TCAP-Level
Motivation toAchieve
New PossibleIdentities
Social Development
Intermediate Outcomes Improved
AcademicAchievement
Ultimate Outcomes
ImprovedAttendance
Access toCommunityResources
Academic•Integrated•Acceleration•Targeted
Arts & Culture
StudentAssessment
Lifestyles and Career
Reduced HealthRisk Behaviors
Improved Mental Health
Students Experience…Physical and Psychological SafetyAppropriate StructureSupportive RelationshipsOpportunities to BelongPositive Social NormsSupport for Efficacy and MatteringOpportunities for Skill BuildingIntegration of School, Family, and CommunityExposure to new opportunities
Staff Experience…Support Tools to perform
Families Experience…Engagement in EducationParenting SupportAccess to resources
Psychological and Emotional Development
Professional Development
Organizational Structure
Communication skills
Family Support Services
Community Service
Improved Social Relationships
Relationship with Teachers
Leadership skillsStudents and Families
Relationship w/ families
Relationship w/ school
Improved Civic Engagement
Evidence-Based Decision Making
78
What’s the issue?
How big is it?
How important is it?
What is the cause?What should we do about it?
What are we going to do about it?
How are we going to do it, when, with what results?
How will we know if we have made a difference?