View
217
Download
0
Category
Tags:
Preview:
Citation preview
©Performance Results, Inc.
Developing and Using Logic Models
for Project Planning and Evaluation
NIH Science Education Projects Annual Conference
May 15, 2012
Why Outcomes Matter
Constructive FailureUnconstructive Failure
http://www.philanthropyroundtable.org/topic/excellence_in_philanthropy/failure_in_philanthropy
Results-based Evaluation
Three buckets of data….
Evaluations
Process
PeopleResults
The Evaluation Process Should Answer:
How has my program made a difference?
How are the lives of the program participants improved as a result of my program?
Results-based Evaluation
What are Outcomes?
A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.
Results-based Evaluation
What is a Program?
Activities and services leading towards intended outcomes
Has a definite beginning and end Designed to change attitudes, behaviors, or
knowledge, or increase skills and abilities related to assumed need
Results-based Evaluation
Program Purpose
Program purpose is driven by assumptions about need. It relates to the organization’s mission statement. It defines audience, services, and outcomes.
In summary...We do what, for whom, for what outcome or benefit?
Results-based Evaluation
What: We provide assessment, goal development and one-to one tutoring
To whom: Adults living in Reynolds County and surrounding area who express an interest and demonstrate a need for our services
For what outcomes: Adults will improve their reading, writing and communication skills and achieve their personal literacy goals
Program Example: Literacy
Results-based Evaluation
What: We will establish a community-wide partnership with health professionals, teachers, parents, children, health researchers and the media to provide raise awareness about obesity among children, promote healthy diets and lifestyles, create family health plans and support parents efforts to help their children reach their diet and fitness goals.
To whom: Parents, children Pre-K and K, and community members in Springfield
Results-based Evaluation
Fit from the Start
For what outcomes: 1. Children understand the dangers of bad eating
habits and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a
healthy lifestyle5. Families create, use and adopt a regular healthy
meal and exercise plan 6. The community demonstrate a greater
understanding of the cost of poor dietary and exercise habits of its citizens
7. The community make a commitment to sustaining programs to help families become and stay healthy and active
Results-based Evaluation Fit from the Start
Logic Models
The Evaluation Plan:
An outcomes logic model (evaluation plan) is a clear, graphic representation of the links between program activities, the results these activities produce, and how the results will be measured.
Inputs Activities Outputs Outcomes Impact
Resources; people,
materials, supplies, money
Program Process
Products “Counts and Amounts”
Knowledge, Skills, Behaviors,
Attitude changes of targeted
population
Community-level change
EXAMPLE 1
Logic Models
Impact Outcome Indicators Inputs Activities
CommunityResults
Intended individual
change
Observable and measurable behaviors or conditions
Resources; people,
materials, supplies, money
Program Process
EXAMPLE 2
Logic Models
Outcome Indicators Applied to Data Source Data Interval Target
Intended individual
change
Observable and
measurable behaviors or conditions
The population(s)
being measured
Information sources
When information is
collected
The #/% who will achieve
the outcome
The Practical Model
Logic Models
Inputs Services OutputsResources;
people, materials, supplies, money
Program Process
Products “Counts and Amounts”
Start the Plan with Outcomes First
A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
Outcomes are Logical and Sequential
Immediate Intermediate Long-term IMPACT
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
Outcomes
Outcomes Indicators Data Source Applied to Data Interval Target
Fit from the Start1. Children understand the dangers of bad eating habits
and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a healthy
lifestyle5. Families create, use and adopt a regular healthy meal
and exercise plan 6. The community demonstrate a greater understanding
of the cost of poor dietary and exercise habits of its citizens
7. The community makes a commitment to sustaining programs to help families become and stay healthy and active
Indicators Measure the Outcomes
State the measurable conditions or behaviors chosen to show an outcome was achieved.
Show what you hope to see or know Are observable evidence of
accomplishments, changes, or gains.
Indicators
Outcomes Indicators Data Source Applied to Data Interval Target
Outcome # 1 Indicator:Children understand the dangers of bad eating habits and a sedentary lifestyle
The #/% of children who can identify at least 2 poor eating habits that harm one’s health and
The #/% of children who can identify at least 3 consequences of an inactive lifestyle
Indicators
Outcome # 2 Indicator:
Children improve their diet
The #/% of children who eat 80% of the six food groups at least three times a week
Outcome # 3 Indicator:
Children adopt an active lifestyle
The #/% of children who exercise or engage in active play for 30 minutes at least 6 times a week
Fit from the Start
Outcome # 4 Indicator:Families support one another in maintaining a healthy lifestyle
The #/% of families who have more than one family member supporting the child in the Fit from the Start program
Indicators
Outcome # 5 Indicator:
Families create, use and adopt a regular healthy meal and exercise plan
The #/% of families who take an active role in meal planning and choosing foods which are healthy
The #/% of families who plan and have outings and activities at least 4 times a week month
Fit from the Start
Outcome # 6 Indicator:The community demonstrates an understanding of the cost of poor dietary and exercise habits of its citizens
The #/% of community members who accurately complete the online survey on fitness and diet
Indicators
Outcome # 7 Indicator:
The community makes a commitment to sustaining programs to help families become and stay healthy and active
The #/% of project partners who complete their obligations to this project and
The #/% of project partner who continue their efforts beyond the grant period
Fit from the Start
Data SourcesData captured at critical points in time
Look at indicators to see what data sources are required.
Follow up – Continued Benefit
End – Results - Outcomes
Middle - Progress
Beginning - Baseline
Outcomes Indicators Data Source Applied to Data Interval Target
Data Sources
Used to assess:
Behaviors
Life Condition-Life Status
Attitudes
Skills
Knowledge
Outcomes Indicators Data Source Applied to Data Interval Target
Where Will the Information Come From?
Data sources are tools, documents, and locations for information that will show what happened to your target audience. Pre-post test scores Program records Assessments Records from other organizations Observations Portfolio review
Data Sources
Outcomes Indicators Data Source Applied to Data Interval Target
What kind of data do you need?
Often data sources are categorized into types:
Primary – Information you have collected yourself Secondary – Existing information collected from
someone else Qualitative – Types of information that is describing
quality rather than quantity – answers questions concerning “how”
Quantitative – Information concerning counts, amounts, percentages; answers questions concerning “how much”
Data Sources
Outcomes Indicators Data Source Applied to Data Interval Target
Applied to: Target Audience
Who Gets Measured?
The target audience to whom the indicator applies
Decide if you will measure all participants, completers of the program, or another subgroup
Special characteristics of the target audience can further clarify the group to be measured
Outcomes Indicators Data Source Applied to Data Interval Target
Data Intervals
When to Measure?
Data intervals are the points in time when data are collected:
Data can also be collected at the end of an activity or phase and at follow-up
Data are usually collected at program start and end for comparison when increases in skill, behavior, or knowledge are expected
Outcomes Indicators Data Source Applied to Data Interval Target
Targets
Targets of Performance
Targets, also known as goals, are the stated expectations for the performance of outcomes.
Stated in terms of a number and/or percent Meets influencers’ expectations May be estimated by the program’s past
performance
Outcomes Indicators Data Source Applied to Data Interval Target
Logic Models
Inputs:
The resources you will dedicate to your program.
Staff Facilities Equipment Consultants Materials Curricula
Logic Models
Inputs:
Staff – researchers, health professionals, sports and dance teachers, public relations and web development team
Hands-on activities and curricula Healthy Lifestyle curricula Informal science education PSA’s Print media Website
Fit from the Start
Logic Models
Services:
Services are activities, tasks and events that engage the target audience. Services are designed to have a direct effect on program outcomes.
Workshops Programs Mentoring Classes Training Job Try-outs
Logic Models
Services:
Case assessment Diet and exercise plan and monitoring Age appropriate games and activities Field trips Hands-on experiences Dance and sports orientation Website development Campaign to promote healthy diets and
lifestyles
Fit from the Start
Logic Models
Outputs:
A direct program product, typically measured in numbers:
# Participants # Education sessions
# Plans developed # Online surveys taken
# Plans completed # Activities completed
# Field trips # Non-completers
Characteristics of People Served
Age Race Health Status Income Urban vs. Rural Primary Language Transportation Education Level
Logic Models
Reporting
In summary:
We wanted to do what? We did what? So what?
Three buckets of data….
Evaluations
Process
PeopleResults
Success….
Claudia B. HornPerformance Results, Inc.
20203 Goshen Road Suite 314
Gaithersburg, Maryland 20879
www.performance-results.net240-207-3071
Claudia.horn@performance-results.net
www.resultsonline2.net
Recommended