Embed Size (px)
Citation preview
Case Studies:
The Impact of Engagement in Program Theory on Organizations
Evaluation 2015: Exemplary Evaluations in a Multicultural World
November 12, 2015
Presenter: Charles Gasper, TCC Group
Think about impact
Programmatic Intervention
Programmatic Intervention
Programmatic Intervention
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Mid-Term Outcome
Mid-Term Outcome
Mid-Term Outcome
Long-Term Outcome
Long-Term Outcome
Major ImpactEarth Shattering Impact
Evaluation
Now think about evaluation
Programmatic Intervention
Programmatic Intervention
Programmatic Intervention
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Mid-Term Outcome
Mid-Term Outcome
Mid-Term Outcome
Long-Term Outcome
Long-Term Outcome
Major ImpactEarth Shattering Impact
Evaluation
Now take it one step further
Programmatic Intervention
Programmatic Intervention
Programmatic Intervention
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Mid-Term Outcome
Mid-Term Outcome
Mid-Term Outcome
Long-Term Outcome
Long-Term Outcome
Major ImpactEarth Shattering Impact
Evaluation
Or if we recognize that evaluation can have programmatic effects
Programmatic Intervention
Programmatic Intervention
Programmatic Intervention
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Programmatic Outcome
Mid-Term Outcome
Mid-Term Outcome
Mid-Term Outcome
Long-Term Outcome
Long-Term Outcome
Major ImpactEarth Shattering Impact
Evaluation
But what if we change our focus?
What if the program is just part of the story?
Organizations are multicellular creatures
Program Program
Program Program Program
Not to mention part of systems
And those parts affect one another
Systems Approach
So where are the red apples? Program
Partners
Environment
Organization
So where are the red apples? Program
Partners
Environment
Organization
So where are the red apples? Program
Partners
Environment
Organization
Evaluation impact on organizations
Rudimentary model of change for Public Media
Improved Community Perception
Improved Partner Perception
Improved Partner
Collaboration
Organizational Learning • Changes in how work is done • Changes in capacity to do the work
Our Work Increased
Awareness / Understanding
Discussion American Graduate
Champions
Increased Volunteerism &
Donations
Things Get Better for
Young People
Rudimentary model of change for Public Media
Improved Community Perception
Improved Partner Perception
Improved Partner
Collaboration
Organizational Learning • Changes in how work is done • Changes in capacity to do the work
Our Work Increased
Awareness / Understanding
Discussion American Graduate
Champions
Increased Volunteerism &
Donations
Things Get Better for
Young People
Documented organizational changes
1. Creation of GSD team
2. Movement to impact focus
3. Rudimentary cross-platform/department activity
4. Development of decision matrix for events
Organizational process change in funding
Evaluation
Funding
Decision
Compliance
Evaluation
Learning
Evaluation
Organizational
Culture &
Interests
Financials
Interim/Final Reports
Staff Site Visits
Focus of
Learning
Individual
Program
Initiative/Funding
Stream
Portfolio
Individual
Program Learning
Initiative/Funding
Stream Focused
Learning
Portfolio
Focused
Learning
Foundation
Foundation
Focused
Learning
Documented organizational changes
• Transfer to portfolio focused learning
• Staffing shifts
• Major modifications to design
• Board engagement
Summary of organizational change
• Shifts in organizational focus
• Changes in:
– Staffing
– Activities
– Collaboration
The results are in…
Changes in organizational culture
Efficiencies of labor
Improved quality of products / outcomes
What about the partners?
• Shifts in amount and quality of collaboration
• Changes in organizational activities
Partner impact?
2014 Partnerships
2015 Partnerships
The results are in…
Access to underutilized resources
Improved quality of programming
Increased revenue
What does this look like?
• Dedicated evaluation team member
• Collaboration with key internal stakeholders
• Organic restructuring of work
Looking forward
• Is there a relationship between engagement in evaluation and organizational capacity?
