Developmental Evaluation Cameron D. Norman PhD CE Principal CENSE Research + Design Adjunct Professor, Dalla Lana School of Public Health, University of Toronto
@cdnorman August 28, 2013
Outline
• Thinking about the problem(s)
• What is Developmental Evaluation?
• Methods
• Case Examples
• Discussion & dialogue
Same Evaluation Tools / "Different Thinking Tools
Complexity
Example: Organizing a children’s party
Evaluation & Measurement: Problems & Opportunities
What is an effective coffee date?
Is that the right question?
What is an effective tree?
What is the best practice for being a tree?
What is a fully optimized tree?
Do you know what is inside before you crack it open?
Confusing moving things with static things
The Importance of Theory
What is Developmental Evaluation?
Developmental Evaluation
• First proposed by Michael Quinn Patton (1994)
• Combines Utilization-Focused Evaluation with concepts from Complexity Science
• Recognizes that social innovation and resiliency require a different form of feedback to enable programs to adapt and thrive in changing conditions
• An approach to evaluation, not a specific method.
• Strategic learning / real-time evaluation
Developmental Evaluation
• Assists social innovators in learning strategically about how to intentionally develop their program while acknowledging to complexity
• Supports adaptation to changing conditions
• Can identify and nurture promising interventions
• Documents innovators’ actions
• Identifies emergent processes and outcomes; enhances resiliency
• Determines if an innovation is ready for formative evaluation
Complicated Systems Complex Systems Single causality Multiple causality
Designed & intended Adaptive & emergent
High degree of certainty High degree of uncertainty
Assumed predictability Anticipation of events
Focus on ‘boxes’ Focus on ‘arrows’
Structures determine relationships Structures & relationships interact
Linear Non-linear
Noise, tension & flux undesirable Noise, tension & flux embraced
Adaptation to static environment Interaction with static environment
Reaction Engagement
Adapted from http://www.healthandeverything.org
Program Improvements Program Developments
1. Add a new topic to a training curriculum
1. Change the scope, sequence and delivery of curriculum
2. Provide staff training to enhance skills
2. Change job descriptions, reconceptualize fundamentals of the job
3. Expand recruitment effort to a target area
3. Question and re-evaluate recruitment strategy
4. Expand staff to serve increasing numbers using the same model
4. Expand staff, change the model to suit new scale and scope of practice
5. Fine-tune programming based on participant feedback
5. Change programming to fit ever-changing realities
6. Add more content to provide additional information
6. Change the way the audience interacts with content altogether
Design Thinking
Developmental Design
Methods & Methodology
Mindfulness
Ethnography
Network Analysis
Timelines
Visual Thinking
Gigamapping http://www.systemsorienteddesign.net/
Idea Generation
Knowledge Mobilization
Strategic Learning
Discussion
Cameron D. Norman PhD
@cdnorman
censemaking.com