Upload
daisy-watkins
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Making a DifferenceThrough Program Evaluation
Michael Quinn PattonSeptember 21, 2010
Program Evaluation Quality Assurance Summit
1
LBJ
2
3
Globalizationof the evaluationcommunity –of which you are all a part.
Burkina Faso
4
Evaluation
as a transdisciplineand a profession…
with quality assurance Standardsand professional developmentopportunities.
5
Standards – Quality AssuranceUtility – ensure relevance & useFeasibility – realistic, prudent,
diplomatic & frugalPropriety – ethical, legal, respectfulAccuracy – technically adequate to
determine merit or worth
For the full list of Standards:www.wmich.edu/evalctr/checklists/standardschecklist.htm
6
Meta-evaluation
Evaluating evaluationsfor
ongoing learning, improvement and quality assurance
7
Quality AssuranceEnsuring Value through
Evaluation
E-VALU-ate
8
Quality Assurance
Quality controland/or
Quality enhancement
9
Personal Factor:
Intended Useby
Intended Users
10
Critical success factors:
There are five key variables that are absolutely critical in evaluation use. They are, in order of importance:
• People–People
•People–People
»PEOPLE11
Big news
Evaluators are people too
12
Basic premise
Value and quality assurancein Evaluation
comes through EVALUATORS
13
Essential skills
Add value by being good at what you do
andknowing your roleand playing it well
14
Know thyself
Connais-toi toi-même
Greek: γνῶθι σεαυτόν
Latin: nosce te ipsum
15
Essential Competencies for Program Evaluators
Jean King, a recipient of the American Evaluation Association’s prestigious Alva and Gunnar MyrdalPractice Award, has worked for a number of years with colleagues and students conducting researchon and developing a framework for Essential Competencies for Program Evaluators (Ghere et al. 2006;King et al., 2001).
The final product is a taxonomy of essential program evaluator competencies organized into six primary categories.
16
Essential Competencies1. Professional Practice: knowing and observing
professional norms and values, including evaluation standards and principles.
2. Systematic Inquiry: expertise in the technical aspects of evaluations, such as design, measurement, data analysis, interpretation, and sharing results.
3. Situational Analysis: understanding and attending to the contextual and political issues of an evaluation, including determining evaluabiity, addressing conflicts, and attending to issues of evaluation use.
17
Essential Competencies4. Project Management: the nuts and bolts of
managing an evaluation from beginning to end, including negotiating contracts, budgeting, identifying and coordinating needed resources, and conducting the evaluation in a timely manner.
5. Reflective Practice: an awareness of one’s program evaluation expertise as well as the needs forprofessional growth.
18
Essential Competencies
6. Interpersonal Competence: the people skills needed to work with diverse groups of stakeholders to conduct program evaluations, including written and oral communication, negotiation, and cross-cultural skills.
19
Essential skills
Add value by being good at what you do
andknowing your roleand playing it well
20
The Nature of Expertise
and the role of expert
21
What’s it take to achieve world class expertise?
10,000 hours
22
Identities we take on
• Evaluator• Methodologist• Researcher• Auditor• Inspector• Learning facilitator• Judge
These roles are necessary, important and useful,but there’s another role that offers value:
The jester
23
24
SPEAKING TRUTH TO POWER
25
Jesting is Serious Businessbecause
Speaking TRUTH to Power is serious business
26
In the beginning…
27
28
About this book
Preview this book
Shake Hands with the Devil By Roméo Dallaire
This is a preview. The total pages displayed will
be limited. L
Loading...
Loading...
29
resultSearch Booksresultoe9S6SgfGeneral RPP1PP1ACfU3U1E
Value-added Challenge:Matching the evaluation process and design to the nature of the situation
to achieve
intended use by intended users:
Contingency-based
Evaluation 30
Evaluation’s General, Generic Value
Value of a Particular Evaluation
and Specific Approach31
Mrs. McCave and her 23 Daves
32
Not just experiencebut
Focused Practice and
REFLECTIONon experience
to achieve situational responsiveness 33
Story about ongoinglearning
34
35
Executing
Doing what we know how to do and know we should do
36
The Fine Art of Selling • Successful Selling Methods • What Makes a Great Salesperson?• The Secret to Selling• Increasing Sales Now• Sales Tips from Great Salesmen• Topping the Sales Charts• Breakthrough Strategies For Selling to Difficult People • Expert Hands-On Sales Techniques • Improving Sales Performance - Where to Begin?• Sales Techniques Revealed
37
Lessons from Sales Experts
1. Believe in your product2. Know your product3. Connect your product to what your
customer needs: listen
communicateconnect
38
Elevator speech test
Can you say clearly, in 30 seconds what your product is, why you believe in, and say it in a way that connects with the person you’re talking to?
39
Elevator test
Believe in your product
40
Bob Stake, “Beyond Neutrality: What Evaluators Care About”
41
Beyond Neutrality: What Evaluators Care About
1. We often care about the thing being evaluated.2. We, as evaluation professionals, care about evaluation.3. We advocate rationality.4. We care to be heard. We are troubled if our studies
are not used.5. We are distressed by underprivilege. We see gaps
among privileged patrons and managers andstaff and underprivileged participants and communities.
6. We are advocates of a democratic society.SOURCE: Robert Stake (2004:103–107).
42
Know your product
43
Value-added Challenge:Matching the evaluation
process and design to the nature of the situation to achieve intended use by
intended users
Contingency-based
Evaluation44
Connect your product to what your customer needs:
listencommunicate
connect
45
46
47
PROCESS USE
48
Three Cups of TeaBaltistani proverb:
First cup you share, you are a stranger.
Second cup, you are an honored guest.
Third cup, you are in relationship.
49
Enhancing Quality in Evaluation
1. Believe in evaluation2. Know evaluation (and yourself as
evaluator)3. Connect to your primary intended
users: listen
communicateconnect
50
51
52