Upload
aylin-seller
View
237
Download
3
Tags:
Embed Size (px)
Citation preview
Johns Hopkins University School of Education
Johns Hopkins University
Evaluation Overview
Evaluation Leaders…
Michael Scriven: Claremont Graduate UniversityDavid Fetterman: Stanford University Medical School
DefinitionThe identification, clarification, and application of defensible
criteria to determine an evaluation object’s value (worth or merit) in relation to those criteria.
Examining and judging to determine value…
Evaluation Overview
Evaluation uses inquiry and judgment methods, including:1. Determining standards for judging quality and deciding
whether those standards should be relative or absolute,2. Collecting relevant information, and3. Applying the standards to determine value, quality, utility,
effectiveness, or significance.
Evaluation leads to recommendations intended to optimize the evaluation object in relation to its intended purpose(s) or to help stakeholders determine whether the evaluation object is worthy of adoption, continuation, or expansion.
Evaluation Overview
Research vs. Evaluation
Research Evaluation
Purpose Add knowledge to the field: grow theory
Help stakeholders make a judgment or decision
Who sets the agenda Researcher defines hypotheses
Stakeholders in collaboration with evaluator
Generalizability of results
Designed to be generalizable
Designed to report on a specific context
Criteria Internal validity: causalityExternal validity: generalizability
Accuracy: accurate reflection of realityUtility: practical informationFeasibility: evaluation is realistic, prudent, frugalPropriety: legal, ethical
Preparation Trained in depth in a single field
Interdisciplinary training
Informal vs. Formal Evaluation
We evaluate everyday!
Informal evaluations occur all the time… however, they often lack: breadth and depth systematic procedures formally collected evidence
Informal evaluations can be influenced by experience, instinct, generalization, and reasoning.
Purposes of Evaluation
According to Mark, Henry, and Julnes (1999), evaluation has 4 main purposes:
1. Assessment of merit and worth2. Oversight and compliance3. Program and organizational
improvement,4. Knowledge development
Formative vs. Summative Evaluation
Formative: Provides information for program
improvement
Summative: Provides information to serve decisions
or assist
Activity…
Formative vs. Summative Evaluation Activity
Internal vs. External Evaluation
Internal (employees/members): More familiar with organization and program history Knows decision-making style of organization Is present to remind others of results now and in the
future Can communicate technical results more frequently
and clearly External (outside organization)
Can bring greater credibility, perceived objectivity Typically brings more breadth and depth of technical
expertise Has knowledge of how other similar organizations
and programs work
Discussion
Describe a situation in which an internal evaluator would be more appropriate than an external evaluator. What is the rationale for your choice?
Now Describe a situation in which an external evaluator might be more appropriate.
Trends in Program Evaluation
Increasing priority and legitimacy of internal evaluation Expanded use of qualitative methods Strong shift toward multiple and diverse methods in program
evaluation (mixed methods approach) Increased use of program evaluation by foundations and
other agencies in the not for profit sector Increased education and involvement of stakeholders in
evaluation to empower them, increase buy in Advances in technology make evaluation more accessible Performance measurement in the federal government and
not for profit organizations Increase in standards-based assessment in education as a
means of tracking performance Growth of evaluation internationally
Empowerment Evaluation• Critical Friend or Coach—be on their side but politely
pose questions to help refine and improve
• Cycles of Reflection and Action—provide data to inform decision making, then act on it
• Culture of Evidence—develop a pattern of data collection and documentation to support positions
• Community of Learners—encourage peers to learn together
• Reflective Practitioner—thoughtfully consider data to guide practical day-to-day activities
Setting Goals: Creating a Road Map
Mission Vision and Value Statements Project Goals, Objectives, and Indicators
of Success Theory of Change Logic Model
Goals…
MUST BE
S.M.A.R.T
S.M.A.R.T Goals
Activity
See you all Next Week!