Upload
aftab-hamid
View
216
Download
0
Embed Size (px)
Citation preview
8/3/2019 Lecture 11 PMEC
1/30
8/3/2019 Lecture 11 PMEC
2/30
M&E system
Theextent of progresstoward theprojects
objectives, and thereasons for success or
failure.This is what a monitoring andevaluation system provides.
8/3/2019 Lecture 11 PMEC
3/30
Monitoring and Evaluation
Monitoring and evaluation is the process of collecting andanalyzing information about the project that tells youwhether you are on track to reach your objectives, andwhether or not the project achieved or contributed to the
desired impact. In order to know whether or not you are on track to
achieving your projects objectives, you must monitor theproject during implementation as well as evaluate itsimpact at the end of the project.
It is necessary to plan for monitoring and evaluationwhen you design your project; this will help you both todesign an effective program and ensure that you plan(and budget) for appropriate monitoring and evaluation
activities.
8/3/2019 Lecture 11 PMEC
4/30
Project M&E
The process of monitoring and evaluation generally
helps you answer the following questions:
Inputs: Were program inputs available,
adequate, timely?
Activities: Were activities performed onschedule?
Outputs: Were outputs produced? Were they ofacceptable quality?
Effects: Were effects observed?
Impact: Was impact achieved?
8/3/2019 Lecture 11 PMEC
5/30
What is the need for conductingEvaluation?
The evaluation of projects contributes towards
Decision-making,
Strategic Planning
Reporting or Modification
Organizational learning
Accountability
Impact
8/3/2019 Lecture 11 PMEC
6/30
Project Evaluation
Project evaluation is a systematic and objectiveassessment of an ongoing or completed project.The aim is to determine the relevance and level ofachievement of project objectives, developmenteffectiveness, efficiency, impact andsustainability.Evaluations also feed lessons learned into thedecision-making process of the project
stakeholders.Evaluation is based on certain standards, criteria andmeasures
8/3/2019 Lecture 11 PMEC
7/30
Project Evaluation
Evaluations provide an opportunity for in-depth reflection on the strategy andassumptions guiding the project.
They assess progress made towards theachievement of a projects objectives andmay recommend adjustments to itsstrategy.
They are also a means by which to assesshow well project-level actions link tohigher level strategies and objectives
8/3/2019 Lecture 11 PMEC
8/30
Key Elements of ProjectEvaluation
The concept and principles of projectevaluation;
Policies for project evaluations and roles
and responsibilities;
The implementation of project evaluationand evaluation report;
Follow-up, dissemination andknowledge sharing of evaluationoutcomes
8/3/2019 Lecture 11 PMEC
9/30
Why Organizations need tocarry out Project Evaluation?
Evaluation is an important tool that your organization canuse to demonstrate its accountability,
Improve its performance, increase its abilities for
obtaining funds or future planning, and fulfill theorganizational objectives
By communicating the results of the evaluation, yourorganization can inform its staff, board of directors,
service users, funders, the public, or other stakeholdersabout the benefits and effectiveness of yourorganizations services and programs
8/3/2019 Lecture 11 PMEC
10/30
Project Evaluation is anAccountability Function
By evaluating a project, you monitor the process to
ensure that appropriate procedures are in place
forcompleting the project on time, and you identify
and
measure the outcomes to ensure the effectiveness
and achievements of the project. All these efforts
make you organization capable of reporting
answering all inquiries, and being accountable for
itslans
8/3/2019 Lecture 11 PMEC
11/30
Evaluation
Identify ways to improve or shift your project activities
Prepare project reports (e.g., mid-term reports, final reports) Inform internal and external stakeholders about the project Plan for the sustainability of the project
learn more about the environment in which the project is beingor has been carried out
learn more about the target population of the project
Present the worth and value of the project to stakeholders andthe public
Plan for other projects
Compare projects to plan for their futures Make evidence-based organizational decisions Demonstrate your organizations concerns to be accountable
for implementing its plans, pursuing its goals, and measuringits outcomes.
8/3/2019 Lecture 11 PMEC
12/30
Evaluation Plan
Creating an evaluation plan is the very first
stage of conducting an evaluation. Your plan
should answer questions such as:
What is being evaluated? Why is it being evaluated?
Who wants the evaluation?
Who will do it? How will it be done?
What results do you expect?
8/3/2019 Lecture 11 PMEC
13/30
Purpose of Evaluation
To assess the degree to which projectobjectives were achieved.
To document the lessons learned.
To provide recommendations for projectdevelopment and improvement.
To examine the changes that resulted fromdoing the project.
To provide input to guide decision makingfor the upcoming renewal and extension of
project funding.
Id if i E l i
8/3/2019 Lecture 11 PMEC
14/30
Identifying Evaluationstakeholders Stakeholders are the individuals or organizations
that have an interest in your project; they maymake decisions, participate in the projectactivities, or be affected by those activities.
Your project may have both primary and
secondary stakeholders. Primary stakeholders are those who are closelyand directly involved in or affected by the resultsof your project (e.g.the participants themselvesand an organization that has invested in your
project). Secondary stakeholders are those who are less
involved and less affected by your project but mayhave some benefits in your project (e.g., an
organization that is interested in knowing aboutthe results of your project).
8/3/2019 Lecture 11 PMEC
15/30
Evaluation Types
Selecting an evaluation type provides direction foryour evaluation. It helps keep the evaluationprocess focused on its main purpose anddetermines the evaluation questions that should beanswered and the data that should be collected.The most common types of evaluation are: Formative
Process Summative Outcome
8/3/2019 Lecture 11 PMEC
16/30
Types of Evaluation
Formative Evaluation is an ongoingevaluation that starts early in a project. Itassesses the nature of the project, the needs
the project addresses, and the progress andimplementation of the project. It can identifymajor gaps in the projects content and
operational aspects (i.e., what was done andhow) and suggest ways to improve them.
8/3/2019 Lecture 11 PMEC
17/30
Types of Evaluation
Process evaluation is used to monitor activities to makesure a project is being implemented and completed asdesigned and on time.It can be complementary to formative evaluation.
Although formative evaluation has a larger scope thanprocess
evaluation, there are many similarities between them: bothfocus on the effectiveness and the operational aspect of a
project; both start at a very early stage of a project and can beperformed by internal staff; and both require a strongmonitoring mechanism to track operational activities andto collect information related to the process
8/3/2019 Lecture 11 PMEC
18/30
Types of EvaluationSummative Evaluation is an overall assessment of the projects
effectiveness and achievements. To provide adequate information, asummative evaluation requires a set of well-defined goals and objectives
for the project and a plan that keeps the focus of evaluation on the end-
results.
It reveals whether the project did what it was designed to do.
It provides information for future planning and decisions and usually iscompleted when the project is over.
This type of evaluation usually does not directly affect the current
project, but it helps stakeholders decide the future of this or similarprojects.
This type of evaluation should be conducted by either an externalevaluator or by staff and practitioners who are well equipped with therequired knowledge, resources, and tools for this type of evaluation
T f E l i
8/3/2019 Lecture 11 PMEC
19/30
Types of EvaluationOutcome Evaluation assesses the extent to which a project has
achieved its
intended effects, and other effects it could have had on the projectsparticipants or the environment. It focuses on immediate, intermediate, orultimate outcomes resulting from the completion of the project. The
resultsof this evaluation should identify both the desirable and undesirable
impacts
of the project.
To conduct this type of evaluation, need a good understanding ofthe project process, the outcomes, and the relationship betweenthe two
Then, you need to collect sufficient evidence during the project todemonstrate how certain outcomes are related to a specific set ofproject activities
In this type of evaluation, identifying evaluation indicators,collecting appropriate data, and interpreting the results are crucialas well as challenging
8/3/2019 Lecture 11 PMEC
20/30
How to select and EvaluationType The objectives and priorities of your project The purpose of the project evaluation
The nature of the project (i.e., whether it is
process-oriented or outcome-oriented) The time frame for conducting the
evaluation (i.e., during or after the project)
How, and by whom, the results will be used
The time frame and budget for completingthe evaluation.
8/3/2019 Lecture 11 PMEC
21/30
Evaluation Questions
Evaluation questions are the key questions that youneed to answer to ensure the successful completion ofyour project or to understand its impact, effectiveness,and achievements.
Asking and answering the right questions will lead to
useful evaluation results that can be easilycommunicated with external audiences or put to use inyour organization.
Evaluation questions play a crucial role in the analysis
and interpretation of the data you collect. Therefore, you should allow for enough time and
discussion with your key stakeholders to develop andselect them.
You may also need to revisit and revise these
8/3/2019 Lecture 11 PMEC
22/30
Evaluation Tools Evaluation tools help you gather the information you
need to answer your evaluation questions. They can be different from the tools you use to
carry out the core activities of the project. Forexample, conducting workshops to train
organizations is a project tool; interviewing keyparticipants to ask their opinions about the contentand success of the training workshops is anevaluation tool.
Evaluation tools can use both formal and informalmethods for gathering information.
Formal evaluation tools include focus groups,interviews and survey questionnaires.
Informal evaluation tools include observations,informal conversations, and site visits.
8/3/2019 Lecture 11 PMEC
23/30
Quantitative and QualitativeInformation
Depending on your evaluationquestions, you may need a tool thathelps you gather quantitative information
by numbering, rating and rankinginformation.
Or you may need a tool that helps you
gather qualitative information such asstories, anecdotes, opinions about aparticular aspect of the project
8/3/2019 Lecture 11 PMEC
24/30
8/3/2019 Lecture 11 PMEC
25/30
Evaluation Indicators
Indicators are measurable factors or evidence thatshows the extent of the projects progress, success,or achievements. Identifying indicators can help youin collecting useful data and in your search forrequired evaluation tools and information sources.
Indicators can be quantitative, such as the number ofparticipants, number of website visits, andrate or rank of opinions.
They can also be qualitative, such as positive or
negative feedback, problems, complaints, andcomments. You can also use some project outputs as indicators
if they show the projects progress toward anobjective
8/3/2019 Lecture 11 PMEC
26/30
Types of Data
Quantitative data describe a situation by using numbers. Thesedata could be the actual numbers; for example, the number ofevents your organization held, the number of services youprovided, the number of materials you produced.
Also, the quantitative data could be some numbers that
represent the relationships among concepts; for example, thenumbers that express the rates (e.g., in a six-point scale) orranks of opinions, feelings, skills, and knowledge.
You may collect this type of data when you use a surveyquestionnaire, an evaluation form, a test (e.g., knowledge test),or some outputs of your project.
Once you complete your analysis, the most commonquantitative findings you look at and use for interpretationscould be total numbers, percentages, frequencies, averages,ratios, ranks, and orders.
8/3/2019 Lecture 11 PMEC
27/30
Types of Data
Qualitative data take the form of words,texts, ideas, and stories that explain asituation.
You typically collect this data when projectparticipants describe a situation in theirown words, whether you use formal orinformal conversations, interviews, or focus
groups.
You can also use project-relateddocuments such as notes about direct
observations, minutes of meetings, andournal entries
8/3/2019 Lecture 11 PMEC
28/30
Evaluation Budget
Project evaluations can be costly, particularly ifthey aim to capture various aspects of both theprocess and outcomes of the project.
You should plan your budget in a way thatmakes your evaluation realistic, manageable,efficient, and productive.
In some cases, projects have a fixed budgetand evaluators need to adjust their activities to
that budget. In other cases, evaluators need to develop a
budget.
8/3/2019 Lecture 11 PMEC
29/30
8/3/2019 Lecture 11 PMEC
30/30
When to use Evaluation
Any time there is an unexpected result orperformance outlier that requires furtherinvestigation
When resource or budget allocations are beingmade across projects, programs, or policies
When a decision is being made whether or not toexpand a pilot
When there is a long period with no improvement,and the reasons for this are not clear
When similar programs or policies are reportingdivergent outcomes