Introduction to Developing, Monitoring and Evaluating Projects
Noah Keuzenkamp
TGEU Capacity Building Officer
Overview
Project development Monitoring & Evaluation Putting both together: LogFrames Group Exercise Presentations A few points to keep in mind Questions
Baseline
Who has developed a project before?
Who has written a fundraising application with a monitoring & evaluation plan before?
Who has monitored a project before?
Who has evaluated a project before?
Who knows what a LogFrame is?
Thinking about project development
with a logic model
Key terms
Goal (Impact)
Outcomes
Outputs
Activities
• The outcome at the end of the project – often the expected benefits to the target group(s)
• Key words: “increased, improved, etc.”• Example: Participants have increased knowledge of M&E
• The broad impact to which the project contributes – e.g. at the national level or over the long-term. Your project may not achieve this immediately/by itself.
• Example: The trans* community obtains more funding OR: To build the capacity of the trans* community to obtain funding
• The direct/tangible results (goods &services) that the project delivers (mainly under your control)
• Key words: “delivered, produced, conducted, etc.”• Example: M&E workshop delivered to 20 participants
• The main tasks carried out to eventually deliver the outputs• Key words: “prepare, design, develop, research, etc.”• Example: Develop a workshop on M&E
Key terms: another example
GoalThe broad impact to which the project contributes:Trans* concerns are better represented in politics
Outcomes Expected outcome/benefit at the end of the project:Trans* rights were mentioned 5 times in parliament
OutputsDirect results delivered:Briefings delivered to 30 parliamentarians
ActivitiesTasks carried out: Identify friendly parliamentarians and organise meetings PL
AN
NED
INTE
ND
ED
Thinking about project development
with a logic model
How do we know we are approaching our goal?
Monitoring:
the systematic and routine collection of information during the implementation of a project to assess whether the project is progressing as planned
Evaluation:
a systematic and ‘objective’ assessment of a completed project, especially its outcomes and impact, in order to inform future strategic decisions and projects
Development, Monitoring & Evaluation in the Project Cycle
Why conduct M&E?
In practice often: Because it is a donor condition To show that you are doing what you are funded for To obtain more funding in the future
But, it can also: Help you learn and improve Help you understand how change works (or doesn‘t) Help you create, preserve and disseminate knowledge Make you accountable to your communitie(s)
Logical Framework Matrix (LogFrame)
Helps you systematically develop, plan, implement, monitor and evaluate a project
Presents clear and organised connections between impact, outcome, outputs and activities
Many donors use/require (a kind of) LogFrame
Using a LogFrame
Project Summary
Indicators Means of Verification (MoV)
Assumptions
Goal
Outcomes
Outputs
Activities
If...then
Monitoring & Evaluation
Horizontal logic
Project Summary
Indicators Means of Verification (MoV)
Assumptions
Goal Variables/ information used to measure change in a process or phenomenon
The source of the indicators and the means of obtaining them.
= M&E method for obtaining information
Assumptions that need to be fulfilled to eventually achieve your goal
Outcome
Output
Activity
Indicators: examples
„More positive public attitude towards trans* people“
Increase in number of positive news items Decrease in number of hostile statements by politicans Increase in positive attitude of individuals (survey) Gender identity is included in anti-discrimination laws ... Etc.
Indicators: examples
„Trans* people are more visible in society“
... ... ...
Indicators: SMART
S Specific E.g. nr of people, % increase
M Measurable Can it be measured in general? Can you measure it?
A Achievable Is this realistic?
R Relevant Does it measure the ‘concept’? Or: reliable
T Time-bound By when or from when to when
Not SMART: People will know more about M&E
SMART: By the end of the workshop, 70% of people who didn‘t know what a LogFrame is report that they know „fairly well“ how to fill in a LogFrame
Means of verification (Mov)
Method for collecting indicators to conduct monitoring and evaluation, e.g.
Surveys / Questionnaires Focus groups Observation Official statistics News reports Photos Testimonies / stories Etc...
Project Summary
Indicators Means of Verification (MoV)
Assumptions
GoalThe trans* activist community is well funded
Income of participating organisations rises by 20% by 2016
Online survey - M&E plans are key in grant writing-Participants will write grants
Outcomes 70% of participants have increased knowledge of M&E
Nr. of participants who report having gained knowledge
Questionnaire handed out after session (self-report)
LogFrames are not too complex
Outputs Workshop on M&E delivered to 20 people
Nr. of participants
Sign-in sheet People are interested in M&E
Activities Develop a workshop on M&E
InputsStaff, time, technology, etc.
CostWork time, travel, accommodation etc. = ?
PreconditionsTGEU has capacity to deliver
This workshop: full example
Noah‘s ArkProject Summary
Indicators Means of Verification (MoV)
Assumptions
Goal Earth remains populated by people and animals
Census count 100 years after the flood
Census survey - Plant life returns- All animal pairs can/want to reproduce
Outcomes All people and animals have survived the flood on the Ark
Census count after landing
Checklist upon unloading
- No one will die from illness- Food will not run out
Outputs Ark is loaded with people and pairs of all animals
Census count on the Ark
Checklist upon boarding
Animals can be found and will happily get on the Ark
Activities
Build ark
InputWoodNoah’s time and skills
CostBarter
PreconditionsCan find a safe plot of land to build on
Sometimes it‘s better not to be Noah
When developing a new project, resist the temptation to (always) start with the Ark, i.e.(specific) activities:
„We should run a training, because that‘s what we usually do“
Starting with the goal or the problem and then working from top to bottom is better for productive brainstorming and creative problem solving
Working from top to bottom also helps identifying the best target groups or stakeholders
Use the LogFrame as a tool for logical and creative thinking
Group Exercise
Brainstorm about what the main problems are Your project only needs to contribute to the goal Complete 1-2 outcomes/outputs/activities first, then add
more if you have time Resist the temptation to start with the activities Write your LogFrame on flipchart paper Agree on presenter(s)
Time for preparation: 30 mins
A few points to keep in mind
When writing a grant proposal Language matters (concise and precise) Terminology choice matters (what does the donor
use?) Describing activities: usually active verbs and present
tense, e.g. „compile a factsheet“, „perform a play“ Describing outputs and outcomes: usually past tense,
e.g. „confidence has increased“, „question has been raised in parliament“
A few points to keep in mind
Don’t let the language of donors confuse you
e.g. outcome = purpose / (intermediate) objective / result The same outcomes can be measured with different indicators:
chose the one(s) best suited to your goals and resources (money, time, skills, technology, …)
Evaluation does not always have to be quantitative: qualitative data, such as personal stories, can be used, too
M&E is political: who determines what the outcomes should be? who is asked for their opinion? who conducts the evaluation? ( feminist evaluation)
What should be SMART?
SMART outcome with a variable as indicator: Outcome: 80% of participants have increased their knowledge
to a „good“ level by the end of the project Indicator (variable): Number of participants with „good“
knowledge
Outcome with a SMART indicator: Outcome: Participants‘ knowledge is increased Indicator: 80% have a „good“ level of knowledge by the end of
the project
Concept vs Indicators
Trans* people are SAFER in our city
1: Trans* people‘s report of how safe they feel
2: Nr. of attacks on
trans* people
reported to police
3: The nr. of anti-Pride protestors
4: Nr. of trans* customers at LGBT bars
Indicators
Sometimes they are direct:
Planned outcome: A trans* person gets elected into parliament
Indicator: Did a trans* person get elected into parliament?
But usually they are indirect:
Especially when talking about social change, attitudes, knowledge, awareness, skills, etc.
Example: this workshop
I could monitor: Baseline: what was people‘s knowledge before? Status: how many people are in the room? How many
people are listening? Trend: are people leaving? Switching off? Getting
more engaged? Implementation: is the workshop being implemented?
Are there interruptions? (Ongoing) effectiveness: are people learning
something right now?
Example: this workshop
I could evaluate: Efficiency: was a workshop the best/cheapest method
for people to learn about M&E? Effectiveness: did people learn about M&E? Outcome/Impact: do people feel better prepared to
complete an M&E plan? will they use the knowledge when writing funding applications? will they get more funding?
Sustainability: will people still remember this in a year? will they pass the knowledge on?
What is a project?
„An individual or collaborative enterprise that is carefully planned to achieve a particular aim” (Oxford Dictionary)
Has a start and end date Has constraints (time, money, resources,...) Aims to create change
Introduction
“I would like to have money for a project to…”