Monitoring & Evaluation “Watch your school succeed”Sep 28 2012

Maram Barqawi, M&E Manager, mbarqawi@hotmail.com Jordan Education Initiative

Monitoring & Evaluation Watch your school succeed

1 Introduction M&E2 What is monitoring? 3 What is Evaluation? 4 Theory of Change4 Agreeing on Outcomes5 Selecting Key Indicators to

Monitor Outcomes

6 Set Baseline and Target 7 Outcome Matrix 8 Indicators Matrix

Monitoring & Evaluation

Management tool

Measures how well your school is performing

0

20

40

60

120

80

100

Monitoring & Evaluation

M&E shifted from auditing and inspections

to supporting the performance of program’s

work to achieve the demanded goals

The Power of Measuring Results

If you can demonstrate results, you can win public support

If you can not recognize failure, you can not correct it

If you can not see success, you can not learn from it

If you can not reward success, you are probably rewarding failure

If you can not see success, you can not reward it

If you do not measure results, you can not tell success from failure

Monitoring & EvaluationWhat is Monitoring?Day-to-day follow up of activities during implementation to measure progress and identify deviations

Answers the question, “what are we doing?”

Monitoring ToolsExamples of monitoring tools in the school: • Personal diaries• Check lists, • Attendance sheets • Field visits reports • Students scores

Monitoring & Evaluation

What is Evaluation?assessment of overall achievement and impacts

Answers the question, “what have we achieved and what impact have we made”?

Theory of ChangeIDEAYou want to make CHANGE

in your school?

You have to have a THEORY about how to make that CHANGE.

Theory of Change It is an explanation of how the use of (inputs) by (activities) produce a group of (outputs and outcomes) that sets the stage for producing (impact).

Theory of Change is how to use the

resources you have to win the change you

want

Theory of Change (Logical Model)

Outcomes• Changes in behaviors or

skills as a result of the implemented project.

Outputs• Products and services

produced

Activities• Tasks personnel undertake

to transform inputs to outputs

Inputs• Financial, human, and

material resources

Goal(Impacts)

• Long-term, widespread improvement in the school

Imp

lem

en

tati

on

Resu

lts

Theory of Change - Adult Literacy program

Outcomes• Increased literacy skill;

more employment opportunities

Outputs • Number of adults completing literacy courses

Activities • Literacy training courses

Inputs• Facilities, trainers,

materials

Goal(Impacts)

• Higher income levels; increase access to higher skill jobs

Imp

lem

en

tati

on

Resu

lts

Why Is It Important to Choose Outcomes?

“If you don’t know where you’re going, any road will get you there.”

Paraphrased from Lewis Carroll’s Alice in Wonderland

“If you don’t know where you’re going, any road will get you there.”

Paraphrased from Lewis Carroll’s Alice in Wonderland

Why Is It Important to Choose Outcomes?

Note: When Choosing Outcomes, Remember – “Do Not Go It Alone!”

key stakeholder

Participative approach

How to Developing Outcome Statements?

School is not clean

Clean environment is created in the school

Children are dropping out of School

ICT skills are low in the school

Encouraged families To Keep Kids In School

Improved ICT literacy among students and teachers

From To

How to Developing Outcome Statements?

Consider this Outcome Statement: “Students in rural areas improve learning and gain better quality of life”

What are the measurement issues??

Input? Activity? Output? Outcome? Impact?

Theory of Change - Simple Example (Training)

Input •Materials •Trainers •Participants

Activity • Conduct

training• Followup

Output • # of

Participants trained

• # of workshops

Outcome • Capable

participants to apply M&E techniques

Impact • Better

decisions

Theory of Change (Logical Model)

Group work 1 (15min):

• Suggest Theory of Change related to school’s intervention, discuss the component and draft the flow chart.

Outcomes must be translated to a set of key indicators

Outcomes must be translated to a set of key indicators

To know your progress you need to measure outcomes?

IndicatorAn outcome indicator identifies a specific numerical measurementthat tracks progress (or not) toward achieving an outcome

Selecting Outcome IndicatorsA good performance indicator must be CREAM:

Clear (Precise and unambiguous)

Relevant (Appropriate to subject at hand)

Economic (Available at reasonable cost)

Adequate (Must provide a sufficient basis to assess performance)

Monitorable (Must be amenable to independent validation)

Indicators ?• % of students finished the Elementary

level• % of students finished the Secondary

level• % of children in child labor• More teachers hired

Outcome: Reduction in dropping out of school.

Indicator

• You will need to develop your own indicators to meet your own needs.

• Developing good indicators often takes more than one try!

• Arriving at the final indicators you will use will take time!

• Pilot, Pilot, Pilot!

Outcomes MatrixOutcome Indicators Baseline

Targets

June. 2013 Dec. 2013 June . 2014

Outcome Matrix

Group work 2 (10min):

• go back to your intervention ToC and suggest indicators for each outcome

Indicator Baseline

“If you do not know where you are, you will have difficulty determining where you need to go.”

Harry HatryUrban Institute, 1999

“If you do not know where you are, you will have difficulty determining where you need to go.”

Harry HatryUrban Institute, 1999

• Is the starting point from where implementation begins, or the situation prior to a development intervention, against which progress can be assessed or comparisons made.

Indicator Baseline

Indicators Matrix Indicators are usually summarized in a matrix which will

guide the process of monitoring those indictors.

Indicators matrix include information about the data

sources ,collection methods, frequency a, it also assign

the responsibilities for data collection ,analysis and

reporting.

Indicators Matrix Outcome1 (objective1)

Indicator Baseline Data source

Data collection method

Who will collect data

Frequency of data

collection

Who will analyze and report data Target

1.

2.

3.

4.

Source of Indicators • Written records (paper and

electronic)• Individuals involved with the

program• General public• Trained observers• Mechanical measurements and

tests

Setting Indicators’ Targets

A target is :“ the desired level of performance to be

reached within a specific time”

Be Realistic

Data Collection Methods:

Quantitative Research Methods1.Focus Groups2.Interviews3.Observations

Quantitative Research Methods1.Surveys2.Records

How to develop a Monitoring & Evaluation planSe

t out

com

es

Deve

lop

Indi

cato

rs

Set I

ndica

tors

ba

selin

es

targ

ets

Data

colle

ction

& R

epor

ting

Data

Ana

lysis

Diss

emin

ation

an

d Us

e

Indicators Matrix

Group work 3 (15min):

• Use the outcome matrix to develop the indicators matrix.