iSTEP: Technology Field Research in Developing

Communities 15-302

Instructor: M. Bernardine Dias

CAs: Sarah Belousov and Ermine Teves

Spring 2009

LectureMonitoring and

Evaluation

3

Outline

• M&E• IRB• Q&A

Monitoring and

Evaluation

5

The Need • Project managers, donors, and other

stakeholders need to know the extent to which their projects are meeting their objectives and leading to their desired effects.

• M&E build greater transparency and accountability in terms of use of project resources.

• Information generated through M&E provide project staff with a clearer basis for decision-making.

• Future project planning and development is improved when guided by lessons learned from project experience.

http://www.passia.org/seminars/2002/monitoring.htm

6

Relationship Between M&E

http://www.passia.org/seminars/2002/monitoring.htm

7

Logical Framework Approach

http://www.passia.org/seminars/2002/monitoring.htm

8

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

9

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

10

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

11

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

12

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

13

Methods and Tools

http://www.passia.org/seminars/2002/monitoring.htm

14

Selecting M&E Methods • What information is needed?• Of this information, how much can be collected and analyzed

in a low-cost and practical manner?• How accurate will the information be?• Will the methods get all of the needed information?• What additional methods should and could be used if

additional information is needed?• Will the information appear as credible to decision makers?• Are the methods appropriate for the target group? If group

members are illiterate, the use of questionnaires might not be appropriate unless completed by the evaluators themselves.

• Who can administer the methods? Is training required?• How can the information be analyzed?

http://www.passia.org/seminars/2002/monitoring.htm

15

Types and Models • Many types of evaluations – but 2 main

categories:– Formative (process evaluation)

• Examines the development of the project and may lead to changes in the way the project is structured and carried out

– Summative (outcome or impact evaluation)• Examines what a project has actually accomplished in terms of

its goals

• Popular evaluation methods– Pre-test Post-test model

• Measures the situation before the project starts and repeat the same measures after the project is completed. The differences or changes are attributed to the project

– Comparison Group model• Compares project results on two comparable groups at the same

period of time, where one group represents beneficiaries of the project and the other represents a group that has not benefited from the project. The difference between the two groups are attributed to the project

• Key Evaluation Indicators are: Effectiveness, Efficiency, Relevance, Impact, and Sustainability

http://www.passia.org/seminars/2002/monitoring.htm

16

Planning a Monitoring System 1. Decide what should be monitored

• The careful selection of monitoring indicators organizes and focuses the data collection process

2. Decide how to gather information• Select methods to track indicators and report on progress

(observation, interviews, stakeholder meetings, routine reporting, field visits, etc.)

3. Decide who will gather information and when/how often • Project staff at various levels will do most data collection,

analysis and reporting. Staff should agree on what the monitoring report should include. 

4. Decide on feedback process• Progress reports should be reviewed by project staff and major

stakeholders. Feedback should be collected by project managers on a regular basis.

5. Allocate resources for monitoring plan• Needed funds and staff time should be allocated to ensure

effective implementation

http://www.passia.org/seminars/2002/monitoring.htm

17

Planning an Evaluation1. What are the purposes of the evaluation?

• Identify a manageable number of evaluation purposes and prioritize them. Ask who needs what type of information and for what reason.

2. What evaluation model is the most appropriate for the project?

• The evaluation model that a specific project would utilize should be selected during the project design phase.

3. What is the timing of evaluation within the project cycle?• Evaluation timing is determined by the project plan, the identification

of significant problems during monitoring, donors’ request, etc. 4. What is the scope and focus of the evaluation?

• Identify the geographic area, type of activity and time period that the evaluation should cover to clarify the types of questions to be asked.

5. What methods of gathering data will be used?• Existing data should be identified and assessed. Note that extensive

data gathering is time-consuming, expensive, and can result in a lot of unnecessary information. 

6. What resources are needed for the evaluation?• Sufficient human, financial and logistic resources should be allocated.

http://www.passia.org/seminars/2002/monitoring.htm

18

Analysis and Dissemination

• Data management is very important and needs to be planned so that effective analysis and dissemination can follow

• Both quantitative and qualitative data can be useful/required

• Numerical and statistical methods are often used to analyze quantitative data

• It is important to find effective ways to combine the quantitative and qualitative data analysis to produce overall results and recommendations

• Effectively disseminating evaluations (through presentations and reports) is important for many reasons

• Suggested content for evaluation report:http://www.passia.org/seminars/2002/ME/Chapter6.htm

http://www.passia.org/seminars/2002/monitoring.htm