Evaluation Terms of Reference

Stong End Programme Evaluation TOR- September 2021 1

Evaluation Terms of Reference

Stoung Area Programme TOR Writing Date: August 31, 2021

Programme Number: 02606

Programme Phase: Final Implementation Phase

World Vision Cambodia


Contents I. Acknowledgements ............................................................................................................. 3 II. Affirmation .......................................................................................................................... 3

III. III. Glossary ..................................................................................................................... 4

IV. VI. Introduction ............................................................................................................... 5

1. Evaluation Summary ............................................................................................................... 5

2. Description of Programme or Project Being Evaluated ..................................................... 6

3. Evaluation Target Audiences (i.e., For whom is the evaluation intended?) ....................... 7

4. Evaluation Type ................................................................................................................... 7

5. Evaluation Purpose and Objectives..................................................................................... 8

5.1 Evaluation Purpose: .......................................................................................................... 8

5.2 Evaluation Objectives: ...................................................................................................... 8

6. Evaluation Methodology ...................................................................................................... 9

7. 7. Limitations ..................................................................................................................... 10

8. 8. Authority and Responsibility ......................................................................................... 11

8.1 Team Members and Roles .............................................................................................. 11

8.2 Partners .......................................................................................................................... 12

9. Team Advisors .................................................................................................................. 12

10. Logistics ......................................................................................................................... 12

11. Products ........................................................................................................................ 13

12. Budget ............................................................................................................................ 13

13. Documents .................................................................................................................... 13

14. Lessons Learned ............................................................................................................ 13

15. Appendices .................................................................................................................... 13


I. Acknowledgements Firstly, I would like to express deeply thanks to District Governor, Operational District of Health

Chief, District Office of Education (DoE) Chief, Health Center Chief, Commune Leaders, Commune

Council, Commune Committee for Women and Children (CCWC), Village Leader (VL), Youth Groups

(YG), Village Health Support Groups (VHSGs), Agriculture Cooperative (AC), Community Lead Child

Monitoring (CLCM), Sanitation Action Group (SAG), Water Management Committees (WMC) who

are actively participating in programme activities and also provided the consultation and advice to

develop the Terms of Reference (ToR) for Evaluation.

Secondly, I really appreciated for Stoung Area Program (AP) team such as Community Development

Facilitators (CDF) and Community Engagement and Sponsorship Plan (CESP) project coordinator who

provided necessary data for this evaluation TOR. Moreover, special thanks to M&E team and especially

Senior M&E Officer, Mr. Keo Sereivuth, who provided technical support for the whole process of

writing of the ToR.

Thirdly, I would also like to express my special thanks to management team, including Mr. Leng Vireak,

Operation Director (OD), Mr. Ny Sopharan, Associate Director (AD) and Mr. An Lyhorn, Regional

Manager (RM), of WVI-C, who always provided support for the development and review of the ToR.

Finally, we would like to give deeply thanks to Support Office, WV New Zealand to review and approve

the fund for the programme evaluation.

II. Affirmation Except as acknowledged by the references in this paper to other authors and publications, the

evaluation ToR described here consists of our own work, undertaken to advance learning, as part of

the requirements of World Vision’s Design, Monitoring and Evaluation learning system.

Primary quantitative and secondary data are collected throughout the evaluation process which

remains the property of the communities and families are described in this document. Information and

data must be used only with their consent.


III. Glossary

AC Agriculture Cooperative

AD Associate Director

AP Area Program

CATCH Caring All Together for Child’s Health

CCWC Commune Council for Women and Children

CDF Community Development Facilitator

CESP Community Engagement and Sponsorship Plan

CLCM Community Led Child Monitoring

CP&A Child Protection and Participation

CSMP Child Sponsorship Management Project

CY Calendar Year

DME Design, Monitoring and Evaluation

DMQ Decision Making Question

DoE District office of Education

FGD Focus Group Discussion

FY Fiscal year

HC Health Centre

HHS Household Survey

KII Key Informant Interview

LEAP Learning through Evaluation with Accountability and Planning

M&E Monitoring and Evaluation

MY WASH Mobilizing Youth for Water, Sanitation and Hygiene

NGO Non-Government Organization

OD Operational District ( Government Health Structure)

OD Operation Director ( World Vision Cambodia)

RC Register Child

RM Regional Manager

SAG Sanitation Action Group

SO Support Office

STO Sectoral Technical Officer

TCEP Together for Child’s Education and Protection

ToR Terms of Reference

TP Technical Project

VHSG Village Health Support Group

VL Village Leader

WASH Water and Sanitation Hygiene

WMC Water Management Committees

WVC World Vision Cambodia


IV. Introduction

Stoung district is located in the West of Kampong Thom province, where is approximately 52 Km from

Kompong Thom town with good condition of National Road number 6. It is bounded by Preah Vihear

province in the North, Siem Reap province in the West, Tonle Sap Lake in the South and Prasath

Balang and Kompong Svay districts in the East.

Stoung district is comprised of 13 communes with 133 villages with a total population of 110,596

people from 21,025 families. The total land area is 221,242 hectares in which farming land equal to

31,333 hectares. Moreover, the Buddhism is the official national religion and 99% of the people in the

district are Buddhist and most people are subsistence farmers. Stoung Area Programme implements

its activities in 3 communes including Banteay Stoung, Chamnar Leu and Trea communes with 27 target

villages are the primary focus area.

Stoung AP has started since October 2005 with AP assessment and design under support of WV New

Zealand. The first phase of programme covered 10 villages of the three target communes. With this,

AP team went through a long process of 1 year to conduct an assessment and design sponsorship

management and community mobilization projects for implementing in FY 2008. Later on, AP designed

02 sectoral projects, food security and education, in FY 2009 to implement in FY 2010, and finally the

program designed the WASH project in FY 2010 for the implementation in FY 2011. After starting

with 10 villages in October 2008, the program extended more target areas to 05 villages in FY 2009

and 05 more villages in FY 2010. The Stoung AP has been implemented its sector project in FY 2008

after 2 years-program assessment and design with the new approach, focused on community

mobilization and empowerment. This approach, the AP has engaged community people with

development activities in their community through the appropriated facilitation skills of the AP staffs.

In the five-year redesign of the phase 2 from 01 October 2013 to 30 Sep 2017, AP implemented

the Education project, Health & Nutrition project, food security project and Mobilize community for

Child Protection project and the Implementation Social Accountability Framework (ISAF).

Since 2018, AP has adapted LEAP 3 and conducted Decision Making Question (DMQ) with community

people, children and local authorities and key partners including DOE, OD, HC and school directors

to collect the information and issues related to child well-being. Based on the result of DMQ, Strong

Area Programme has selected Community Engagement and Sponsorship Plan (CESP) Project, and three

other Technical Projects (TPs) such as TP-Education and life skill, TP-Child Protection and TP-Integrate

Nutrition. During the new strategy revision in FY20-FY22, AP selected 2 core project models for

implementing such Child Participation & Advocacy and Education for Life and Community Engagement

Sponsorship Plan (CESP) for implementing on ward.

1. Evaluation Summary

Programme/Project Stoung Area Programme 02606

1. Community Engagement and Sponsorship Plan (CESP)

2. Child Protection and Participation (CP&A)

3. Education for Life

4. Nutrition (Heath/Nutrition and Wash)

Programme Phase Transition phase from 01 October 2017 to 30 September

2022 of Final Implementation Phase Evaluation Type End programme evaluation

Evaluation Purpose To assess the impact and progress of the programme goal and

outcome and level of sustainability, community ownership and

empowerment, as the result of this program implementation.

Primary Methodologies Quantitative:

1. Household survey (HHS)

2. Health and Nutrition Survey


Qualitative:

1. Focus Group Discussion (FGD)

2. Key Informant Interview (KII)

3. Observation and field visit

Evaluation Start and

End Dates December 2021

Anticipated Evaluation

Report Release Date

February 2022

2. Description of Programme or Project Being Evaluated

The Stoung AP has started its operation in Stoung district in 2005 with AP assessment and design

under support from WV New Zealand. During the first 5-year of implementation, the programme

selected to implement five projects including Food Security Project (FSP), Water Sanitation and

Hygiene Project (WASH), Education, and Child Sponsorship Management and Community Mobilize

project (2008-2012). In 2013, the redesign of phase II for programme implementation continued to

work with key partners and primary focus area. With this, The Stoung AP selected five projects

implemented such Education, Health & Nutrition, Food Security project, Child Sponsorship

Management and Mobilize Community for Child Protection. Otherwise, AP also implemented the

Implementation Social Accountability Framework (ISAF) project from 2015 to 2022.

In this transition phase, Stoung AP has still selected 02 technical projects (TPs) such Education for Life,

Child Protection & Participation and one Community Engagement and Sponsorship Plan (CESP) Project

for implementing, ISAF, Covid-19 onward. The projects below will be evaluated with specific outcome

and models as following: Community Engagement and Sponsorship Plan: To enhance the potential for World Vision's

technical programmes to produce sustainable improvements in child well-being.

1. Communities demonstrate increasing ownership over planning, implementing and monitoring

their development and child well-being activities.

2. Effective child participation and protection strengthened.

3. Improved policy implementation (Enabling model I-SAF)

4. Objective 1: Scale up preventive measures to limit the spread of disease (COVID 19)

5. Objective 2: Strengthen health systems and workers

6. Objective 3: Support for children impacted by COVID-19 through education, child protection,

food security and livelihoods

7. Objective 4: Collaborate and advocate to ensure children are protected

Child Protection and Participation: Children and adolescents are protected from all forms of

violence and participating as agents of positive change with following outcome

1. Children and adolescents, especially the most vulnerable girls and boys, are meaningfully

participating and are engaging with local decision makers or local government to end violence

against children in their communities and countries.

2. The community, especially parents and caregivers, demonstrates attitudes and behaviours that

provide a caring and protective environment for all children and adolescent especially the Most

Vulnerable Children (MVC).

3. CP formal and informal mechanisms are collaborating effectively to protect girls and boys,

especially most vulnerable children, from abuse, exploitation, neglect and other forms of violence

4. Governments take all appropriate measures to ensure the protection of girls and boys. (NO

level)

Education for Life: Children are equipped with literacy for life with the four following outcomes

1. Increased support by families & communities for children's learning.

2. Strengthened classroom & school-based instruction in literacy,

3. Increased proportion of schools achieve "Quality Learning Environment"

4. Government systems supporting children’s education strengthened.


Nutrition project has been implemented in the AP target during the previous phase and has

dropped out in FY20. Health& Nutrition will also include in the end programme evaluation. The goal

objective of Health & Nutrition is to increase in children who are healthy and well-nourished with

the following outcomes

1. Increase in children who are healthy and well nourished

2. Children are free from infection and disease

3. Caregivers exhibit good infant and young child feeding practices

4. Improved legal and policy frameworks

Programme

Direct

Participants

Type CESP Education CP&A Nutrition Total

Men 7000 3377 1991 7000 19,368

Women 8585 4237 2542 8585 23,949

Boy 4217 662 1266 4217 10,362

Girl 4206 676 1334 4206 10,422

Funding

Source WV New Zealand, Sponsorship Funding

3. Evaluation Target Audiences

The main purpose of the evaluation is to assess the impact and progress of the programme goal and

outcome and level of sustainability, community ownership and empowerment, as the result of this

program implementation toward the transformation lives of the communities and children resulting

from the programme implementation.

Additionally, this evaluation result will be clearly informed other sector programme staff and Support

Office (SO) colleagues to know how they can better provide support to AP staff and to help improve

the other future AP implementation to maximize impact in the lives of community members and

vulnerable children. To achieve the goal, the evaluation target audiences are mentioned below:

1. Local Partners/stakeholders– The local cooperation partners level including Youth Groups

(YGs), Community Small Groups (CSG), Commune councils, district government offices:

Development sector, Child Protection sector and Health sector will learn from their involvement

in community development activities and review and share achievements, challenges, gaps, and

opportunities in relation to their capacity for leadership and management, and level of

responsibility. They will play major roles in planning their community future plan.

2. World Vision in Cambodia – WV Cambodia will learn from its current development strategy,

and its impact on the well-being of community members, especially children both micro and macro

levels. The evaluation will help WV to see how and to what extent its strategies have contributed

to the changes of the lives of community members and understand the mechanism and the aspect

of sustainability.

3. World Vision New Zealand – The Support Office of WV New Zealand will be provided with

new evidence illustrating AP progress, learning and AP contribution toward improvement of child

well-being.

4. Evaluation Type

According to LEAP (Learning through Evaluation with Accountability and Planning) guidelines the

evaluation will be an end programme evaluation. The evaluation should be conducted no later than five

years after the start of first programme phase. The external consultant team will lead the whole process

of end programme evaluation from planning to write the evaluation report.


5. Evaluation Purpose and Objectives

5.1 Evaluation Purpose:

To assess the impact and progress against the program goal and outcomes especially measure the level

of sustainability, community ownership and empowerment, as the result of this program

implementation.

5.2 Evaluation Objectives:

Effectiveness: To assess the progress and effectiveness of AP interventions to improve well-being

the community members

To what extent has the program achieved its goal and program objective.

What have been the factors that affect the program/project implementation?

Impact: To assess the impact of the Programme implementation contribute to the well-being of

children

What have been the changes (attitude, knowledge, and practice) of community members result

from AP intervention?

What is the long-term social change at the community level the project has contributed to?

Sustainability: To determine the sustainability of potential programme impact and results by assessing

the level of community ownership and strength of partnership with community partners by the AP.

What are the hindering factors to the function and ownership of communities, stakeholders, and

partners?

What enabling factors exist to support the function and ownership of communities?

Which organizations, groups, individual are critical in fostering sustainability and what should

their role be?


6. Evaluation Methodology

Initial Planning

Document Review After reviewing the documents listed in section 13, information gaps will be determined, and the scope

of other data collection methods refined accordingly.

Quantitative Data Collection and Analysis

Household Survey DME team with cooperation with every Sectoral Technical Officer (STO) has developed generic survey

questionnaires. The generic questionnaires will be used for this quantitative data collection. The

evaluation lead will check the questionnaires against the matrix to ensure that it covers all in indicators

stated in Matrix and add more question as appropriate to the generic questionnaires. The indicators

from relevant secondary strategies will, as far as possible, be included within the survey questionnaire.

The questionnaire will be finalized by the evaluation team in consultation with the AP team.

The HHS data collection team will be formed by the AP team with assistance from the DME Officer.

Key community members and local stakeholders will be involved with household data collection

including HHS usage training.

The calculation of household survey sample size will be based on 95% confidence level, a design effect

of 1.0 and margin of error of 0.05. The sampling frame will be households with children 0-18 years and

the mothers with children under 24 months old in the AP target communities. The sample size

calculated using the actual household statistic of each subgroup.

There are 3797 households with children 0-18 years old in the AP target communities, and as the

result of the sample size calculation, 384 households to randomly select for the survey. The head of

household or spouse will be targeted for interview and if the head or spouse are not available for

interview, any members who are over 18 years old or are less than 60 years old can also be targeted

for interview. The care survey questionnaires will assess education, child protection, and economic of

the target community members

There are 1563 mothers with children under 24 months in the AP target communities, and as the

result of the sample calculation, 339 mothers with children under 24 months to randomly select for

the interview. Only the mothers will be targeted for the interview, if the mothers are not available for

the interview, the mothers with children under 24 months in the nearest household will be selected

as the replacement. The Health and Nutrition survey will assess health care of the mothers and

children.

Qualitative Data

Focus Group Discussions Focus Group Discussions (FGDs) will be held with program beneficiaries and program partners to

obtain qualitative responses and data of relevance with respect to the scope and requirements of the

Evaluation Matrix (in the Appendix X).

As standard practice for AP evaluations at least 15 Focus Group Discussions will be conducted with

programme stakeholders, including (but not limited to government authorities, Community Small

Groups, school teachers, school students, community members including Parents of RC, Registered

Children, and mothers with children under 24 months. The FGD participants will be organized

homogenously to help ensure participation and minimize the scope for domination or elite capture

during FGD. The numbers of participants for each FGD will be between 8 to 12 participants.


The consultant team will discuss with the AP team to determine groups of stakeholders or partners,

then the respondents for the FGD will be randomly selected from groups identified.

Key Informant Interviews Between 10-12 key Informant Interviews (KIIs) will be held with programme stakeholders, such as

commune and district governors, relevant department representatives, peer/partner agency

representatives and some AP staff to obtain qualitative data following the information needs outlined

within the Evaluation Matrix in the Appendix A.

Field visit and observations It is expected that there will be field visits by the evaluation team to more directly observe actual

programme practice or observable results from the AP’s activities, which should also be guided by the

Evaluation Matrix. A Field visit and observation guide will be developed by the evaluation lead and

oriented to evaluation team prior to the visit. The visit will assess on the functionality and utility of

hardware/facilities that was provided by ADP such as well, latrines etc.

Qualitative Data analysis

The consultant team will use his/her own approaches in doing qualitative data analysis and the

preliminary result will be discussed with the AP team and M&E.

Evaluation findings dissemination

It is required that once the analysis has been completed; it will be summarized into easy way to

understand modules such as factsheets and share to AP team to validate the findings and explore more

understanding to the AP implementation and recommendation.

7. Limitations

Unpredictable and unstable situation of Covid-19 pandemic in the communities may prolong, so

AP will keep contact with local authorities to follow up the situation and get permission to

conduct data collection with individual or with a small group of participants. Moreover, AP will

prepare the preventing materials for using during data collection such as face mask, alcohol and

keep distancing. If the pandemic of Covid19 remain the issues that can lead to mobility restrict

and face to face interview is prohibited, the phone interview will be considered for the

substitution of face-to-face interview and the qualitative will also consider conducting through

phone interview with group of FGD members virtually from their home.

In every 5 years, Cambodia country always conduct commune election one year before national

election, so in 2022 all the communes in Cambodia usually conduct election campaign awareness

and voting event. To avoid from this duplicate event, AP will keep consult with local authorities

and plan ahead to conduct evaluation at the beginning the calendar year which will be in

December 2021.

Some of the randomly selected households might migrate out to seek jobs outside their

communities, so it might be time consuming for the surveyors that affects to the completion of

the survey as deadline. However, the evaluation team shall work with AP team, local authorities,

and community partners to ensure that those households shall be replaced before the fieldwork

for evaluation commenced.


8. 8. Authority and Responsibility

8.1 Team Members and Roles

This evaluation will be led externally by consultant team with geographically guidance by AP team with

the participation of M&E officer to help ensure the quality of the data collected. M&E officer will ensure

the quality of the evaluation for the whole process of evaluation from writing TOR, planning and data

collection and the evaluation report.

The detail roles and key partners are in the table below:

Evaluation

Phase

Responsible

Person Primary Task

Planning

AP team

Together with DME officer, develop ToR

Refine ToR, budget and schedule with evaluation team

leader.

Obtain the list of target respondents for random selection

to collect the data. (List of households with children 0-18

years and list of mothers with children under 24 months)

Gather all relevant document for the consultant review

after the consultant selected

Senior M&E

Officer

Review and finalize the TOR and send to Regional Manager

to review and send to Support for further comments and

endorsement of the TOR.

Participation in the recruitment of qualified consultant to

lead the whole evaluation process

Procurement and

People & Culture

Prepare the announcement to recruit consultant

Coordinate in selecting the shortlist of the applicants

Coordinate with applicants and the panel to interview

selected consultant for the interview

Prepare the contract for the consultant

Data

collection

AP team

Inform the authority on the schedule of the evaluation and

get their approval to do data collection in the communities.

Prepare data collection schedule with consultant team

leader

Consultant team

leader

Provide training on qualitative data collection tools and

skills to data collection team

Closely monitor the data collection team to ensure quality

of data

Conduct the analysis and interpretation of technical data.

Provide training on quantitative data collection tools to all

enumerator to understand data collection process and the

tools

Closely monitor the data collection to ensure quality data

Develop Database for data encoding using any software

he/she familiars with. Data cleaning and analysis

Regional Manager

(RM)

Troubleshoot any management issues arose during the

course of the evaluation.

Support the analysis and interpretation of technical data


Evaluation

Phase

Responsible

Person Primary Task

Reporting

and follow-up

Consultant

Formulation of recommendation for the redesign with

WVC staff through preliminary findings presentation to AP

and Management team

Prepare the evaluation report

Circulate the draft for comments and inputs and feedback

Send final Report to AP, RM and OD for review and

comments

RM, M&E and AD

Provide input to the findings

Develop practical recommendations

Review and approve final evaluation report

AP team Provide input to the findings for validation

Develop practical recommendations

8.2 Partners

Partner Role SO Input into Evaluation TOR

Review Evaluation Report and provide feedback

AP stakeholders Will fully participate in the evaluation process as the informants

9. Team Advisors

Here is the proposed Team advisor to ensure the quality of evaluation process and report.

No. Position Roles

1 Associate Director

of WVC

The Associate Director will provide overall support and oversight

for coordination of the evaluation, especially for trouble shooting any

structural or systematic issues.

2 Senior Manager –

ME quality assurance

The Senior Manager DME will provide overall guidance to the

evaluation and ensure that it follows LEAP guidelines and meets

donor expectations and approves the evaluation design (namely the

evaluation matrix and sampling framework). He will approve

evaluation ToR and matrix as well as any significant changes.

3 Regional Manager The RM will provide overall support with regard to evaluation data

collection processes, logistic and other needs identified by the

consultant Team.

10. Logistics

The logistics for the evaluation will be strongly supported by Admin and Finance Assistant (AFA),

driver, project coordinators, and so everything is managed such as meeting rooms, Computers, LCD,

handouts, refreshment, etc.

No Responsible

staff Materials needed Description of Responsibilities

1

Community

Development

Facilitator and

Project

Coordinator

List of project partners, HH

statistic, and village maps, list of

CSGs, list of wells/pond, list of

local NGOs, record of capacity

self-assessment,

All relevant documents must be in

hand before starting process.

Involve in the evaluation process

and data collection.


11. Products

The key findings will be organized into a summary sheet by project and share this to the ADP and

management team to validate and or further review findings and to explore more recommendation.

The final evaluation report will be circulated to line management of WVC as well as to SO for

feedback.

The report from all ADP team, line management and support office will be incorporated into the

final report.

12. Budget

The funding for the programme evaluation has already allocated in CY22 budget. The total budget for

whole process of evaluation is XXXXX US Dollars. Please refer to the embedded file in the appendix

for more detail budget breakdown for this evaluation.

13. Documents

The major documents that should be reviewed in the document review phase of the evaluation as

below:

1. AP design documents including sector Design documents

2. AP progress report, Semi and Annual report.

3. Sponsorship Monitoring Report

4. Sector Evaluation report

5. Evaluation report

14. Lessons Learned

The evaluation planning and process will be recorded and documented properly including training, filed

work and other key activities into minutes. The evaluation team will meet regularly to do a reflection

after completing each key activity through using development tool. The lesson learned will be

incorporated into the evaluation report and will be reflected by the M&E team to improve evaluation

practice for the next evaluation.

15. Appendices