REPORT ON RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION · RESULTS‐BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY ‐ 15 ‐ 19 AUGUST 2016 ELD TRAINING Page

RESULTS‐BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY ‐ 15 ‐ 19 AUGUST 2016 ELD TRAINING

www.eldtraining.com Page 1

REPORT ON TRAINING ON RESULTS‐BASED

MANAGEMENT, MONITORING AND EVALUATION

ISTANBUL, TURKEY

15 ‐ 19 AUGUST 2016

ELD TRAINING www.eldtraining.com



CONTENTS INTRODUCTION ....................................................................................................................................... 3 SUMMARY AND EVALUATION ................................................................................................................ 3 DAY ONE .................................................................................................................................................. 5 INTRODUCTION ................................................................................................................................... 5 PRINCIPLES OF RBM ............................................................................................................................ 7 THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION .................................................................. 8 THE LOGICAL FRAMEWORK APPROACH: ANALYSIS ............................................................................ 8 DAY‐END FEEDBACK ............................................................................................................................ 9

DAY TWO ............................................................................................................................................... 10 PROBLEM ANALYSIS ...................................................................................................................... 10 STAKEHOLDER ANALYSIS ............................................................................................................... 10 SETTING OBJECTIVES ..................................................................................................................... 10 SELECTING A STRATEGY ................................................................................................................ 10

DAY‐END FEEDBACK .......................................................................................................................... 11 DAY THREE ............................................................................................................................................ 11 THE LOGICAL FRAMEWORK APPROACH: PLANNING ........................................................................ 12 DAY‐END FEEDBACK .......................................................................................................................... 12

DAY FOUR .............................................................................................................................................. 13 PROJECT MONITORING AND EVALUATION ....................................................................................... 14 ESTABLISHING AN M&E SYSTEM ....................................................................................................... 14 DAY‐END FEEDBACK .......................................................................................................................... 15

DAY FIVE ................................................................................................................................................ 16 DATA ANALYSIS ................................................................................................................................. 17 RESULTS‐BASED REPORTING ............................................................................................................. 17 REVIEW, REFLECTION AND EVALUATION.......................................................................................... 18 PARTICIPANT LIST .............................................................................................................................. 18 TRAINING SCHEDULE ........................................................................................................................ 20 WORK PRODUCED BY PARTICIPANTS................................................................................................ 22 SYRIAN AND TURKISH YOUTH IN HATAY SOCIETY ........................................................................ 22 COMMUNITY‐BASED EDUCATION IN KANDAHAR, KABUL AND FARYAB ...................................... 35



INTRODUCTION This report documents the process and evaluates the outcomes of a five‐day training on Results‐Based Management, Monitoring and Evaluation held at Istanbul, Turkey, 15 ‐ 19 August 2016. The objectives of the training were that participants would:

Understand the philosophy and principles of RBM

Have a thorough knowledge and understanding of the Logical Framework Approach (LFA)

Be able to use the Logical Framework Approach in their own fields of work for the planning, implementation and monitoring of projects

Understand how to apply Results‐Based Management throughout the project cycle (identification – formulation – implementation – monitoring/evaluation and reporting)

Have developed a Monitoring and Evaluation plan for their selected projects The training was designed and delivered by ELD Director Neil Kendrick, and there were six participants in total, representing:

Save the Children Afghanistan (3 participants)

Save the Children Syria (1 participant)

The UN Office of the Resident Coordinator Turkey (2 participants) The training was held at the Double Tree by Hilton Hotel, Moda, Kadikoy, and ran from 09.00 to approximately 17.00 daily. This report is based on daily feedback from participants, trainer's observations and a final evaluation from participants.



SUMMARY AND EVALUATION The training was monitored through daily feedback from participants which is included at the end of each day's narrative in this report. A final evaluation was conducted on the last day which indicated a high level of satisfaction. Despite its ambitious range of objectives, the training was successful in delivering what was expected. All participants felt their expectations were very much met and indicated that the design and methodology of the course contributed to this. All participants said they had, to a great extent, learned what was needed and felt confident about applying the tools and approaches in their working areas. (Although a couple of participants did suggest they needed to review what was covered prior to implementing.) One of the highest rated questions was whether the training would have an impact on their projects, which scored a solid '9' out of a possible 10. Several participants suggested that one or more additional training days would have been useful for further practising LFA and M&E planning. The final evaluation results ‐ where participants rated the course out of a possible 10 on a range of criteria ‐ are below. Scores presented are the average score from participants.

EVALUATION QUESTION AVERAGE RATING / 10 (based on 6 participant responses)

OVERALL, WERE YOUR EXPECTATIONS AND OBJECTIVES MET? 8.8

WERE THE MATERIALS, METHODOLOGY AND TOPICS COVERED SUPPORTIVE IN YOUR LEARNING?

9.3

DID YOU LEARN WHAT YOU NEEDED? 8.5

DO YOU FEEL CONFIDENT / ABLE TO APPLY WHAT WAS LEARNED?

8.1

DO YOU THINK THIS TRAINING WILL HAVE A POSITIVE IMPACT ON FUTURE PROJECTS?

9



DAY ONE The first day established norms and expectations for the training, explored the philosophy and concepts of Results‐Based Management (RBM) and introduced the Logical Framework Approach (LFA). Participants also began the extended project work of the course, developing a Context Analysis and Problem Tree for their selected projects. INTRODUCTION The first session set the parameters for the training, allowing participants to introduce themselves and share their expectations. Participants also became familiar with the overall plan for the training. Due to the delayed of arrival of two participants coming from Ankara, it was agreed to delay the start of the first session to accommodate this, and the course began around 30 minutes late. During the first session participants introduced themselves and shared their expectations of the training (collected in the Mind Map below). These expectations would serve both to help adapt the course to participants' specific needs and interests, as well as to provide a baseline against which progress could be measured on the final day.



After discussing these expectations the training plan was shared.

After a discussion on logistics, the introduction session was concluded.



PRINCIPLES OF RBM The second session responded to the participants' expectation of learning the theory and principles of RBM. After a brief explanation of the history of RBM and the reason it was brought into mainstream development thinking, we walked through a series of steps to clarify our conceptual understanding the approach. As well as slides, participants shared their own experiences and understanding freely both with the whole group as well as in small brainstorming sessions. First we examined the concept of results, seeing the results are changes in a state or condition that derive from a cause‐and‐effect relationship and that these changes can be intended, unintended, positive or negative and can occur at output, outcome and impact level. The UNDG definition of RBM shared as a management strategy by which all actors ensure that

their Processes Products Services

Contribute to the achievement of desired results: Outputs Outcomes Higher level goals / impact

A key component of RBM is performance monitoring to objectively measure how well results are being achieved and report on measures taken to improve them

Next, participants shared their own ideas around the question 'Why RBM?' before looking at key features of the approach which included: Problem analysis to understand causes Stakeholder analysis Structuring of programmes around a chain of desired results Causality in the chain of results (if… then logic) Use of ‘change language’ (future conditional) Feedback loops to apply learning

Strategic selection of projects was also covered, considering the importance of projects selected reflecting value (contribution to a national or regional goal), support / partnership and capacity. Participants then discussed, in teams, both the benefits of applying RBM as well as the potential risks / limitations, before the trainer wrapped up the session with a summary of the key points.



THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION The first afternoon session examined the Logical Framework Approach (LFA) ‐ its history and purpose, the Log Frame (LF) itself, as well as overviewing the steps of the LFA process. After a discussion of the LFA's purpose and origins, we examined the Log Frame itself ‐ the different sections and columns, and what each was used for. This was illustrated with a case study from a camp for IDPs (Internally Displaced People), where participants built the LF from cards provided by the trainer. Participants learned the two types of logic ‐ the vertical logic of the Results Chain and the diagonal where each future hypothesis rests on assumptions at the level below. The session wrapped up with an overview of the 10 steps of the LFA process. THE LOGICAL FRAMEWORK APPROACH: ANALYSIS The rest of the day looked at the first two steps of the LFA ‐ Context Analysis and Problem Analysis. First, various tools were introduced for Context Analysis. These included:

Focus Group Discussion (including roundtables, forums and workshops) PESTEL SWOT PRA and PLA – Social Mapping, Resource Mapping, Transect Walks, Problem Tree,

Ranking Activities Desk Study Survey Observation Interview Gender Analysis

Particular attention was given to the Problem Tree as a tool for Problem Analysis. Participants then formed two teams to agree on a project to develop throughout the training. In one team, the three staff of Save the Children Afghanistan selected the issue of low completion rates in among 7‐15‐year‐olds in Community‐Based Education in Kandahar, Kabul and Faryab provinces. The second team ‐ comprising of two Ankara‐based UN staff and one Hatay‐based Save the Children Syria participant chose to address lack of active involvement of Syrian and Turkish youth in Hatay society. Teams then developed a Context Analysis for their selected issue (with both teams selecting PESTEL analysis) and started to develop their Problem Trees.



DAY‐END FEEDBACK At the end of each day participants noted down what they were happy about / worried about from the course so far. Feedback from Day One was overall, extremely positive.

Happy about ... Questions and Concerns ...

Clear ideas and presentation Proper / uncomplicated introduction to RBM Good materials and handouts Great venue Participants' and trainer's experience made the day fun Personal stories from trainer and participants made the training interesting Group exercises Learning on Results Chain Laid‐back facilitator, open to questions and adapting time schedule according to participants' interests and wishes Group work, tasks and exercises Good environment Good timing (breaks and lunch) Good trainer Good training Trainer involved all participants Useful practical work Collecting expectations Trainer communicated clearly Good sessions Met my expectations Lunch and other services all great

Not enough time to practice Problem Tree Still having difficulty understanding the problem tree Some slides were too fast Not applying RBM in detail yet



DAY TWO On the second day teams were fully involved in group work on their selected projects. Problem Trees were reviewed, finalised and tested, stakeholder analysis carried out and project objectives set. Some steps were also taken towards identifying a project strategy. PROBLEM ANALYSIS The Problem Tree had been introduced on the first day, and in the first session teams developed their final Problem Trees for their selected projects. They used Mind Manager (a Mind Mapping software) for this, and the results are annexed. Problems were brainstormed, a core problem identified and the causes and effects organised into a logical sequence. Problem Trees were then tested for validity. Considerable time was devoted to this step due to its fundamental role in the LFA process and, consequently, its being a foundation for any results‐based intervention. STAKEHOLDER ANALYSIS The day continued with an in‐depth look at Stakeholder Analysis. As well as the standard approach ‐ identifying various stakeholders' levels of interest in and influence on the project ‐ we also explored Stakeholder Communication and Stakeholder Engagement. Teams analysed the various stakeholders' roles in the project and developed a communication strategy for each stakeholder to ensure clear communication as well as to take account for any resistance. Stakeholder Management Plans are annexed. SETTING OBJECTIVES The day continued with objectives setting. Participants learned how, my rephrasing the negative statements from the Problem Tree as positives, and testing the new tree (Objectives Tree) as a means‐ends hypothesis. Those statements that could not be changed were set aside ‐ later to emerge as either Assumptions (things beyond the project's ability to directly control) or Context (the environment in which the project will operate). SELECTING A STRATEGY In the final session, participants attempted to find common areas in the 'roots' of the objectives tree that represented various possible approaches to addressing the original core problem / achieving the project objective. The various approaches were then compared using a ranking system to assess the viability of each in order to combine those approaches into a comprehensive and prioritised project strategy. (Due to the absence of two participants in the final session this step was reviewed and repeated as a team exercise in the first session of Day Three.)



DAY‐END FEEDBACK


Everything went well Room fine Good facilities Enough practice Learned (and enjoyed) a lot Learning by doing Going according to my expectation Clear guidance for group work Practical exercises were useful Good facilities Great to spend almost the whole day on group work! Practical day Good to practice Problem Tree again Mind Manager is a useful tool Good guidance and cooperation

Unclear about how to select strategy Problem Tree needs time and practice! How does the Problem Tree relate to RBM?



DAY THREE Day Three again saw a lot of practical, hands‐on work. Teams finalised their strategy selection for their projects and built their Log Frames, completing the Narrative Summary before identifying assumptions and testing the project logic. Indicators were also identified for each level of result. The third day's first session was used to review and finalise the group work of the previous day. Teams discussed the various approaches and combined these into a strategy to achieve the project objective. THE LOGICAL FRAMEWORK APPROACH: PLANNING After sharing the results so far in the plenary, teams moved onto the planning steps of the Logical Framework Approach. Groups worked step‐by‐step through the day, turning their analysis into a clear plan. Using a simple Log Frame template they set outcome and goal, followed by outputs, activities and inputs. Teams then examined the project context, at precondition level as well as at output and outcome levels, before testing the Log Frame hypothesis. Following discussion and inputs on designing indicators, teams addressed the M&E section of the Log Frame, defining indicators and means of verification at each level. By the end of the day participants' Log Frames were near complete.



DAY‐END FEEDBACK


Very good practice Enough time for group work Clear guidance and instruction for practical work Good environment Good facilities Great practical work Good understanding of all topics Training is going well Great, interactive training methods

Not enough time to fully develop all indicators Tiring day by the end!



DAY FOUR Day Four saw participants explore the M&E aspect of the training in more depth. The morning was spent discussing various concepts of Monitoring and Evaluation, and in the afternoon teams developed detailed M&E frameworks for their selected projects. PROJECT MONITORING AND EVALUATION The day began with a plenary discussion, sharing and presentation on Monitoring and Evaluation. Both terms were defined, and the differences between Monitoring / Evaluation clarified. After discussing what we monitor and why, we examined the differences between traditional and results‐based monitoring ‐ and how the purpose in results‐based monitoring is to inform learning / decision making rather than to track, for the ultimate purpose of measuring change / progress towards results. The session concluded by examining a key component of M&E planning ‐ defining Key Evaluation Questions. Evaluation questions define the key issues to be explored by an evaluation. They are developed and prioritised by program staff, evaluation personnel, funders, and other stakeholders. ESTABLISHING AN M&E SYSTEM The next session explored the steps of establishing an M&E system / framework. After defining what an M&E system is and its purpose, steps discussed were: 1. Defining the results 2. Establishing indicators 3. Determining how data will be collected, from where this data will come 4. Articulating risks and assumptions 5. Grouping indicators 6. Designing data collection tools 7. Data collection and management 8. Analysing the data and examine the trends and meaning behind the data 9. Interpreting, compiling and presenting the analysed data 10. Using the data to inform learning Results and indicators had already been developed as part of the LFA process; and many of the methods had already been covered by the Log Frames' MOV and during the first day's sessions on Context Analysis. However, we also explored two additional qualitative methods for data collection ‐ Case Study and MSC (Most Significant Change) ‐ and discussed methodological consideration when selecting / combining data collection methods. The importance of clarifying frequency and responsibility ‐ responsibility for collection, verification, analysis and reporting ‐ were also covered. On step 4, we spent time examining how to identify, measure and evaluate risk. Participants were introduced to the concept of logging risk through a Risk Register, and the importance of including periodic risk monitoring as a component of their M&E planning. Group indicators were discussed as a way to ensure more efficient collection of data. This means identifying indicators which can be verified using the same method or tool, ideally from the same source and at the same time / with the same frequency.



The importance of having clear tools ‐ named and explained ‐ was also covered, to ensure that data is collected consistently by all staff. Throughout the afternoon teams developed their own M&E plans, which are annexed. The process forced them to critically examine each of their indicators and identify efficient, economical and verifiable ways to gather the information. As their plans developed a very comprehensive framework emerged, which, if applied to their selected projects, will lead to better monitoring and, consequently, a greater likelihood of achieving results. The Hatay team presented their M&E plan at the end of the day for sharing / learning and feedback, while the Afghan team presented back the next day.



DAY‐END FEEDBACK


Everything was clear today Great practical work Session on M&E planning will be good for our projects Now I know how to plan for Monitoring and Evaluation Risk Management was interesting Most Significant Change technique looks useful

N/A



DAY FIVE On the final day we reviewed the M&E plan from the Afghan team, and looked at the remaining steps of M&E planning: Data Analysis and Reporting. DATA ANALYSIS Two exercises were conducted to practice Data Analysis. In the first exercise, participants examined and discussed data gathered from a countrywide survey of peoples' opinions in Nepal at the height of that country's civil war. The exercise was interesting in that there was often more than on interpretation for each graph; and through study and discussion it was possible to draw out underlying issues not explicitly obvious from the question. A useful learning from the task was that data is never neutral ‐ everything has some significance. Also, it is the reporter's job to extract meaning from the data and guide readers / decision‐makers to the main message the data offers. The second exercise was more qualitative and gave participants further practice in using Mind Mapping. From a list of 'facts and figures' on the status of children, teams first categorised the information and then compiled into a comprehensive map. From the map they were able to find underlying issues of cause and effect ‐ such as early marriage and its connections to girl education, labour and maternal mortality ‐ that the data, in isolation, are not immediately apparent. A key learning was that, through a three‐step process, the meaning / conclusions of the data can become apparent in an organic way. RESULTS‐BASED REPORTING The next session looked at reporting. We discussed the responsibilities of the reporter and the purpose of reporting, before examining specific questions to be considered when setting report objectives. We also examined the concept of the Four levels of Evaluation ‐ Reactions and Feeling, Learning and Knowledge, Change in Behaviour and Impact. We saw how, over time, we can move 'up' the levels, and with each periodic collection of data we can gather information on more levels as we move towards evaluation. We looked at the different types of reports produced during an intervention ‐ progress reports, periodic reports and evaluations. Each was presented as a Mind Map. We also noted how these maps, due to their flexible nature, can be designed to also include Stakeholder Monitoring and Risk Monitoring. The session wrapped up with a discussion of key issues to consider during the reporting process.



REVIEW, REFLECTION AND EVALUATION The final part of the day was spent reflecting on what had been covered so far. We reviewed what RBM is and its common features. Next, participants brainstormed the strengths and limitations of RBM, and also challenges they might face operationalising the approach in their organisations / projects. Next, participants completed personal action plans detailing what steps they would take to apply what was learned when returning to their respective stations. After a brief review of the key points of the course, and a look back at the first day's expectations, participants completed a short exit survey (see Summary and Evaluation) before certificates were presented.



PARTICIPANT LIST

Rahmatullah Rahmani Save the Children International Afghanistan

Mahram Ali Nadir Save the Children International Afghanistan

Ajmal Shirzad Save the Children International Afghanistan

Nil Memisoglu Mit UN Office of the Resident Coordinator, Turkey

Tina Stoum UN Office of the Resident Coordinator, Turkey

Maher Al Kasem Save the Children Syria



TRAINING SCHEDULE

1. INTRODUCTION

OBJECTIVES EXPECTATIONS TRAINING PLAN LOGISTICS

2. PRINCIPLES OF RBM

WHAT ARE RESULTS? WHAT IS RBM? WHY RBM? COMMON FEATURES OF RBM RBM AND THE PROJECT CYCLE STRATEGIC SELECTION OF PROJECTS APPLYING RBM

3. THE LOGICAL FRAMEWORK APPROACH: INTRODUCTION

WHAT IS THE LFA? WHAT IS A LOGFRAME? VERIFYING THE PROJECT DESIGN OVERVIEW OF THE LFA STEPS

4. THE LOGICAL FRAMEWORK APPROACH: ANALYSIS

CONTEXT ANALYSIS PROBLEM ANALYSIS STAKEHOLDER ANALYSIS SETTING OBJECTIVES SELECTING A STRATEGY

5. THE LOGICAL FRAMEWORK APPROACH: PLANNING

OUTCOME AND GOAL OUTPUTS ACTIVITIES INPUTS ASSUMPTIONS / PROJECT CONTEXT INDICATORS



6. PROJECT MONITORING AND EVALUATION

MONITORING EVALUATION DIFFERENCES MONITORING: WHAT AND WHY? TRADITIONAL VS. RESULTS‐BASED M&E M&E PLANNING

7. ESTABLISHING AN M&E SYSTEM

DEFINING THE RESULTS ESTABLISHING INDICATORS DATA SOURCES AND METHODS RISKS AND ASSUMPTIONS GROUPING INDICATORS DESIGNING DATA COLLECTION TOOLS

8. DATA ANALYSIS

SURVEY ANALYSIS MIND MAPPING ANALYSIS

9. RESULTS‐BASED REPORTING

PURPOSE OF REPORTING RESPONSIBILITIES OF THE REPORTER SETTING REPORTING OBJECTIVES FOUR LEVELS OF EVALUATION PROGRESS REPORTS PERIODIC REPORTS EVALUATIONS

10. REVIEW, REFLECTION AND EVALUATION

WHAT IS RBM? COMMON FEATURES OF RBM WEAKNESSES, LIMITATIONS AND CHALLENGES OF RBM OPERATIONALISING RBM PERSONAL ACTION PLANNING OPEN SPACE DISCUSSION COURSE EVALUATION CERTIFICATION



WORK PRODUCED BY PARTICIPANTS

SYRIAN AND TURKISH YOUTH IN HATAY SOCIETY

PESTEL ANALYSIS



PROBLEM TREE



MONITORING & EVALUATION PLAN

Introduction

Purpose of this plan

This plan is designed to facilitate monitoring the living conditions of Syrian and Turkish Youth in

Hatay and will focus on integration of marginalized female and male Syrian and Turkish youth living

in Hatay city.

Increase number of Syrian and Turkish female and male youth in the Hatay society reduced tensions

in Hatay especially on youth. Also as a result of not having the documentation of the Syrian

population in Hatay caused difficulty in not responding to the needs of the Turkish society and skills

of the Syrian Youth. This increased the unregistered working culture within the society and also

created an unfair competition among the Turkish and Syrian youth in job hunting.

Access to education and working labour will create a harmony and will increase the satisfaction level

of youth in the society therefore this assessment will focus on the educational and working needs of

the Syrian and Turkish youth both male and female, including a special emphasis on the needs of

women and girls since their population are higher. The objective of this needs assessment is to

answer the following questions:

1. How many youths employed by private sector?

2) How many Syrians enrolled in the Turkish educational system?

3) What is the percentage of youth who reported their involvement in social events and

organizations?

4) What educational opportunities are available for Syrian and Turkish youth?

5) What are barriers to school attendance? How do these reasons/barriers vary by gender and age?

To answer the Key Evaluation Questions, semi‐structured interviews, survey, focus groups, Focus

Group Discussion were conducted with government authorities, Turkish and international

organization staff, and researchers working with Syrians in Turkey, Private Sector and families of the

beneficiaries. In addition, the city of Hatay was used as a case study to better understand social,

educational and working needs for marginalized youth living in this area near the Syrian border.



Project Summary

Title Happy Youth Happy Societies

Starting Date January 2017

Duration 2 yrs.

Partners Government Authorities, Turkish and international organization staff, and

researchers working with Syrians in Turkey, Private Sector and families of

the beneficiaries

Target Area Hatay City

Beneficiaries Male and female Syrian and Turkish youth aged 15‐25

Cost ?

Funding Source International Donors (?) Turkish Government (?) Private Sector (?)

Goal Reduced likelihood of radicalisation among Hatay populations



Logical Framework

NARRATIVE SUMMARY

INDICATORS MEANS OF VERIFICATION

RISKS / ASSUMPTIONS

Goal Reduced likelihood of radicalisation among Hatay populations

Outcomes By 2018 increased number of female and male Syrian and Turkish youth involved actively in Hatay society

1.1 % of youth (18‐25) employed within 6 months of completion of training/courses, disaggregated by ethnicity, age and gender B: 0% T: 35%

1.2 # of Syrians (15‐25 from target group) enrolled in the Turkish educational system, disaggregated by gender and age B:0 T: 50%

1.3 % of the target group (15‐25) who reports to be involved in social organizations, disaggregated by gender and ageB:0 T: 40%

Post‐training survey

1.2.1 Survey

1.2.2 Case Study

1.3.1 Sample Survey

1.3.2 Interviews

1.3.3 Focus Group Discussion

External events

will not increase

tensions between

Syrian and Turkish

populations



Outputs 1. Improved

opportunities and life skills for marginalized Turkish and Syrian male and female youth

2. Reducing Bias

between Syrian and Turkish Youth

3. Documentation Procedures for Syrians in place

1.1 % of target group who completed vocation training and who scored “good” and above in the final test. B: 0% T: 35%

1.2 % of Syrian and Turkish (15‐25) youth who report improvement in their life skills 6 weeks after completion of training B: ? T: 60%

% of Syrian/Turkish youth (15‐25) who report positive relations/positive image of each other

3.1 # of Syrians (15‐25) who have documentation needed disaggregated by sex and age

1.1.1 Test records

1.2.1 Post‐training Survey

1.2.2 FDG after

Private sectors

show interest in

hiring the

marginalized

youth

Target group use

the service and

documentation

centre and local

authorities

continues to

support the

centre.



Activities 1.1 Conduct needs assessment among the Turkish and Syrian Youth in Hatay 1.2 Establish 5 youth clubs 1.3 Organize 30 different Youth Activities over 2 years 1.4 Arrange market‐based vocational trainings for 100 young men and 100 women 1.5 Conduct Turkish language classes for 300 young men and women 1.6 Gender awareness and gender equality sessions for young men and women 2.1 Establish a cross‐community youth‐led council 2.2 Regular cross‐cultural social events and community initiatives 3.1 Establish IT system 3.2 Create registration office 3.3 Organize awareness raising activities on

the document



Indicators

Indicator Syrian youths enrolled in the Turkish educational system

Definition # of Syrians (15‐25) enrolled in the Turkish educational system,

disaggregated by gender and age

Purpose To assess how many youths can enter the educational Turkish system

(school and university). The number would provide evidence on whether

the documentation service and the catch up classes help those students to

integrate within the Turkish system.

Baseline Average number: 0%

Target Average target: 50%

Data Collection The M&E team will conduct surveys with those who accessed the language

catch up classes and the documentation service one year following the

services listed to check what percentage were able to access the Turkish

education system by way of accessing the services. To capture quality, the

M&E team will be conducting some case studies of those who access the

service to document their experience.

Tool Survey with the beneficiaries who accessed the services above

Case study to capture and document the quality of the service

Frequency Every 6 months following the end of the services listed

Responsible M&E team and programme manager

Reporting The percentage of those who access the Turkish educational system

following the services listed above will be reported by each enumerator to

M&E officers. The Program Manager will then combine the data from each

officer to create full list of trainees and their educational status. This will be

used to calculate the average for all beneficiaries. The average score will be

included in the report for the donor submitted every six months.

Quality Control All enumerators and officers will attend a one day training course on how

to complete the post‐access survey. Random samples of the beneficiaries

will be chosen to verify the data provided and their current status.



Indicator Youths participating in social organizations

Definition % of the target group (15‐25) who reports to be involved in social

organizations, disaggregated by gender and age

Purpose To assess the percentage of the target group who participate in social

organizations or events to know how much they are applying what they

have learnt from the project. This would show if the activities carried out in

the project contribute to helping the target group integrate in society.



Data Collection The M&E team will conduct surveys with those who accessed the language

catch up classes and the documentation service one year following the

services listed to check what percentage were able to access the Turkish

education system by way of accessing the services. To capture quality, the

M&E team will be conducting some case studies of those who access the

service to document their experience.

Tool Survey with the beneficiaries who accessed the services above

FGD to assess challenges and deep feedback

Interviews

Frequency Every 3 months following the package of trainings provided

Responsible M&E team and programme manager

Reporting The percentage of those who participate in social organizations and events

will be measured by doing the above mentioned tools and will be reported

by the M&E team to the project manager who will report them to the

donor each 6 months.

Quality Control Field staff and enumerators will be trained on conducting the tools listed

above and how to report them. The qualitative tools will be used to capture

some learning and infuse it into the project



Indicator Youths employed by the private sector

Definition % of youth (18‐25) employed within 6 months of completion of

training/courses, disaggregated by ethnicity, age and gender

Purpose To assess whether youths attending trainings and courses of the project are

being employed or not after all. The number would provide evidence on

whether the training project is effective and based in the market needs or not.



Data Collection The M&E team will conduct a post‐training survey each 3 months following the

end of the first package of courses over the first year of the project to check

how many have been employed following these life skills and vocational

trainings. Over one year following the end of the first training packages, data

will be recorded and analysed again the target indicator.

Tool Post‐training survey with beneficiaries attending the packages of trainings

Frequency Every 3 months following the end of the first package of trainings

Responsible Field enumerators

Reporting The percentage of those who are employed after attending the packages of

trainings will be reported by each enumerator to M&E officers. The Program

Manager will then combine the data from each officer to create full list of

trainees and their employment status. This will be used to calculate the average

for all trainees. The average score will be included in the report for the donor

submitted every six months.

Quality Control All enumerators and officers will attend a one day training course on how to

complete the post‐training survey. Random samples of the trainees will be

chosen to verify the data provided and their current status.



Indicator Youths completing vocational training

Definition % of target group who completed vocational training and who scored

“good” and above in the final test.

Purpose To assess the percentage of target group who complete vocational

training and who score good in the end of this training. The percentage

would explain to what level the trainees have managed to score well in

the end of this training.



Data Collection The trainers will conduct an assessment test for all trainees in the class.

Each trainee will be assessed individually. The trainer gives them some

tasks based on the activities they have had and will provide a score based

on certain criteria. Then the score of each student in the different tasks

will be grouped to give us the total score.

Tool Post‐training assessment with beneficiaries attending the vocational

training

Frequency Following each training for each group

Responsible Field enumerators

Reporting The average of the scores achieved by each trainee will be recorded and

grouped by each trainer following the assessments. The score will be

then reported to the programme manager who will in turn report this

each six months for the donors.

Quality Control All trainers will attend a one‐week ToT to be able to deliver high quality

trainings and to be able to conduct the assessments properly.



Roles & Responsibilities

Role Responsibilities

Field Staff Collecting data (surveys, interviews, FGD)

Cross‐cultural council

Collecting data (what could they assist in?)

Team Leader Checking data, drafting reports

M&E Officer Conducting analysis and quality assurance of tools used

Programme Manager

Reviewing reports, making decisions based on the data

Steering Committee or similar

Reviewing reports, giving feedback



Data Flow



COMMUNITY‐BASED EDUCATION IN KANDAHAR, KABUL AND FARYAB

PROBLEM TREE



OBJECTIVES TREE



MONITORING & EVALUATION PLAN

Introduction

Purpose of this Plan

This plan is prepared by Monitoring and Evaluation Unit for the purpose of on time data collection

with high quality for the project of STAGES (Steps Towards Afghan Girls Education Challenges).

Project Summary

Title Steps Towards Afghan Girls Education Success (STAGES)

Starting Date June 27, 2013 – April 30, 2017

Duration 4 years.

Partners N/A

Target Area Kandahar, Faryab and Kabul.

Beneficiaries Children

Cost £1,698,085.82plus

£ 117,662.94 UK matching funds

Funding Source DFID Funded with a match funding liability

Goal More children complete the primary education in KKF



Logical Framework

NARRATIVE SUMMARY

INDICATORS MoV RISKS / ASSUMPTIONS

Goal More children

complete the

primary education

in KKF

The percentage

of children

completing the

primary

education

increases from

60% to 85% in

KKF by 2020.

Attendance sheet

Survey

Outcomes The attendance

rate among 7‐15‐

year‐olds is

increased in

Kandahar, Kabul

and Faryab

provinces

The attendance

rate is increase

from 75%‐90% of

the enrolled

children.

Attendance sheet Children are not

pulled out for early

marriage or

domestic labour.



Outputs 1. Positive

Changes in families' attitudes towards children's continued education

2. Increased availability of education.

3. Improved Quality of Education

1. 200 parents understand the value of education.

2. 20 CB classes have the adequate facilities.

2.2 20 CB classes received class kit

2.3 20 teacher receive full teachers kits

2.4 400 students receive student kits

3. At least 75% of CB teachers trained meet minimum standards from month 6, rising to 85% by end of Year 1

1.1 FDG 1.2 Interview

2.1 Observation

2.2 Class survey

2.2.1: Distribution

List

2.2.2:

Observation

2.3.1: Distribution

List

2.3.2:

Observation

2.4.1: Distribution

List

2.4.2:

Observation

3.1: Training

Report

3.2: teacher

observation.

Parents can afford

to send the

children to

community based

classes.

Trained teachers

continue to work

in community

based education.



Activities 1.1 Fact‐finding meetings,

interviews, and FDGs with 20 Community councils, 60 teachers, and 200 parents.

1.2 Conducting 12 x quarterly training and other community based social awareness activities in each of the 20 communities

2.1 Providing education materials for 20 CB classes

2.2 Renovation of 20 community based classes.

3.1 Monthly teacher training modules conducted in 20 districts

3.2 Regular / ongoing Monitoring of 20 CB classes

1 Human resource 2 Financial

resource 3 Time

Security situation

doesn’t

deteriorate after

parliamentary

elections.

District governors

approve.



Indicators

Indicator Reading proficiency among children in Grade 6

Definition Sum of all reading proficiency test scores for all students in Grade 6 divided

by the total number of students in Grade 6.

Purpose To assess whether reading proficiency at the schools participating in the

program is improving over time. This would provide evidence on whether

the reading component of the program is effective.

Baseline Average score: 47

Target Average score: 57

Data Collection The class teacher will conduct a reading proficiency test for all students in

the class. Each student will be assessed individually in a separate room. The

teacher will ask them to read a list of words, sentences and paragraphs out

loud and will mark each one that they have difficulty with. Any students not

present on the day of the assessment will be excluded.

Tool National Reading Proficiency Assessment questionnaire (See Annex A)

Frequency Every 6 months

Responsible Teachers

Reporting The individual score for each student will be reported in the six monthly

progress reports submitted by each teacher to the Program Manager. The

Program Manager will then combine the data from each class to create full

list of students and their scores. This will be used to calculate the average

score for all students in Grade 6 using the definition above. The average

score will be included in the report for the donor submitted every six

months.

Quality Control All teachers will attend a one day training course on how to complete the

assessment. To verify the accuracy of the test scores submitted by the

teachers the Program Manager will randomly select one class every six

months to audit. This audit will involve re‐testing all the students in the

class and comparing the results to the results submitted by the teacher.



Indicator The percentage of children completing the primary education increases

from 60% to 85% in KKF by 2020.

Definition The percentage of students should be increased from 60% to 85% by end of

2020 in all three provinces, Kandahar, Kabul and Faryab.

Purpose TO have more children complete the primary education in Kandahar, Kabul

and Faryab

Baseline 60%

Target 85%

Data Collection Survey, Attendance sheets

Tool Students attendance sheet and School survey form

Frequency Once in a quarter

Responsible M&E Unit

Reporting The students’ sore report will be reported to donor by end of each quarter

in the narrative quarterly report, done by M&E unit.

Quality Control The M&E team will be highly oriented by collection the data from schools

and will calculate the average of that by end of the quarter.



Indicator The attendance rate is increase from 75%‐90% of the enrolled children.

Definition All the students’ attendance in the class has to be increased from 75% to

90%.

Purpose The attendance rate among 7‐15‐year‐olds is increased in Kandahar, Kabul

and Faryab provinces

Baseline 75%

Target 90%

Data Collection Statistical reports of students attendance sheet

Tool Students Monthly Attendance sheet

Frequency Every Month

Responsible M&E Unit

Reporting The students’ sore report will be reported to donor by end of each quarter

in the narrative quarterly report, done by M&E unit.

Quality Control The M&E team will be highly oriented by collection the data from schools

and will calculate the average of that by end of the quarter.



Indicator 200 parents understand the value of education.

Definition For understanding the parents there will be training and social awareness

events and in order the parents will know about the value of education.

They will thoroughly understand what education is and what the

advantages of education are.

Purpose The parents should understand the advantage to education and let their

children to continue education.

Baseline

Target 200 parents

Data Collection 1.3 FDG 1.4 Interview

Tool FDG form, Parents interview forms

Frequency Every three months

Responsible Project Officers with support of M&E unit

Reporting Once the project officer conducts the interview and FGDs then the findings

will be shared with M&E unit for analysing and verification of data. After

that the M&E Unit will use the pre‐made forms and will report in the

narrative report.

Quality Control M&E team will conduct training for project officer and will make them

understood about the method of conducting interview, FDGs and the

needed data collection.



Indicator 20 CB classes have the adequate facilities.

Definition Adequate facility: A class should have all the following criteria to meet the

adequate facility:

a. There is enough space for all children in the class

b. There is hand washing facilities near to class

c. There is a teacher for the maximum 30 students

d. There is heater and cooling system available in the class

e. The roof is waterproof

Purpose The class should meet the above criteria and have the adequate facilities.

Baseline

Target 20 classes

Data Collection 2.1 Observation

Tool Class survey, and Class observation form

Frequency Once in a quarter

Responsible M&E team with support of field team

Reporting Once the M&E team with support of field team conduct observation in a

class, they will use the class observation form and then the M&E team will

analyse the data and will enter into specific database. After the analysing

the M&E unit will report in Quarterly narrative report “section 2.1,

adequate facilities”

Quality Control M&E team will conduct training for field team and will make them

understood about the method of conducting observation and accurately

filling the class observation form.



Indicator 3.2 20 CB classes received class kit

Definition Class kit include (one chair, 3*4 meter carpet…) and will distribute at the

start of each academic year for each CB classes

Purpose All classes has to receive the kits for improving the quality of teaching

Baseline 0 %

Target 100%

Data Collection Distribution list & Distribution report

Tool Minimum standard checklist

Frequency After each distribution (one academic year)

Responsible Project team

Reporting At the end of each quarter the report will be drafted by M&E officer from

his/her relevant province and will be shared with M&E manager.

Quality Control After distribution of class kits for CB classes, M&E team will observe from

each classes to insure about distributed kits



Indicator 3.3 20 teacher receive full teachers kits

Definition Teacher kit include (One bag, pen, pencil, ruler…) and will distribute at the

start of each academic year for each CB teacher

Purpose All teachers has to receive the kits for improving the quality of teaching

Baseline 0 %

Target 100%







Quality Control After distribution of teacher kits for CB teachers, M&E team will observe

from each classes/teacher to insure about distributed kits



Indicator 3.4 400 students receive student kits

Definition Students kit includes (One bag, pen, pencil, ruler…) and will distribute at the

start of each academic year for each CB students

Purpose Student kits distributed for each students to improve the quality of

education.

Baseline 0 %

Target 100%







Quality Control After distribution of student kits for CB teachers, M&E team will observe

from each classes to insure about distributed kits



Indicator At least 75% of CB teachers trained meet minimum standards from month 6, rising to 85% by end of Year 1

Definition The trained teachers have to meet the Minimum standard (tools for

checking the quality of education accepted by MoE)

Purpose At least 85% of teachers have to meet the minimum standards after

receiving training

Baseline 75%

Target 85%

Data Collection Observation, interview

Tool Lesson observation & Teacher mentoring.

Frequency After each training


Reporting For each training the report will be prepared by project team will be shared

with M&E manager

Quality Control From each training will be monitored to ensure regarding quality of training

and also after conducting the training the M&E team will interview with

teachers



Roles & Responsibilities

Project Manager Project Management, Project Implementation, review of reports and sharing with Donor

Project Coordinator

Project Implementation, Project team leading

MEAL Manager Project Monitoring, Evaluation and Reporting

M&E Officer Project Monitoring and Evaluation, Reporting, Data collection, verification, and analysing.

Project officer Project implementation on field level

M&E assistant Data collection and Data entering

Field assistant Community mobilization and data collection

Trainer Teachers, parents and other parties’ trainings



Data Flow

Documents

REPORT ON RESULTS-BASED MANAGEMENT, MONITORING AND EVALUATION · RESULTS‐BASED MANAGEMENT, MONITORING AND EVALUATION ISTANBUL, TURKEY ‐ 15 ‐ 19 AUGUST 2016 ELD TRAINING Page