Results based M&E: Experiences of school level education in Nepal

Teertha DhakalJoint Secretary

Government of Nepal

Presentation outline

• Context- National M&E system (PMAS)

• Educational planning

• Results-based M&E in education

• M&E design of School Sector Reform Plan (SSRP)

• Lessons learnt and Issues

Nepal

Area (Sq. Km)Area (Sq. Km) 147,181.147,181.

Population (million)Population (million) 28.528.5

Growth rate (%)Growth rate (%) 2.132.13

GDP (CP) (billion)GDP (CP) (billion) US$ 170US$ 170

Growth rate (%) Growth rate (%) 3.53.5

Per Capita GDPPer Capita GDP US$ 562US$ 562

Poverty (%)Poverty (%) 25.425.4

Literacy Literacy ((%)%) 6363(Economic survey, 2009/10)(Economic survey, 2009/10)

Context

• Despite difficult political situation Nepal has been institutionalizing MfDR in planning, budgeting and M&E with– Broad-based interventions – Piloting in key agencies

• The country is in the process of – restructuring the state building new constitution,

and – expedite development efforts focusing on results

• This presentation includes efforts to institutionalize results based M&E in education sector in Nepal

MfDR Practices in NepalLevel Planning Budgeting Monitoring

National MTP/ TYIPLog frame,Result matrixes

MTEF PMASPRSP progress reportsNDAC

Sectoral SWApBusiness plan as a sectoral strategy

MTEF sectoral papersRBB

Performance indicatorsMDAC

Project Log frames mandatory

Performance based budgetRBB

Performance M&E

Local DPPs Log frame based

D-MTEFFormula based grants

DPMASSocial audits

MfDR- piloting in some agencies

• Piloted in some ministries under ADB TA- Education, Local Development, Roads, Drinking Water)

– Readiness assessment done and Business Plan refined– Results matrix prepared– Internal capacities enhanced– Piloted in 4 districts- awareness and capacity enhanced

• ADB TA 7158- NEP: Strengthening capacity for MfDR under implementation (RBB, capacity building)

• Other initiatives e.g. performance contracts

ContextContext

• Education budget as share of public budget: 17%

• Primary edu budget as share of edu budget- 60%; as share of GDP- 2.22%

• Primary education free by law

• Primary NER: 89.1% (GPI: 0.98)

• Home to school distance (half an hour walk): 91% hhs

• High repetition & dropout rates– 29% repetition; 15% drop out in grade1

• Survival rate in Grade 5- 81.1 %

ReformsReforms

• Efforts made to improve educational governance

• Community mgmt of schools

• Increased public investment in education

• Per capita funding to schools

• Moving towards universal completion of basic education (grade 1-8)

• Capacity devt at different levels

• Shift from project approach to SWAp

• Focus on results7

Primary Education in Nepal

Level Plans Responsible Edu. plan Responsible

National MTPP(PRSP)

NPC Edu plan

SSRP

MOE

District Dist. Devt. Plan

DDCs Dist Edu. Plan

DEOs

Village/ Municipality

V/M Devt. Plan

VDCs, Muni. V/M Edu Plan

V/M Edu. Com.

Institutional SIPs SMCs

Education Planning

Poverty Monitoring and Analysis System (PMAS)

(A national level Results-based M&E system)

Why Poverty Monitoring and Analysis System (PMAS)?

• PMAS aims to-– support the implementation of the national plan– coordinate monitoring the national plan and

tracking the MDGs – communicate results as feedback into policy

making process– make central M&E system results-based

PMAS- Major Components

Outcome Monitoring

Implementation Monitoring

Impact AnalysisImpacts

Processes

Outputs

Outcomes

Inputs

Communication/Advocacy

PMIS

Components of the PMAS

Performance-based budget release

PRSP progress report

Education indicators in PMAS

Outcome Intermediate/outcome Indicators

Access •Number of primary schools•NER (primary)•Proportion of primary students receiving scholarship •Proportion of population more than half an hour's walk to school

Quality •Pupil completing primary level (%)•Primary/secondary teachers trained (%)•Proportion of vacant teacher's positions•No. of schools transferred to communities•Absenteeism of teachers

Equity •Share of female teachers in primary schools (%)•Girls students receiving scholarships (primary/secondary)•Schools having separate latrines for girls•Enrolment/ cycle completion etc (social groups/ gender)

Literacy •Adult literacy rate (Male/ female)

Sources of information for M&E

PRSP Sectoral Targets

MTEF/Budget allocation

Release/Disbursement of allocated budget

Outputs

OutcomesImpact

EMIS, Flash reports, PETS, Social audit, client

satisfaction survey

HH Surveys (NLSS, NDHS), Impact Analysis

Results based education M&E

• Logic models in educational planning, programming and design of projects

• Business plan and MTEF in the education sector

• Linkages between national and sectoral monitoring

• Education M&E in National Poverty Monitoring & Analysis System (PMAS)

• SWAp in primary level has contributed to harmonize M&E system

• Streamlined and strengthened survey systems- Separate education modules in;

– Nepal Living Standards Survey (5 yrs)– Nepal Demographic and Health Survey (5 yrs)– Nepal Labour Force Survey (5 yrs)

• Periodic reviews inviting multiple actors/ stakeholders

• Formative research institutionalized in school education

• Learning achievement tests introduced

• Focused on capacity building

Results based education M&E …

EMIS and Flash Report System

• The EMIS gathers school level educational performance data from all schools twice a year– Flash I- snapshot of data at the beginning of the school year – Flash II- focuses on end of school year data

• The national flash report is prepared based on district flash reports

• The reports cover enrolment, pass rates, repetition & survival rates, training status of teachers, supply of textbooks and learning materials

• Te reports also show participation of SMCs/ PTAs in various activities of schools

M&E of School Sector Reform Plan (SSRP)

M&E in SSRP

• The SSRP is the school sector SWAp in Nepal which is funded by more than 10 devt partners in addition to government funding

• The M&E aims to improve education services through informed decisions tracking achievements at each level of service delivery

• M&E functions based on performance indicators include;– Assessing compliance with acts and regulations– Measuring progress against milestones and targets– Evaluating the impact of policies/strategies

• Utilizes a decentralized monitoring system, use flash reports data, social audit, and provisions an external evaluation of outcomes

• Education Review Office conducts external audits of agencies based on norms and standards and submits annual reports to the Education Policy Committee

Key education indicators in SWAp1 Share of education budget in GNP, GDP

2 Share in education budget in primary, secondary

3 Teachers with required qualification and training

4 Teachers with required certification

5 Pupil/ teacher ratio

6 Enrolment in Grade 1 (ECD experience, GIR, NIR)

7 GER (Primary, Basic & Secondary)

8 NER (Primary, Basic & Secondary)

9 Repetition rate (Grade 1 & 8)

10 Survival rate by cohort method (Grade 5 & 8)

11 Coefficient of efficiency (Basic & Secondary)

12 Learning achievements (Average score of students in core subjects in Grade 5 & 8)

13 Pass rate (Percentage of students pass in SLC)

14 Literacy rate (15-24, 6+ & 15+)

15 Literacy GPI (15+)

Summary of key results of M&E activities

Key results Log-frame level

Frequency Data source Responsible

Client satisfaction baseline survey

Baseline Once (2009) Service recipient MOE, ERO

Combined status report

Key results Annual Schools, DEOs DOE

District level status report

Key results Annual Schools DEOs

Flash I Key results/ obj Annual Schools DOE

Flash II Key results/ obj Annual Schools DOE

Agency Review Reports

Key results, agency performance

Annual Concerned agency

ERO

SSR progress review reports

Strategic interventions, Key results/inputs

Annual Schools, DEOs Agency

ERO

Summary of school-level fin audit reports

Strategic interventions, Key results/inputs

Annual Auditors’ school audit report

DEO/DOE

Summary of key results of M&E activities

Key results Log-frame level

Frequency Data source Responsible

Social audit summary report

All Annual Schools/ DEOs DEO/ERO

Client satisfaction survey

Key results/ inputs

Annual Service recipient DEOs

Trimester progress report

Key results/ obj Trimester DEOs record DEOs

FMR Key results/Inputs Trimester DEOs DOE/ DEOs

Commissioned research studies

Key results/ obj As required As per the TOR MOE/DOE

National studies on learning assessment

Objectives Twice (in plan period)

As per the TOR MOE/ ERO

Evaluation of model building exercises

Key results/ obj As required As per the TOR MOE/ DOE

Mid-term evaluation Theme/ LF level Once (midterm) As per the TOR MOE

End-of-programme evaluation

Theme/ LF level Once (after completion)

As per the TOR MOE

Lessons learnt

• SWAp contributory to institutionalize MfDR

• Participatory design of M&E system crucial

• Commitment and capacity key to success- focus lower levels and retain talents in the M&E function

• Link incentives with credible M&E/ MIS data

• Triangulation of methods/tools for data reliability

• Strengthen the system rather re-inventing it

• Make multi-party review meetings results- based

• Produce data what is needed and usable

Challenges/ Issues

• Make the system functional and effective

• Cascading results approaches to lower levels- – capacity issues, – cost incurred

• Data issues– frequency and reliability (survey vs. MIS)– disaggregated data- diverse social groups

• Harmonize TA funding for capacity development

• Frequent changes in leadership and key staff

• Use of M&E data in managerial decision making

Discussion

• Promote country level capacities monitoring education MDGs and EFA agenda through regional capacity building initiatives

• Use of VLN of COP- MfDR to further build capacities through exchanging ideas

Thank you