Document of The World Bank · 7 06/11/2015 Moderately Satisfactory Moderately Satisfactory 10.00 H. Restructuring (if any) Restructuring Date(s) ... policy research on teaching and

Document of The World Bank

Report No: ICR00003529

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-11836)

ON A

GLOBAL PARTNERSHIP FOR EDUCATION FUND GRANT

IN THE AMOUNT OF US$10 MILLION

TO

MONGOLIA

FOR AN

EARLY CHILDHOOD EDUCATION PROJECT

December 9, 2015

Education Global Practice East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective October 30, 2015)

Currency Unit = MNT (Tugriks) 1.00 MNT = US$ 0.000503271

US$ 1.00 = 1,992.00 MNT

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank IDA International Development Association

CBA Cost Benefit Analysis IRR Internal Rate of Return EAP-ECDS East Asia Pacific Early Childhood

Development Scale ISM Implementation Support

Mission CCU Client Construction Unit LSMS Living Standards and Measurement

Survey CPS Country Partnership Strategy M&E Monitoring and Evaluation ECD Early Childhood Development MTR Mid-term Review ECE Early Childhood Education MDGs Millennium Development Goals EFA-FTI Education for All-Fast Track

Initiative MECS Ministry of Education, Culture, and

Science EMP Environmental Management Plan MRTCUD Ministry of Roads, Transportation,

Construction, and Urban Development

ESMP2 Education Sector Master Plan NCB National Competitive Bidding FM Financial Management NEMA National Emergency Management

Agency GER Gross Enrollment Rate NPV Net Present Value GGSEDM General Guideline for Socio-

Economic Development of Mongolia PAD Project Appraisal Document

GOM Government of Mongolia PDO Project Development Objective GPE Global Partnership for Education PIU Project Implementation Unit ICR Implementation Completion Report READ Rural Education and Development

Project

Senior Global Practice Director: Claudia Maria Costin

Practice Manager: Harry Anthony Patrinos

Project Team Leader: Prateek Tandon

ICR Team Leader: Prateek Tandon

i

A. Basic Information

Country: Mongolia Project Name:

Mongolia Global Partnership for Education Early Childhood Education Project

Project ID: P125445 L/C/TF Number(s): TF-11836

ICR Date: 12/09/2015 ICR Type: Core ICR

Lending Instrument: SIL Grantee: MONGOLIA

Original Total Commitment:

USD 10.00M Disbursed Amount: USD 9.88M

Revised Amount: USD 9.88M

Environmental Category: B

Implementing Agencies:

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 06/21/2011 Effectiveness: 03/30/2012 03/06/2012

Appraisal: 08/19/2011 Restructuring(s): 06/09/2014 02/27/2015

Approval: 02/13/2012 Mid-term Review: 10/25/2013 10/25/2013

Closing: 06/30/2014 06/30/2015 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Satisfactory

Grantee Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Moderately Unsatisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Moderately Satisfactory

ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Pre-primary education 100 100

Theme Code (as % of total Bank financing)

Education for all 100 100 E. Bank Staff

Positions At ICR At Approval

Vice President: Axel van Trotsenburg Pamela Cox

Country Director: Bert Hofman Klaus Rohland

Practice Manager/Manager: Harry Anthony Patrinos Luis Benveniste

Project Team Leader: Prateek Tandon Prateek Tandon

ICR Team Leader: Prateek Tandon

ICR Primary Author: Sandra F. Beemer F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)The project development objective was to assist the Government to cope with the surge in total fertility rates by providing access to ECE for children in disadvantaged communities. Revised Project Development Objectives (as approved by original approving authority)

iii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : GER of children aged 2-5 in urban districts in UB and where new kindergartens are provided (disaggregated by gender)

Value quantitative or Qualitative)

UB = 67.9% Boys = 66.8% Girls = 69.1%

UB= 67.9% Boys = 66.8% Girls = 69.1%

N/A

UB=72.64% Boys=71.9% Girls=73.3% Source: MECS

Date achieved 06/30/2010 06/30/2014 06/30/2015 Comments (incl. % achievement)

GER in the areas where project fixed kindergartens were provided increased overall by 4.7 percent; boys increased 5.1 percent and girls increased 4.2 percent. See footnote 11 for explanation as to why the target and baseline are the same.

Indicator 2 : Number of children attending new mobile ger-kindergartens Value quantitative or Qualitative)

0 1500 N/A 3,781 Source: MECS


Target exceeded

Indicator 3 : Average daily attendance rate of the mobile ger kindergartens Value quantitative or Qualitative)

0% 90% 99.8% Source: parent satisfaction survey


Target exceeded. Indicator was modified during 2013 proposed restructuring to focus on ger-kindergartens rather than fixed kindergartens.

Indicator 4 : Satisfaction of parents with the mobile ger service Value quantitative or Qualitative)

0% 60% 91% Source: parent satisfaction survey


Target exceeded. Indicator was modified during 2013 proposed restructuring to focus on ger-kindergartens rather than fixed kindergartens.

Indicator 5 : Percentage of beneficiaries from disadvantaged communities Value quantitative or Qualitative)

0% 50% N/A 71% Source: parent satisfaction survey


Target exceeded. Disadvantaged means 72.5 percent of the parents sending their children to ger-kindergartens were herder families with unstable earnings or

iv

unemployed; 71 percent were 100km from the aimag center where fixed kindergartens are located.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Increase state-owned kindergarten facilities by 3.4% Value (quantitative or Qualitative)

722 759 745 777 or 7.6% increaseSource: MECS


Target exceeded. Indicator target was modified from 5.1 to 3.4 percent in 2013 based on reduction of GPE fixed kindergartens. The overall increase is attributed to additional government kindergartens in the amount of 60-70 each year as of 2013.

Indicator 2 : Increase mobile alternative teaching classes by 14.1% Value (quantitative or Qualitative)

711 811 N/A 658 or a 7.4% decrease Source: MECS


The project supplied 100 mobile ger-kindergartens but government has not replaced state-provided mobile ger-kindergartens because of rural migration to urban areas and increased attendance in fixed kindergartens.

Indicator 3 : Increase in the number of children enrolled in kindergartens by 4,000 Value (quantitative or Qualitative)

142,065 147,565 143,565 206,636 Source: MECS

Date achieved 06/30/2010 06/30/2014 06/30/2014 06/30/2015 Comments (incl. % achievement)

Target exceeded. The increase in kindergarten enrollment from 2010-15 was 64,571.

Indicator 4 : Direct project beneficiaries of which female (%) Value (quantitative or Qualitative)

0 3,200 (47%) N/A 6,738 (49.2%) Source: PIU


Target exceeded. This indicator was added during the 2013 proposed restructuring to accommodate the new IDA indicator requirements.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements

(USD millions) 1 07/07/2012 Satisfactory Moderately Satisfactory 2.00

v

2 01/28/2013 Satisfactory Moderately Satisfactory 2.00 3 09/17/2013 Moderately Satisfactory Moderately Satisfactory 2.00 4 05/15/2014 Moderately Unsatisfactory Moderately Unsatisfactory 4.35 5 09/26/2014 Moderately Unsatisfactory Moderately Satisfactory 6.66 6 11/26/2014 Moderately Satisfactory Moderately Satisfactory 8.10 7 06/11/2015 Moderately Satisfactory Moderately Satisfactory 10.00

H. Restructuring (if any)

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

06/09/2014 MU MU 4.35

(i) extend the project closing date by 12 months from June 30, 2014 to June 30, 2015; and (ii) reallocate proceeds from Category 1 (Works) to Category 4 (Operating Costs) to facilitate monitoring expenses related to the supervision of the fixed kindergartens.

02/27/2015 MS MS 9.85

(i) reallocate project savings of approximately US$1.5 million in Category 1 (civil works) due to devaluation of the Mongolian Tugrik. These funds were reallocated to Category 2 (goods), Category 3 (consultant services) and Category 4 (operating costs) to supply additional learning materials and equipment to kindergartens throughout the country, conduct some preliminary policy research on teaching and learning practices on ECE, and facilitate additional management costs and audit expenses; and (ii) formally amend the project description and adjust the results framework that had already been approved January 23, 2013 but not formally countersigned by the Recipient.

vi

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1. Country Context. In 1990, Mongolia transitioned from a planned economy to a market democracy and has experienced two decades of dramatic transformation. Mongolia was seen as a country with a rich nomadic culture and history, but facing a range of development constraints. It was (and remains) the least densely populated country in the world with 1.7 persons for one square kilometer of land, making service delivery extremely costly. It is landlocked, making it necessary to import most of its goods by means of a slow and expensive land transport system. It faces extreme climate conditions ranging from over 40 degrees Celsius to minus 50 degrees Celsius, which make the construction seasons short, and can dramatically affect livestock and agriculture. Although the development of a market economy and mineral resources increased public revenue, the proportion of people living below the national poverty line was 35 percent in 2008. 2. At the time of appraisal, approximately 40 percent of the population lived in the capital city, Ulaanbaatar, and migration to urban areas continued to increase. Poverty was significantly higher in rural areas than in urban centers, at 47 percent versus 27 percent. In education, the urban and rural disparity was manifested in access, school completion rates, and learning outcomes and inequity in education was perpetuating the cycle of poverty. 3. Sectoral Context. The Ministry of Education, Culture and Science (MECS) developed a new Education Sector Master Plan (ESMP2) for 2006-2015 that built on the General Guideline for Socio-Economic Development of Mongolia (GGSEDM) for 2006-2008. The GGSEDM identified five priority actions for education: (i) reduce school dropouts and provide elementary education for all children (raising primary education coverage to 94.5 percent by 2008); (ii) transform secondary education into an 11-year system by 2006 and a 12-year system by 2007; (iii) improve the learning environment and physical facilities at secondary schools and assure sufficient supply of teachers and textbooks; (iv) lower gender inequality in primary and secondary school enrollment as well as literacy levels; and (v) increase access to schools for children with disabilities by establishing necessary equipment at schools. The ESMP2 sought to sequence the government priorities by: (i) upgrading education quality at all levels of schooling and producing citizens who could function effectively in a modern market economy; (ii) providing education services that could be accessed by children in all parts of the country, including rural areas, and poor and vulnerable groups; and (iii) improving the management capacity of central and local educational institutions at all levels. 4. The 2010 population census showed that Mongolia had a very young population – nearly 30 percent under the age of 14 and about 12 percent under the age of 5. Basic education in Mongolia had followed the trends of decline and recovery in the economy. The 1990s saw a drastic decline in school enrollment ratios; the 2000s witnessed a recovery to the pre-transition level in pre-primary through lower secondary education and it was expected that the current decade would see an extension of access in pre-primary and secondary education and an effort to improve quality. 5. The recovery in basic education began with the development of the ESMP2 (2006-2015) which received considerable donor support. The Bank-supported Rural Education and

2

Development Project (READ) (IDA-H2210 IDA-H3730 TF-96333) and then by the Global Partnership for Education (GPE) (formerly the Educational for All-Fast Track Initiative (EFA-FTI)) Project (2007-2012) were part of the ESMP2 support. The GPE’s involvement was consistent with the GPE goal of providing financial assistance to help low- and middle-income countries around the world achieve the Millennium Development Goals (MDGs) in education. The Bank’s involvement was also consistent with the Government of Mongolia’s (GOM) commitment to the achievement of the MDGs by 2015. Moreover, the project was consistent with the Interim Strategy Note (ISN) 2009-2010 pillar two of inclusive growth. The government knew that low levels of educational attainment were key determinants of poverty and that poverty could be a key factor that limited access to and quality of schooling. Therefore, investment in education was important because improvements in the sector could break the intergenerational transmission of poverty.

6. In 2012, 24 percent of the 2-5 age group1 (44,903 children) did not have access to early childhood education (ECE) due to the lack of kindergarten facilities, and the vast majority of them were from disadvantaged communities such as herders and the poor in peri-urban areas. The rising total fertility rate from 2.2 in 2000 to 2.7 in 2009 was in part due to the government introduction of child care cash transfers payable to every family for each child. It was expected if the 2009 level of fertility and capacity held, without an increase in supply, that gross enrollment rate (GER) in kindergartens would decline from 76 percent in 2010 to perhaps just over 50 percent by 2015, instead of rising to the target of 79 percent in 2012. A MECS study on school readiness, done in collaboration with the United Nationals Children’s Fund (UNICEF) in 2008, found major differences in school readiness between 6-year-olds who had attended kindergartens and those who did not. Therefore, a decline in GER in kindergarten could create a domino effect: far fewer children, particularly those from disadvantaged backgrounds, would have access to free school meals offered in kindergartens to improve their nutritional status; school readiness would necessarily suffer, student achievement would decline, the dropout rate would increase, and overall educational quality would be hampered. 7. The MECS recognized that investments in ECE were a highly cost-effective strategy to break the inter-generational transmission of poverty and to improve long term productivity. The Mongolian Pre-school Education Law of 2008 stipulated that the provision of food, books, manuals and appropriate toys for children attending state-owned kindergartens as well as the norm-based variable costs, should be financed from the state budget. Prior to the law’s implementation, parents were obligated to pay 50 percent of food costs and other expenses -- major obstacles for parents to enroll their children to preschool. The law lifted this financial barrier, and drastically increased demand for preschool education. The trend in public spending per student in kindergarten showed a strong commitment from the Government. Between 2006 and 2011, despite an increase in enrollment from 95,038 to 122,126 students, per student spending increased from US$146 to US$643.

1 In 2010, GER in kindergartens rose dramatically from 57 percent in 2007 to 76 percent in 2010, largely due to the rapid increase of donor-financed mobile ger-kindergartens. However, ger-kindergartens operate only in the summer to reach out to the herders. While having ger-kindergartens is better than none, they are not comparable to the services delivered by fixed kindergartens because of the short duration.

3

8. Signifying the Government’s commitment to ECE, a national policy on integrated ECE had been adopted jointly by MECS, the Ministry of Health, and the Ministry of Social Welfare and Labor. The policy coordinated the efforts of state central organizations in charge of mother and child health care, nutrition, social welfare and protection, and education. In addition to building kindergartens for children aged 2–5, the policy provided in-service training for all ECE teachers in Mongolia, consultation services for families, mothers, and women through pediatric and psychological home services, non-formal education services, telephone and internet counseling services, training and information campaigns on radio, television, in newspapers, and other print media. These outreach activities included counseling information on nutrition, hygiene, immunization, health care, and child-rearing practices that can stimulate social and cognitive development. These activities were codified into the national law on ECE in Mongolia. 1.2 Original Project Development Objectives (PDO) and Key Indicators 9. The project development objective was to assist the Government to cope with the surge in total fertility rates by providing access to ECE for children in disadvantaged2 communities. The key indicators were: (i) GER of children aged 2-5 in the urban districts in Ulaanbaatar (UB) and in aimag3 centers where new kindergartens are provided (disaggregated by gender); (ii) number of children attending new mobile ger-kindergartens; (iii) average daily attendance rates in the 37 fixed kindergartens; (iv) parental satisfaction with the ECE service; and (v) percentage of beneficiaries from disadvantaged areas.4 The project objective was to be achieved by increasing the supply of fixed kindergartens in urban and peri-urban areas, creating alternative preschool classes in mobile ger-kindergartens that follow herders in the summer, improving teaching and learning environments, supplying learning materials, furniture, equipment and toys, and undertaking monitoring and evaluation (M&E). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification 10. The PDO was not modified over the life of the project however, the key indicators were modified in January 2013 and subsequently monitor and reported on in project Implementation Status and Results (ISRs) reports. The modification are as follows:

2 Disadvantaged communities were selected based on children between ages 2-5 that had relatively low enrollment ratios in kindergarten. 3 Mongolia is divided administratively into UB and 21 aimags (provinces). UB is divided into districts and Khoroos (city wards). The aimags are divided into districts known as soums, and soums are divided into baghs (villages). 4 Baseline data used for the indicators was for school year 2009-10. School year 2011-12 data was not available until after the project became effective in March 2012. Therefore, baseline data for the PAD and the first ISR remained the same.

4

Original PDO Indicator Adjustments made in 2013

GER of children aged 2-5 in the urban districts in Ulaanbaatar (UB) and aimag centers where new kindergartens are provided (disaggregated by gender)

Changed to remove sub-indicators that were tracking aimag center GER because the number of new kindergartens was reduced and were constructed in UB.

Number of children attending new mobile ger-kindergartens

Continued as originally designed

Average daily attendance rate of the 37 fixed kindergartens

Changed to track the attendance rates of mobile ger-kindergartens

Satisfaction of parent with service Modified to track satisfaction of mobile ger-kindergartens only.

Percentage of beneficiaries from disadvantaged communities


Intermediate Indicators Increase state-owned kindergarten facilities by 5.1 percent

Changed to 3.4 percent due to decrease in the number of kindergartens to be built during the project period.

Increase mobile alternative teaching classes by 14.1%


Increase the number of children enrolled in kindergarten by 5,500

Changed to 4,000 due to decrease in the number of kindergartens to be built during the project period.

Direct project beneficiaries (number) of which female (%)

Added to meet corporate requirements

1.4 Main Beneficiaries 11. The Project Appraisal Document (PAD) identified children between the ages of 2-5 who live in communities which have relatively low enrollment ratios in kindergarten as the primary beneficiaries of the project. It was expected that each year approximately 4,000 additional children would be able to attend fixed kindergartens and 1,500 additional children would benefit from the supply of mobile ger-kindergartens. Direct project beneficiaries having benefitted from project interventions reached approximately 6,738 children of which 49.2 percent were female. This does not include the indirect beneficiaries such as parents and decentralized ministry staff who benefitted from knowledge transfer and other forms of capacity building activities as well as teachers who benefited from being able to use new and better learning materials. 1.5 Original Components 12. Component 1: Increasing Kindergarten Capacity in Urban and Peri-urban Areas (GPE-US$9,315,000 and Counterpart--US$2,849,000) was to provide a set of interventions that were directed towards ensuring that children ages 2-5 in the urban and peri-urban areas would have access to kindergartens by: (i) building 37 new kindergartens, of which 21 would be located in aimag centers and 16 in UB and (ii) providing furniture, equipment, indoor and outdoor playgrounds, teaching and learning materials and kitchens for the new kindergartens.

5

13. Component 2: Creating Alternative Pre-school Classes in Rural Areas (GPE -US$475,000- and Counterpart-US$0.0) was to provide financing to: (i) purchase 100 mobile ger-kindergartens and (ii) produce and print learning materials for alternative classrooms. 14. Component 3: Grant Management and Monitoring and Evaluation (GPE -US$210,000-and Counterpart US$0.0) was to provide financing to: (i) support the project implementation unit (PIU); (ii) conduce the parent survey; and (iii) overall project monitoring and evaluation (M&E).

1.6 Revised Components 15. The components were not revised over the life of the project however some of the targets were modified following the mid-term review (MTR), which were recorded in a project restructuring in January 23, 2013.

1.7 Other Significant Changes 16. On December 15, 2011, the GPE approved a grant in the amount of US$10 million to implement the ECE project. The grant agreement was signed on March 6, 2012 between the GOM and the Bank acting as the supervising entity of the GPE funds. 17. On November 20, 2012 the Mongolian Ministry of Finance requested that the project be restructured and on January 21, 2013, the project was restructured and approved by the country director with an amendment letter dated January 23, 2013 to: (i) reduce the original number of kindergartens to be built from 37 to 25 (17 from GPE funds and 8 from government funds) in response to cost escalation of raw materials and to remain within the budget;5 (ii) adjust the results framework to accommodate the reduction in the project interventions; (iii) add a contingency expenditure category with 13 percent of project funds to absorb any possible future price escalation; and (iv) reallocate project funds from the works category to consultant services to hire a civil works officer to help the PIU with implementation. Based on these modifications and agreements, the Bank’s ISRs were adjusted to monitor the revised indicators and data was collected according to the revisions. However, the January 23, 2013 amendment letter was not countersigned by the Recipient because of elections and a change in the government which replaced personnel in the Ministry of Finance. 18. On June 9, 2014 the project was restructured to: (i) extend the project closing date by 12 months from June 30, 2014 to June 30, 2015; and (ii) reallocate proceeds from Category 1 (Works) to Category 4 (Operating Costs) to facilitate monitoring expenses related to the supervision of the fixed kindergartens.

5Cost of some construction materials had increased by more than 50 percent; distribution costs had increased with price increases in gasoline; and the government’s benchmark standard pricing for machinery increased between 3-5 times.

6

19. On February 27, 2015 the project was restructured to: (i) reallocate project savings of approximately US$1.5 million in Category 1 (civil works) due to devaluation of the Mongolian Tugrik. These funds were reallocated to Category 2 (goods), Category 3 (consultant services) and Category 4 (operating costs) to supply additional learning materials and equipment to kindergartens throughout the country, conduct some preliminary policy research on teaching and learning practices on ECE, and facilitate additional management costs and audit expenses; and (ii) formally amend the project description and adjust the results framework that had already been approved January 23, 2013 but not formally countersigned by the Recipient. The lack of countersignature was discovered in February 2015. With the GPE Secretariat’s approval on February 3, 2015, the amendment to the grant agreement and reallocation was signed between the Bank and Ministry of Finance on March 4, 2015. 2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 20. Project Preparation. The ECE project was prepared as a GPE-supported “graduation grant” because being an International Development Agency (IDA) Category III classified country, Mongolia no longer qualified to receive funding from the GPE. However, in June 2011 the GPE Board of Directors considered that IDA Category III countries could be eligible for GPE funding as an exit strategy from the GPE. Based on this decision, the GOM requested a grant of US$10 million from the GPE board which was allocated and approved by the GPE board in December 2011. The Bank was selected, by the government, to be the supervising entity and to assist the MECS in the preparation of the application in close collaboration with other donors in Mongolia. 21. The preparation team used the Specific Investment Loan (SIL) instrument given the targeted nature of program interventions. The team included project activities based on lessons learned from other Bank-supported education projects as well as those being implemented by development partners (UNICEF, Asian Development Bank (ADB), and GPE). These included: (i) use of mobile ger-kindergartens to reach the most disadvantaged areas; (ii) construction of child-friendly, low cost, energy efficient kindergartens; and (iii) provision of learning materials. Project preparation also benefited from the fact that the PIU from the first GPE project was still in place at the time of preparation and had extensive experience with implementation of investment projects as well as procurement, financial management (FM), contract management and working with the Education Donors’ Consultative Mechanism (EDCM). These experiences and lessons learned helped the team prepare and appraise the proposed program in seven months and meet GPE’s December 2011 submission deadline.

22. Program preparation was thoughtful, collaborative and technically sound and benefitted from a deliberative process between MECS, donor partners and the Bank. The Bank preparation and appraisal teams consisted of technical experts that were appropriate for the development of the program and included peer reviewers with extensive ECE knowledge and experience that provided sound guidance on the project design. 23. Project Design and Quality at Entry. The project design fit into the government’s Education Sector Master Plan (ESMP2) for 2006-2015 that provided the platform for sectoral planning and financing, the GPE goals of meeting the 2015 MDG goals, and the Bank’s Interim

7

Strategy Note 2009-10. The project design took into consideration comments from the peer reviewers during the concept and appraisal review meetings which included: (i) focusing the project on intermediary outcomes such as quality of the preschool environment; (ii) including indicators that measured average attendance rate of preschoolers and disaggregated participation in Early Childhood Development (ECD) within the 2-5 age group; and (iii) keeping the design simple considering the short two year implementation period for the GPE project. 24. The PDO was in line with the government’s priorities and was reflective of the program’s activities. The key indicators were appropriate for measuring progress toward achieving the PDO and the targets reflected the original project targets. As indicated, in January 2013, the targets were revised to reflect a reduction in the number of kindergartens being constructed due to considerable escalation in construction costs. At the same time the IDA core indicator6 that measures direct project beneficiaries was added to the results framework. The government public investment plan was to build 300 kindergartens nationwide over a period of 5 years beginning in 2013 and with the project resources the government was able to add additional schools to the plan. The additional schools, at the ministry request, targeted the areas in UB with the highest access disparities.

25. The project design included a two year implementation period which was shorter than the usual three year implementation period for GPE projects. This was because GPE considered the project allocation to be bridge financing as Mongolia graduated and could, at the time, only guarantee the funding for two years. As a result, the project design was kept simple and comprised three components, primarily related to access to ECE, with a very limited number of activities. In addition, at the request of GPE, the design included counterpart funds to show government commitment to the sub-sector since Mongolia was transitioning from GPE supported projects. The design also included alternative preschool classes in rural areas especially for herders’ children by providing mobile ger-kindergartens, a UNICEF innovation, and a continuation of an activity from the first EFA-FTI project.

26. The implementation arrangements included the use of a PIU with staff from the first GPE project to accommodate the short two year implementation period as well as a third party monitor to ensure the quality of construction of the fixed kindergartens. The design also included a parent survey to assess parent satisfaction with the new kindergartens. As indicated, this was modified in January 2013 to conduct a parent satisfaction survey of the mobile ger-kindergartens. The preparation team identified project risks and proposed adequate mitigation measures. (See paragraph 57 for further discussion on risks.) 2.2 Implementation 27. Program implementation was initially slow. The project became effective in March 2012 and by the first supervision mission in June 2012 there was little physical progress. Applications had been made as early as December 2011 to get approval from the Ministry of Roads,

6 “Core indicators” are a set of indicators introduced by the Bank in July 2009 to allow for the systematic capture of results data at a country, sector, regional, and institutional level. These indicators are required for operations of IDA-eligible countries.

8

Transportation, Construction, and Urban Development (MRTCUD) for the two-storied buildings using Canadian wood framed technology 7. However, by June 2012 the approval was still pending because of changes in government. Until the MRTCUD approved the use of the wood framed technology and the National Emergency Management Agency (NEMA) made recommendations on the needed fire safety requirements, design work on the kindergartens could not begin. In addition, by September 2012, the Bank, the ministry Client Construction Unit (CCU) and PIU had to work together to re-estimate the project costs because: (i) the price of some construction materials had increased by more than 50 percent as a consequence of the economic volatility of Mongolia’s commodity-dependent economy; (ii) distribution costs had increased with price increases in gasoline; and (iii) the government’s benchmark standard pricing for machinery increased between 3-5 times as a new material pricing directory became effective under the order of the Ministry of Construction. Based on this, it was agreed that project could no longer finance 37 kindergartens and agreed to reduce the number of schools constructed with GPE funds to 17 and government funded kindergartens to 8. This led to the November 2012 project restructuring. The restructuring was appropriate considering that the price escalations were well beyond the inflation rates considered during the original project costing. Furthermore, with the reduced numbers of schools, it was appropriate to adjust the indicator targets which were directly tied to the number of kindergartens being constructed and to focus some of the outcomes on the mobile ger-kindergartens given the project investment in these kindergartens. 28. During the first week of January 2013, the PIU team was released from their positions, with the exception of the financial officer, because a new government had been elected and almost all technical-level MECS staff were being replaced. By April 2013, a new PIU staff was in place. They had limited experience in project implementation so the Bank team in Mongolia provided intensive support on a daily basis to the PIU and MECS in order to quickly overcome the capacity constraints. Also in January 2013, after a 15 month delay, the MRTCUD and NEMA sent the CCU the approval for using the wood framed technology so that engineering firms could begin to design the schools. By August 2013, (i) the designs had been approved and the Bank’s final no objection letter (NOL) had been issued on the bidding documents; (ii) it was expected that the foundations could be set during the 2013 construction season thereby ensuring a timely completion of the construction; (iii) the mobile ger-kindergartens has been procured and distributed; (iv) the plan for sampling of the parental satisfaction survey had been agreed with the Bank; (v) review of the Interim Financial Reports (IFRs) showed no unusual items; and (vi) the team was planning for a October 2013 MTR. 29. In November 2013, the MTR confirmed that the project was still relevant and remained in line with the ESMP2 and that the development objectives remained valid. However, at the time of the MTR the project was downgraded to moderately unsatisfactory. This was because construction

7 The applications for the designs were submitted to the Ministry of Roads, Transportation, Construction and Urban Development in December 2011 and the approvals were received in January 2013. The designs were Canadian wood framed kindergartens that were selected because: (i) Canada had similar temperature fluctuations to that of Mongolia and (ii) they were considered less expensive than the traditional brick structures used in Mongolia. Given that Mongolia plans to build approximately 60-70 new kindergartens over the next five years, the Canadian wood framed technology is being considered for nationwide construction beginning in 2016.

9

on two of the 17 kindergartens was again delayed due to: (i) a land dispute which had been resolved to the full satisfaction of the ministry and the complainants; and (ii) the foundation for one school could not be set because of severe weather in the Baganuur district. These delays would necessitate a one year extension of the project closing date from June 30, 2014 to June 30, 2015 in order to ensure achievement of the PDO. 30. By October 2014, construction had progressed significantly with: (i) ten kindergartens completed and occupancy certificates issued and handed over to the school principal; (ii) six kindergartens at advanced stages of completion with four expecting occupancy certificates; and (iii) one kindergarten approximately 55 percent completed. The third party monitor for reviewing the quality of construction had been appointed and was providing input to ensure quality. Demands for enrollment in the new kindergartens had increased by 70 percent and principals were requesting extra furniture based on the increased demand. Moreover, the 100 mobile ger-kindergartens had been supplied and distributed and had approximately 2,500 children from herder families were attending in the summer. The parent satisfactory survey had also been administered to approximately 2,000 parents from the 100 mobile ger-kindergartens. The project disbursements had also increased to 81 percent of total grant funds. All-in-all substantial progress after the initial delays was evident. 31. As mentioned previously, in February 27, 2015 the project was restructured to reallocate project savings due to devaluation of the Mongolian Tugrik that resulted in a project savings of approximately US$1.5 million. At that time, there was agreement to use the additional funds to supply toys and books to 500 public kindergartens nationwide because the state budget had not been able to cover the costs of toys and books over the previous five year period. The decision was appropriate because it supported the overall effort to lay the groundwork for important school readiness concepts and would enhance the project’s development impact significantly. 32. By project closing all activities had been completed. (See Annex 2 for details on project outputs). There were six Bank review and implementation support missions from September 2012 to May 2015. The makeup of the Bank teams was consistently appropriate and well balanced. Each team composition reflected the needs of the respective mission and included specialists from the areas of education, FM, procurement, construction, environment, social, and monitoring and evaluation specialists as necessary. The total project disbursements were approximately US$10.0 million or 100 percent of the total of US$10.00 million and all activities were completed prior to project closing on June 30, 2105. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 33. Design. The M&E design had two principal dimensions. The monitoring of the results framework and project outputs was to be done by the PIU. As indicated above, the results framework was restructured in 2013 to modify targets based on the reduced size of the project inputs which was the appropriate course of action. Secondly, the project included a parental satisfaction survey to assess parent’s satisfaction with the mobile ger-kindergartens and to gather data on their perception of their children’s development upon receipt of the kindergarten services. The survey collected data from a representative sample of parents of the 100 mobile ger-

10

kindergartens in the 14 targeted aimags.8 This was an important design feature for understanding the effectiveness of this intervention. 34. Implementation and Utilization. The PIU and MECS/EMIS regularly provided Bank missions with data and information that allowed teams to make judgments on progress toward achieving the PDO and the project’s overall implementation. This information was consistently reported through the Bank’s aide-memoires and Implementation Status and Results (ISRs) reports. The PIU also monitored the project outputs and used the information to make the appropriate adjustments in its annual plans and progress reports. The results of the parental satisfaction survey were used to develop a database that monitor and study the sample group of children’s physical well-being and motor development; social and emotional development; approach to learning/language development; cognitive development; and general knowledge. This is a substantial achievement for a three year implementation period. 2.4 Safeguard and Fiduciary Compliance 35. Environment and Social Aspects. The program was rated a Category “B” operation, requiring partial assessment. The program’s compliance was rated “satisfactory” throughout its life. The project prepared Environmental Safeguards Guidelines that included an Environmental Management Plan (EMP) to be used in mitigating any adverse impacts caused by construction activities that were used by the construction engineers. In addition, the two-story wood frame building technology required the approval of the Ministry of Roads, Transportation, Construction and Urban Development prior to being constructed and the National Emergency Management Agency on fire safety requirements. The construction engineers used the EMP during construction which was routinely verified during implementation support missions (ISMs) so that by project closing, the Bank’s engineer and environmental safeguards specialist confirmed that civil works were completed in compliance with the EMP and relevant environmental laws. Compliance to the safeguards guidelines as well as fire safety were monitored and reported on extensively during ISMs by the Bank team engineer who participated all mission. By project closing, all the additional recommendations made related to fire safety had been completed.9 36. There was no land acquisition expected since all works were to be located on state-owned land. However, at the time of the October 2013 MTR, there was a land dispute concerning garages registered with the MECS and Bayangol District Property Administration. Under the oversight of

8 The sample survey methodology was to assess the satisfaction of parents from the 100 mobile ger-kindergartens provided by the GPE project. ECE offices within education departments in each aimag were trained on how to conduct interviews and fill out questionnaires. The survey was done in August/September 2014 and the sample covered 2,002 parents sending their children to mobile ger-kindergartens in 14 aimags. 9 The fire safety recommendations and actions included fully functioning fire sprinkler systems and smoke detectors. To ensure that the sprinkler systems were fully functional contractors made modifications as requested such as: (i) moving sprinkler system pipes that were above the floor-to-ceiling down to avoid freezing during the winter; (ii) ensuring that they systems had a separate electrical circuit; (iii) ensuring that all schools had fire extinguishers; (iv) ensuring that water storage tanks circulated water so that the water did not remain stagnant, (v) ensuring exit lighting along with drawing of fire exit schemes; (vi) training for school principals and staff on running and maintaining the fire system; and (vii) ensuring that fire zones were clearly marked on the fire panels.

11

the Bank’s social development specialist, the dispute was resolved to the satisfaction of all parties and all related documents were filed as required. The Bank’s OP/BP 4.10 was triggered but a separate Indigenous Peoples Plan (IPP) was not required because the Kazakh communities were the overwhelming majority in Bayan-Ulgii and the elements of an IPP were included in the overall project design. 37. Financial Management. Financial management remained satisfactory throughout the life of the project. The unaudited interim financial reports (IFRs) were submitted on time. The project audit reports were also submitted on time and were always unqualified (clean). Disbursements during the first two years of project implementation were slow mainly due to the delays in the construction of the kindergartens. As mentioned above, once the kindergarten construction began in earnest, disbursements were on target. 38. Procurement. Early during project implementation procurement was rated moderately unsatisfactory due to the delays related to construction already mentioned. However, by project closing procurement was rated satisfactory because procurement capacity within the PIU had been improved which was evidenced with the procurement of books for kindergartens nationwide in February 2015. Improved capacity was further evidenced by the fact that in May 2015 one of suppliers filed a complaint that that the PIU was able to handle to the full satisfaction of the Bank. Procurement plans were prepared and routinely submitted for review by the Bank, found to be acceptable and used as a basis to judge overall implementation progress. Bank implementation support missions carried out ex-post reviews and also confirmed that the project was in conformity with the project and the Bank’s procurement guidelines. 2.5 Post-completion Operation/Next Phase 39. The MECS and government are fully committed to continuing support for ECE. The 2008 education law mandates that 4 percent of the total education budget be allocated to ECE. The government, in parallel with the ECE project implementation, began constructing kindergartens to accommodate the high birth rate of approximately 80,000 children in 2013 and 2014. Starting in 2013, approximately 60-70 kindergartens per year are being constructed for an estimated total cost of US$35 million per year 10 . The government is also hiring the needed teachers for these kindergartens. The Bank and Save the Children have also conducted a study11 that: (i) looks at ECE outcomes throughout the country and how they differ by social and economic status and (ii) analyzes possible reasons for variations in outcomes. Once completed, the study will be used by the government to inform future policy decisions for ECE as well as future project development. In addition, the Bank has an ongoing Mongolia: JSDF Grant for Improving Primary Education Outcomes for the Most Vulnerable Children in Rural Mongolia Project that is providing a school preparation program that works with vulnerable children (5-6 years of age) by introducing home based school preparation with active participation of parents and community stakeholders. Building on the experience of the ECE project and the Japan Social Development Fund (JSDF)

10 The total costs includes operating costs. 11 The results of the study will be available by May 2016.

12

grant project, the MECS and Bank will continue to focus future cooperation on ECE related activities. Discussions are underway for further IDA support to ECE under IDA 17. 3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation 40. Relevance of Objectives. The project development objectives were highly relevant to the country’s sectoral needs when the project was developed. The objectives fit into the government’s ESMP2 for 2006-2015 and the Bank’s 2009-2010 Interim Strategy Note. They shared many common objectives related to increased access of poor and vulnerable groups to basic education services. The project objectives continue to fit into the FY2013-17 CPS for Mongolia and the ESMP2, both indicating that improving the delivery of education services is a national priority and a core part of the government’s overall strategy for education. The objectives also continue to be aligned with the Bank’s twin goals of eradicating extreme poverty and increasing shared prosperity given that literature on human capital has documented the positive correlation between human capital investment and social and economic development. 41. Relevance of Design. The project design was highly relevant and, again, aligned with the government’s ESMP2. The overall design was simple to accommodate the original two year implementation period and built on experiences gained from other Bank and donor supported projects. The PDO was simple and the indicators were appropriate to measure achievement of the development objectives. When the project was restructured to reduce the number of fixed kindergartens, the indicators were adjusted appropriately to measure achievement of the PDO. The inclusion of the mobile ger-kindergartens was appropriate considering the herders’ nomadic lifestyle and the ongoing effort of the government to increase the preschool enrollment in the rural areas. While exogenous factors led to the replacement of the original PIU staff, the original design that included staff from the first GPE project was appropriate. The parent satisfaction survey was also an appropriate design feature. 42. Relevance of Implementation. The relevance of project implementation is judged to be substantial. As indicated above, there were implementation challenges related to the construction of the fixed kindergartens. However, the PIU, working with Bank staff, were able to resolve the issues and complete all the fixed kindergartens. The implementation team also completed all the other planned activities as well as the additional activities of purchasing and distributing the additional books and toys nationwide. Moreover, the team conducted the parent satisfaction survey, developed a tracking data base and analyzed the data which will be used for informing future policy decisions. It should also be noted that the implementation team complied with all of the extra requests and requirements related to the fire safety of the construction of the Canadian wood frame kindergartens. After each Bank ISM all requested modifications were completed so that the schools were in compliance with the required safety regulations. These are all substantial achievements for a short three year implementation period. 3.2 Achievement of Project Development Objectives 43. The project development objective was to assist the Government to cope with the surge in total fertility rates by providing access to ECE for children in disadvantaged communities. The

13

project invested in 25 fixed kindergartens in UB and 100 mobile ger kindergartens all in disadvantaged communities to assist the government in meeting the increasing demand for kindergarten services. There were five PDO level indicator selected to measure achievement of the PDO and all were met or exceed. This section evaluates the outcomes of the project against the results framework that was adjusted in January 2013. This decision is based on the fact that: (i) January 2013 was when the Government of Mongolia (GoM) requested the framework be adjusted to accommodate the reduction in project scope; (ii) the Bank completed the formal restructuring and it was approved the Bank although not countersigned by GoM; and (iii) the indicators were officially adjusted in the Bank’s ISR system and data was collected using the new indicators. Figure 1 provides a schematic of project intervention and more details on project outputs can be found in Annex 2. Figure 1: Schematic of Project Interventions

44. There was considerable progress in the PDO level indicators during the life of the project. With the completion of the project kindergartens, the GER of children aged 2-5 in the urban districts in UB and where new kindergartens were provided increased from 67.9 percent in 2012 to 72.64 percent in 2014/15 exceeding the target of 67.912 percent. The percent of boys increases from 66.8 percent to 71.9 percent and girls from 69.1 percent to 73.3 percent again, exceeding the target. In addition to the project GER data, there have been steady increases in GER nationwide for children ages 2-5. In 2014/15, GER was 80.5 percent (2.9 percent increase over academic year 2013/14). In 2015/16, preliminary data show nationwide GER at 84.8 percent. In 2015/16, the actual enrollment in the 17 project-supported fixed kindergartens was 2,957 (49.3 percent female). This number exceeds the original capacity estimation of approximately 1,700. The demand for kindergarten is extremely high, one project kindergarten reported that all of the parents of the 1,000 children ages 2-5 in the catchment area wanted to enroll their children in kindergarten. However, the kindergarten could only enroll 380 children.

12 At the time of project preparation there was a rapid increase in fertility rates. The rapid increase in the number of new children being born was putting downward pressure on enrollment rates and new construction was needed just to keep pace with the rapid increase in fertility rates. The targets were selected based on these realities.

OBJECTIVE 1st ORDER DRIVERS 2nd ORDRER DRIVERS INTERVENTIONS

Facilities Fixed Kindergarten Construction

Learning Materials Provision of Books and Toys

Facilities Mobile Ger Kindergarten Provision

Learning Materials Provision of Books and Toys

Rural Children

Urban Children

To assist the Government to cope

with the surge in total fertility

rates by providing access to ECE

for children in disadvantaged

communities

14

45. With the provision of the 100 new mobile ger kindergartens, the number of children attending new mobile ger-kindergartens increased from 0 in 2009/10 to 3,781 in 2014/15 more than double the target of 1,500. In addition, the average daily attendance rate of the new mobile ger-kindergartens increased from 0 percent in 2012 to 99.8 percent exceeding the target of 90 percent. This data was gathered while administering the parent satisfaction survey. The survey also showed that 64 percent of parents had not enrolled their children before the project provided the mobile ger-kindergartens. Satisfaction of the parents with the mobile ger service was 91.2 percent exceeding the target of 60 percent. The data to support this finding is based on the project 2014 parent satisfaction survey and is measured by the percentage of parents that rated the services good or above.13 46. Percentage of beneficiaries from disadvantaged communities exceeded the target of 50 percent. There were two categories included in this assessment: (i) 72.5 percent of the parents sending their children to mobile ger-kindergartens were herders (unstable earnings) or unemployed; and (ii) 71 percent of mobile ger-kindergarten sites were 100km+ from the aimag center where the fixed kindergartens are located. Additional information shows that in 2014/15, 2,568 (48.7 percent female) children ages 2-5 that lived 100km+ from the aimag center attended the project mobile ger-kindergartens. 47. In addition, to measuring parent’s satisfaction with the mobile ger-kindergartens, the survey conducted classroom observations which indicated that: (i) 98.7 percent of children were able to recite numbers in order; (ii) 97.2 percent could draw and color; and (iii) 98.4 percent could hold a spoon/pen/pencil and chopsticks. These are substantial achievements among targeted children in the rural areas as a result of the project focus on in the early childhood subsector. 48. The intermediate results indicators selected were: (i) increase state-owned kindergarten facilities by 3.4 percent; (ii) increase mobile alternative teaching classes by 14.1 percent; (iii) increase the number of children enrolled in kindergartens by 4,000; and (iv) direct project beneficiaries (number) of which female (%). The results are as follows:

The state owned kindergartens increased from the 2012 baseline of 722 kindergartens to 777 in 2014/15 for a 7.6 percent increase thereby exceeding the target of 3.4 percent.

Mobile alternative teaching classes decreased from a baseline of 711 to 658 instead of increasing to the target of 811. The number is decreasing because population numbers are decreasing in the rural areas as families migrate to UB and other urban areas and begin to enroll their children in fixed kindergartens. As a result, the overall total of mobile ger- kindergartens decreased, even though the project delivered the 100 mobile ger-kindergartens. If the state kindergarten stock had been constant, the target would have been met. The indicator was phrased in this way to align with the Ministry’s Statistical Yearbook. As indicated, the government is now building between 60-70 kindergartens a year in an attempt to meet the overall demand for fixed kindergarten enrollments.

13 Parent satisfaction is calculated based on the percentage of parents that gave the ger-kindergarten services a score of 3 or above (equal to good or above). The survey covered eight services: (i) toys; (ii) meals; (iii) exercise; (iv) children’s desks and chairs; (v) teaching and learning kit; (vi) furniture; (vii) facilities; and (viii) sanitary conditions. The five point rating scale was: (i) insufficient; (ii) sufficient; (iii) good; (iv) very good; and (v) excellent.

15

The number of kindergarten children increased from the 2012 baseline of 142,065 to 206,636 in 2014/15 exceeding the target of 161,181 by 45,455 children.

The direct project beneficiaries were 6,738 of which 49.2 percent were female surpassing the original target of 3,200 and 47 percent female.

3.3 Efficiency 49. The PAD did not undertake a traditional economic analysis for the project, and no internal rate of return was estimated. The PAD provided an economic justification of the project interventions by citing research on how the project’s investments could be expected to generate significant differences in the quantity and quality of skills, the quality of employment and labor incomes. It also argued that targeted investments in education can break the intergenerational transmission of poverty and reduce inequality. 50. This Implementation Completion Report (ICR) has prepared a cost-benefit analysis (CBA) of the project’s interventions based on the actual quantity and costs of the project as implemented. The expansion of ECE and the provision of learning materials for disadvantaged children are amenable to this type of analysis and were appraised in a CBA detailed in Annex 3. The program’s direct costs during the implementation period have been realized. Post-project direct costs were estimated based on current levels. The CBA allows the ICR to calculate indirect costs and benefits both with the program and without the program. 51. The results of this economic analysis demonstrate that the financial outcomes from the ECE expansion program and learning materials provision for disadvantaged children are significant, with a combined net present value (NPV) of US$20.77 million and an internal rate of return (IRR) of 15.28 percent. The ECE service expansion component has a NPV of US$14.79 million and an IRR of 15.49 percent. The NPV and IRR of learning material provision for disadvantaged children are estimated at US$5.98 million and 12.26 percent, respectively. To assess the validity of these results under different assumptions, a thorough sensitivity analysis was conducted by changing several important assumptions. The results remained robust to changes in virtually all of the critical variables. 52. A financial analysis was included in the PAD in an attempt to estimate recurrent costs generated as a result of this program. This was based on past public expenditure trends and future projections, focusing primarily on the costs incurred as a result of this project for salaried of school teachers’ and the provision of learning materials. Additional analysis of the country’s national expenditures suggests that overall the financial sustainability of project-initiated primary activities has been achieved. This is evident by the mainstreaming of most of the key interventions of the project into MECS’s core activities. 53. Annex 3 also discusses a number of non-financial and long-term development impacts of the project. Although better educational attainment leads to increased private returns, there are a number of socio-economic benefits that are not captured by private financial benefits computed or the IRR and NPV calculations. Annex 3 discusses how exposure to ECE services has been documented to have impacts on improving school outcomes such as lower repetition and dropout rates, higher enrollment rates, better long-term cognitive development, and greater school

16

attainment. In addition, improved educational attainment through expansion of access to ECE and the provision of learning materials for disadvantaged children could support the easing of socio-economic inequality by addressing inequality in opportunities, fostering improved nutritional outcomes and school readiness, and providing other benefits. 54. Despite the implementation delays and the project extension, the economic analysis strongly suggests that the project outputs have and will continue to generate long-term cost benefits. The efficiency of the project is rated as substantial. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 55. The overall outcome rating of the project is moderately satisfactory. The relevance of the project objectives, design and implementation was high. The PDO and design focused on the government’s priorities to increase access to ECE. This was and continues to be relevant given the high birth rates in Mongolia and the government’s policy to lay the groundwork for important school readiness. Although there were challenges during implementation, the MECS and PIU, working with the Bank team, made necessary adjustments to successfully complete the activities and draw lessons for future interventions. Efficacy is rated satisfactory because the PDO was achieved as measured by the achievement of the PDO indicators. Finally, the program was implemented efficiently. Table 1: Project Rating Original Project – 3/6/2012 – 2/28/ 2015 – 99.72 percent disbursement-net project funds1

Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Moderately Satisfactory Substantial Moderately SatisfactoryProject Restructuring– 3/01/2015 – 6/30/2015 – 0.28 percent disbursement-net project fundsProject Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Satisfactory Substantial Satisfactory

Overall Project Ratings – 98.8 percent disbursement of grant as of 11/30/2015

Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating High Moderately Satisfactory Substantial Moderately Satisfactory2

1The net amount for the total project is US$9.88 million. 2 Overall rating = (4)(.9972) +(5)(.0280) = 4.128 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 56. As indicated above, the project was focused on access to ECE in disadvantaged areas in Mongolia. By project closing, the project supported fixed kindergartens in the eight UB disadvantaged districts saw enrollments increase by approximately 4.8 percent in one year. All eight districts reported that demand for ECE is higher than supply. The government will continue to provide 60-70 kindergartens a year to accommodate the overwhelming demand. Enrollments in the 100 mobile ger-kindergartens more than doubled the expected targets. Moreover, approximately 72 percent of the parents that sent their children to the mobile ger-kindergartens were herders that had unstable earnings or were unemployed and 71 percent were approximately

17

100km+ from any fixed kindergartens. The project’s support was instrumental in providing needed access to ECE services for disadvantaged families. (b) Institutional Change/Strengthening Already discussed in the document. (c) Other Unintended Outcomes and Impacts (positive or negative) Not applicable 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not Applicable 4. Assessment of Risk to Development Outcome Rating: Moderate 57. The original project preparation identified the overall risk rating as moderate. This rating was primarily based on risks associated with kindergarten construction. The mitigation measures incorporated in the project, with the inclusion of a third party monitor, operations manual and detailed procurement plans, were appropriate and ultimately responsible for ensuring completion of the project procurement. In addition, the inclusion of a highly experience engineer/architect on all Bank ISMs helped to ensure the quality of construction as well as helped to institute strong fire safety measure for the wooden framed construction. Going forward, the risks identified during preparation should remain moderate. This is partly due to the fact that government is incorporating the Canadian-wood kindergarten construction into the Ministry’s public investment plan. In addition, NEMA will continue to inspect schools and ensure that the: (i) insulation on school water tanks remains in place so the tanks do not freeze during the winter; (ii) schools conduct fire drills at least twice yearly; and (iii) school water tanks remain full and that the water is being recycled to avoid stale water in the tanks. There is a possibility that the supply of mobile ger-kindergartens will be reduced however, the risk is low because of the government’s commitment to providing ECE service over the next five years. Moreover, it is expected that the new IDA project will continue support for the rural and nomadic population. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 58. The project preparation team ensured that the project design was closely aligned with the ESMP2 for 2006-2015, GPE goals of meeting the 2015 MDGs, and the Bank’s 2009-2010 ISN. The design was simple, the PDO was precise and key indicators were appropriate for measuring progress toward achieving the PDO. The design took into consideration the recommendations from peer reviewers and lessons learned from other Bank-supported projects in Mongolia. The preparation team identified the appropriate risks and incorporated design features to mitigate them and included the relevant technical specialists to develop the project. All-in-all, the quality at entry for the ECE project was satisfactory.

18

(b) Quality of Supervision Rating: Satisfactory 59. As previously mentioned, there were six Bank review and implementation support missions which included a MTR. The project team was actively engaged in supporting the government in its efforts to implement the project. Whenever implementation challenges arose, the Bank team worked with government to find appropriate solutions that would not comprise the integrity of the design. The supervision teams consistently reported on FM and procurement progress during supervision missions and worked with the PIU team to build their capacity in these areas. They also systematically documented project progress in aide-memoires, back-to-office reports and ISRs, all of which kept Bank management informed of progress and provided the foundation for the ICR analysis. Key to the project’s achievements was the consistent supervision with a highly experienced engineer/architect that provided support provided to the PIU team on kindergarten construction. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 60. Based on the implementation analysis above, overall Bank performance is rated satisfactory 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 61. At the time of preparation, the government was fully committed to the project. The MECS agreed and worked with the Bank to design a simple, ECE access focused project which would be relevant to the needs of Mongolia. The MECS also provided the project counterpart funds that were used to construct the agreed eight kindergartens. However, the 15 month delay in the approval of the kindergarten designs by the NEMA and Ministry of Construction, within the context of a two year implementation period, created a challenge for implementation. In addition, the MECS replaced most of the experienced PIU staff after the 2012 elections which left the PIU unstaffed for three months. Although the replacement staff were very qualified, they had limited experience in implementing Bank projects and therefore, required substantial support early in their new positions. The government’s benchmark standard pricing for machinery also increased between three to five times under the order of the Ministry of Construction which led to a repricing of kindergarten construction and a reduction in the number kindergartens the project could construct. As discussed above, the price increases were mostly related to exogenous factors however, the project and implementation were impacted. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 62. The PIU was responsible for managing all aspects of implementation. When faced with implementation challenges, the PIU was able to work effectively with the Bank to resolve them as indicated above. The capacity building activities were successful in that FM was constantly rated satisfactory throughout the life of the project. The PIU ensured that an adequate project FM system

19

was in place, which provided accurate and timely information on the utilization of project funds. After the early procurement challenges with design approvals, the project’s procurement performance was rated satisfactory due to fact that procurement files were in proper order and post-reviews revealed that procurement guidelines were followed. The PIU ensured that parent satisfaction survey was conducted, data was analyzed and a report was produced detailing the findings. They provided good detailed project progress reports and continuously updated the data for the results framework. The PIU worked with the CCU to ensure that all the recommendations made related to fire safety of the kindergartens were completed. Based on these aspects, the implementation agency performance is rated satisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderate Satisfactory 63. Based on the implementation analysis above, overall borrower performance is rated moderately satisfactory 6. Lessons Learned 64. Lesson 1. Project designs should be simple when they have short implementation periods. The ECE project implementation period was shorter than the normal three year period for most GPE-supported projects. The preparation team designed a project that was simple and fit into the two year time constraints. Although the project design was simple and the approval process for the new wood framed construction had begun prior to effectiveness, the 15 month delay in approving the kindergarten designs was a key contributor to the implementation delays.

65. Lesson 2. A strong project implementation team must remain in place throughout project implementation. Key positions were replaced shortly after implementation due to changes in the government which also contributed to project implementation delays. These delays along with the short implementation period, contributed to the project implementation challenges. 66. Lesson 3. Surveys included in project designs are valuable tools for assessing impact of project interventions. The parent satisfaction survey was important for understanding the significant role that mobile kindergartens play in preparing children ages 2-5 for primary school. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies Not applicable. (b) Cofinanciers/Donors (c) Other partners and stakeholders Not applicable.

20

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(US$ millions)

Actual/Latest Estimate

(US$ millions)

Percentage of Appraisal

Component 1: Increasing Kindergarten Capacity in Urban Areas

9.31 8.83 95.00

Component 2: Creating preschool classes in rural areas

0.48 0.41 85.00

Component 3: Grant Management and Monitoring and Evaluation

0.21 0.64 304.00

Total Baseline Cost 10.00 9.88 98.80 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00

Total Project Costs 0.00 0.00 Project Preparation Costs 0.00 0.00 0.00 0.00 0.00 0.00

Total Financing Required 10.00 9.881 98.80 (b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(US$ millions)

Actual/Latest Estimate

(US$ millions)

Percentage of Appraisal

Counterpart Funds 2.85 2.682 94.00 EFA-FTI Education Program Development Fund

10.00 9.88 98.80

1 The Government of Mongolia returned US$121,221.02 of unspent funds from the designated account. 2 The tugrik was devalued.

21

Annex 2. Outputs by Component 67. Component 1: Increasing Kindergarten Capacity in Urban and Peri-urban Areas (GPE--US$9,315,000 and Counterpart--US$2,849,000). This component contributed to increasing pre-school enrollment by building new kindergarten facilities, providing furniture, equipment, indoor and outdoor playgrounds and toys, and teaching materials to kindergartens. 68. The indicators used to measure sub-component progress were: (i) increase state-owned kindergarten facilities by 3.4 percent; and (ii) increase number of children enrolled in kindergartens. During the project period, the state-owned kindergarten facilities increased from 722 to 755 or 7.6 percent exceeding the target of 3.4 percent and the children enrolled in kindergartens increased from the 142,065 to 206,636 exceeding the target of 161,181 by 78 percent. 69. Activity 1.1: Building new kindergarten facilities including supervision (GPE-US$7,715,000 and Counterpart-US$2,849,000). The original activity under this sub-component was construction of 37 new kindergartens (21 to be located in aimag centers and 16 in UB in overcrowded urban and rural areas. During the November 2012 restructuring the number was reduced to 25 (17 from GPE funds and 8 from government funds) in response to cost escalation of raw materials. All 25 kindergartens were completed, occupancy certificates were issued and handed over to the school principal. In addition, the 25 Canadian wood framed construction kindergartens, included fully functioning fire sprinkler systems and smoke detectors as recommended by third party technical engineer and the Bank’s team engineer. 70. Activity 1.2: Supply of furniture, equipment, indoor and outdoor playground toys, and teaching materials. (GPE-US$1,600,000 and Counterpart-US$0.00). This activity was to complement activity 1.1 by supplying the needed furniture, equipment, toys and teaching materials for the 25 project supported kindergartens. During the life of the project, all 25 kindergartens were provided with all the need supplies to make them a fully functioning kindergarten which included age appropriate furniture, kitchen equipment, electric appliance, carpet and bedding. In 2014, because the competitive bidding process led to cost savings in the contracts, the MECS, with Bank approval, increased the quantity of goods to cover an additional 35 public kindergartens (12 in UB and 23 rural areas) that had not benefited from any project support.

71. Activity 1.2 also supported the procurement of toys and books for pre-school children. All 25 project schools were supplied with age appropriate toys and books based on the project design. In addition, the project was able to provide 500 public kindergartens nationwide with toys and books. This occurred because of the 2014/2012 depreciation of the local currency vis-à-vis the US$ which resulted in an increase in project funds. The MECS and Bank agreed that the additional project resources would be used for toys and books for kindergartens that: (i) had never been supplied with toys and books which were identified using statistical data of pre-school education; (ii) submitted official letters requesting the materials; and (iii) were recommended by the vice minister, department directors and MECS specialists. The book titles were selected from the list of 100 top preschool books that had been complied during the implementation of the “Improving Primary Education Outcomes for the Most Vulnerable Children in Rural Mongolia” which was jointly supported by Save the Children Japan and the World Bank.

22

72. Component 2: Creating Alternative Preschool Classes in Rural Areas. (GPE- US$475,000 and Counterpart-US$0.00). The intermediate indicator used to measure sub-component progress was to increase mobile alternative teaching classes by 14.1 percent. During the project period mobile alternative teaching classes decreased by 7.6 percent or from 711 to 658. Mobile ger-kindergartens have a shelf life of approximately three years. The government is not replacing the mobile ger-kindergartens because population numbers are decreasing in the rural areas as families migrate to UB and other urban areas and begin to enroll their children in fixed kindergartens. At the beginning of the project, 41.1 percent of the population lived in UB and by project closing 43.4 percent lived in UB, a considerable and significant increase.

73. Activity 2.1: Building New Kindergarten Facilities including Supervision (GPE- US$460,500 and Counterpart-US$0.00). This activity was to provide 100 mobile ger-kindergartens14 to rural areas with large populations of herders’ children and where preschool enrollment rates were particularly low. Each mobile ger-kindergarten consisted of two gers15, desks, chairs, a blackboard, carpet, toys, a stereo player, portable toilet and needed learning materials. This activity built on the 2007-2009 GPE funded project that provided 171 mobile ger-kindergartens. Ger-kindergartens were a UNICEF innovation and, in addition to GPE, have been supported by Japan and the Asian Development Bank (ADB). During implementation, the project procured and distributed 100 mobile ger-kindergartens.

74. Activity 2.2: Develop and Print Visual Aids and Children’s Workbooks in Accordance with the Alternative Pre-school Curriculum. (GPE- US$14,500 and Counterpart-US$0.00). This activity was to develop, print and distribute visual aids and children’s workbooks to mobile ger-kindergartens in accordance with the alternative preschool curriculum that would serve as a primary instruction tool for the kindergartens. The project built on the visual aids and workbooks that had been developed in 2012 with support from the ADB “Early Childhood Education for Rural, Nomadic, and Migrant Children” project. The MECS established a working group to assess the materials that were currently being used and to improve them in accordance with the core or alternative curriculum. The working group studied 100 visual aids from other countries to inform the process of revising the visual aids. Based on this: (i) 53.4 percent of the visual aids were newly developed, (ii) 4.6 percent were improved; and (iii) six new annexes were added to the original visual aids. The workbooks were modified as follows: (i) 108 exercises were assessed; (ii) 16 annexes were added; (iii) 75 new exercises were developed; and (iv) 19 exercises were revised. Overall, approximately 69 percent of the workbook content was improved. In November 2013, the newly revised visual aids and workbooks were printed and distributed to all 100 project supported mobile ger-kindergartens.

14 The project centrally procured and distributed the gers to the relevant soums and baghs. The local education department were responsible for their operation and maintenance. The ger-kindergartens are considered as satellite kindergartens affiliated with fixed kindergartens. The provincial departments of education are asked to deliver services to the herders. The mobile gers are only in operation during the summer months to reach out to the herders. After the summer, the gers can be set up at the fixed kindergartens and used as extra classrooms. 15 One ger is used for teaching and learning and the second ger is used for cooking and sleeping.

23

75. Component 3: Grant Management and Monitoring and Evaluation. (GPE-US$210,000 and Counterpart-US$0.0) This component was to support: (i) the PIU in executing the project and (ii) parent satisfaction surveys after the opening of each fixed kindergarten. As indicated, the project was restructured to conduct a parent satisfaction survey of the 100 mobile ger kindergartens rather than the fixed kindergartens. The project did provide the agreed support and resources for the PIU to conduct their project monitoring and evaluation duties. 76. As mentioned previously, in January 2013 the PIU staffing was changed due to elections and changes in the government. However, by April 2013 the new PIU was in place. The project also conducted a parent satisfaction survey from a representative sample of parents in the 100 mobile ger kindergartens in 14 aimags. The survey questionnaire was encoded into SPSS and the results from the survey showed that (i) 99.8 percent of children attended the new mobile ger kindergartens for more than 5 days a week; (ii) 91.2 percent of parents rated the services of the mobile ger kindergartens as good or above; (iii) 64 percent of parents reported that they sent their children to the mobile ger kindergartens to prepare them for primary school, and 62.6 percent reported that they had not enrolled their children in kindergarten before the project. Moreover, classroom observations indicated that: (i) 98.7 percent were able to recite numbers in order; (ii) 97.2 percent could draw and color, and (iii) 98.4 percent could hold a spoon/pen/pencil and chopsticks.

24

Annex 3. Economic and Financial Analysis Economic Analysis 77. A standard economic analysis of the project involving a calculation of an economic rate of return (ERR) and net present value (NPV) was not undertaken during appraisal. Instead, the economic analysis in the PAD discussed education as a priority investment sector for Mongolia by citing international evidence on high private returns to education in employment and wages, pointing out existing inefficiencies in the system such as the prevalence of dropout and grade repetition, and identifying disparities of education participation among children from disadvantaged households. 78. For the economic analysis of this ICR, two core activities—expansion of early childhood education (ECE) and the provision of learning materials for disadvantaged children—have been selected for a cost-benefit analysis (CBA) using information on the actual quantity and costs of the Project as implemented. For this analysis, the project’s direct costs during implementation have been realized, and post-project direct costs are estimated based on current levels. 79. Evaluation Approach. This cost-benefit analysis (CBA) entails undertaking a projection of education sector indicators (e.g., enrollment, repetition rate, dropout rates, etc.), public expenditure, direct costs to families, opportunity costs of schooling to children, and benefits from education without the program and with the program. The difference in cost and benefits between the with the program scenario and the without the program scenario is attributed to the program. More specifically, current (2012-2015) and projected (2016-2019) single-age population data is used to determine the number of age-appropriate children for enrollment in ECE, primary, and secondary schools. Enrollment, repetition and dropout rates at each level are projected based on current status and reasonable growth rates based on past trends, and projected population size. Based on these projections and information on current families’ and public expenditure per student in primary and secondary school, as well as the estimated potential income of children dropping out from school, the cost to the public and to families of sending children to schools and opportunity cost (to students) of attending different levels of schooling were estimated. Benefits, in the form of private returns to education, are projected based on enrollment and completion of different levels of schooling and returns to education from the Mincer earnings regression in Mongolia. Extensive effort has been made to consider all costs and benefits from the program by accounting not only for direct benefits and costs, but also indirect ones. The baseline result, which is based on realistic assumptions, is presented, and then sensitivity of the results to changes in the critical assumptions is thoroughly tested. Expansion of Early Childhood Education 80. Component one of the project financed the construction of 25 fixed kindergartens, 17 of which were financed by the project. 81. Program Cost. The present value (in 2015 US$) of direct program fixed costs, which include construction, equipment and training costs, during the project period was approximately US$7.31 million. Operating costs, for supplies and teachers salary, during this period are estimated to be US$0.83 million. However, the kindergartens are expected to last at least 25 years and up to

25

40 years, and operation and re-equipping costs post-project are estimated at US$8.63 million. In addition to these direct costs, there are indirect costs to the public and to families and opportunity costs to the children as more children are (and would be) enrolling in primary schools and progressing into lower secondary schools. These indirect costs are estimated at US$42.10 million, during 2015-2040 period. 82. Financial Benefits. The long-term impacts of ECE exposure on schooling and other socio-economic outcomes are neither obvious nor well documented in Mongolia where there are limited studies on long-term impacts of ECE on student performance in primary and secondary schools as well as other long-term life outcomes. There is, however, recent evidence from Mongolia (Rao et al., 2015) on the impact of ECE on cognitive outcomes among children who passed through the kind of ECE program being assessed in this CBA. Rao et al. (2015) conducted a careful evaluation of the impacts of exposure to this ECE program on children’s performance on the East Asia Pacific Early Childhood Development Scale Assessment (EAP-ECDS), and documented that exposure to preschool has a very high impact relative to the control group of children who had no exposure to ECE. This CBA exploits the performance gap in the EAP-ECDS to determine the potential differences in grade repetition rates among children in primary and secondary school with variation in exposure to ECE programs. This analysis also makes the very conservative assumption of allowing the impact of ECE exposure to disappear as children progress from grade 1 to grade 2, from grade 2 to grade 3 and so on. Accordingly, the projection of the gap in school completion, by grade, among these groups of children provided a basis for calculating private returns to education over 25 years after they have completed their respective grades of education, as a result of exposure to early childhood programs. 83. The direct financial benefits from ECE expansion components of the project is computed by estimating the private returns to education for children who have progressed into higher grades in primary and lower secondary school due the their exposure to ECE services. Computing this benefit requires projecting the net-flow ratio (in primary and secondary schools) among children exposed to formal preschool, community-based ECE and home-based ECE, relative to those with no exposure to ECE services, and attributing additional education attainment to the ECE expansion component of the program. 84. The present value of this direct private return to education is estimated at US$10.01 million. In addition to this direct benefit, the expansion of access to ECE services would reduce public and family expenditure on education and opportunity costs of education to children by reducing grade repetition, particularly in the early grades of primary school. The present value of this indirect benefit is estimated at US$4.79 million. The net present value (NPV) of the ECE fixed kindergarten expansion component is estimated at US$14.79 million, and its IRR is 15.49 percent. 85. Sensitivity Analysis. The baseline financial outcomes discussed above are based on current information on relevant variables and adopting realistic assumptions about future trends. To check the robustness of these results, the ICR conducted a meticulous sensitivity analysis on how the estimated financial outcomes might fluctuate due to a change in critical assumptions. The result from this sensitivity analysis is presented is Table 1. Financial outcomes from the program are robust to changes in the critical assumptions: the NPV remained high, ranging from 2008 US$10.71 to US$22.88 million; the IRR ranges from 14.77 to 18.31 percent.

26

Table A3.1: Sensitivity Analysis of Financial Outcomes of ECE Expansion

Changing Scenarios New value

New result Base result Change** Baseline

value IRR NPV* IRR NPV IRR NPV Family expenditure per student per year for primary and secondary school, respectively

US$35 & US$130

US$50 & US$140

16.90% 17.53 15.48% 14.79 1.41% 2.74 Family expenditure per student per year for primary and secondary school, respectively

US$55 & US$150

US$50 & US$140 15.03% 13.84 15.48% 14.79 -0.45% (0.95)

Income of a person with no education US$189 US$164 18.31% 22.88 15.48% 14.79 2.83% 8.09 Income of a person with no education US$198 US$164 14.52% 12.09 15.48% 14.79 -0.96% (2.70) Returns to Primary and secondary education, respectively 8% & 8% 7% & 7% 16.23% 16.92 15.48% 14.79 0.74% 2.13 Returns to Primary and secondary education, respectively 6% & 6% 7% & 7% 14.77% 12.78 15.48% 14.79 -0.71% (2.01) Discount rate 11% 10% 15.48% 10.71 15.48% 14.79 0.00% (4.08) Discount rate 9% 10% 15.48% 19.82 15.48% 14.79 0.00% 5.03 Income forgone by a student in primary and secondary school per year, respectively

US$30 & US$40

US$39 & US$45 16.31% 16.43 15.48% 14.79 0.83% 1.64

Impact of ECE on secondary school outcome Excluded Included 15.43% 14.07 15.48% 14.79 -0.05% (0.72) * NPV is calculated in terms of 2008 US$ million **Change is baseline value of IRR and NPV minus their respective new values with the changing assumptions.

27

The Provision of Learning Materials for Disadvantaged Children 86. Program Cost. The costs associated with this component are broadly classified into direct and indirect costs. The direct cost includes expenditures from the project budget to make learning materials available in classroom libraries, and include expenses on developing children books, learning material purchase and shelves. The direct cost is calculated based on actual unit costs, with proper inflation adjustment. During the project’s life (2012-2015), the present value of the total direct cost is estimated at US$1.63 million. 87. The indirect costs include those incurred (by the public, families and the children) due to that fact that more students would be attending higher grades in primary school due to the program. Estimation of this indirect cost is done based on current per children government spending on education, families expense and opportunity cost of attending school for student. These costs are estimated to be US$1.11 million during the 2015-2022 period.

88. Financial Benefits. Among the many socio-economic benefits of higher years of education that is amenable for quantification and hence can easily be assigned a financial value is private return from education. The benefit considered from the program considered in this CBA is this private return for students as a result of extra-years of education associated with accessing learning materials. The present value of the private return from higher educational attainment, resulting from access to high quality learning materials, is estimated at US$5.98 million. NPV and EIRR of the learning material provision scheme are US$0.95 million and 12.26 percent, respectively (see Table 1).

Table 1: Financial Outcomes from Classroom Libraries: Cost, Benefits, EIRR and NPV

Internal Rate Return and Net Present Value

IRI 12.26%

NPV (US$ million) 0.95

PV of cost (US$ million) 5.04

Direct cost (2014-2017) 3.92

Library Equipment 1.82

Book purchase 1.35

Book development 0.39

Overhead cost 0.36

Indirect cost to the public, students & their families (2014-2022) 1.11

Public expenditure on primary 0.64

Opportunity cost to students in primary 0.17

Family expenditure on primary 0.30

PV of benefits (US$ million) 5.98

Private returns 5.98 89. Sensitivity analysis. The estimated baseline financial returns are based on actual data on school outcomes and reasonable assumptions about the impact of the program, indirect costs, and benefits of the program. To assess the robustness of the results to changes in significant assumptions, a careful sensitivity analysis has been conducted. Table 2 presents the result of the

28

sensitivity analysis, which shows the result is robust to almost all of the changes in assumptions. Except in case of the higher discount rate and lower returns to education, the NPV remained positive with IRR above 10 percent.

29

Table 2: Sensitivity Analysis of Financial outcomes from Learning Materials

Changing Scenarios New value Base value New result Base result Change

IRR NPV IRR NPV IRR NPV

Family expenditure per student in primary school, US$ per year 55 45 12.16% 0.91 12.26%

0.95

(0.00)

(0.04)

Family expenditure per student in primary school, US$ per year 35 45 12.37% 0.98 12.26%

0.95

0.00

0.04

Income of a person with no education, US$ per year 189

164 12.06% 0.87 12.26%

0.95

(0.00)

(0.08)

Income of a person with no education, US$ per year 139

164 12.58% 1.06 12.26%

0.95

0.00

0.11

Annual growth of income for a person with no education, % 2.0% 1.5% 12.89% 1.25 12.26%

0.95

0.01

0.30

Annual growth of income for a person with no education, % 1.0% 1.5% 11.64% 0.66 12.26%

0.95

(0.01)

(0.28)

Returns to Primary education, % 8.2% 7.2% 12.94% 1.24 12.26%

0.95

0.01

0.29

Returns to Primary education, % 6.2% 7.2% 11.62% 0.67 12.26%

0.95

(0.01)

(0.27)

Discount rate, % 11% 10% 12.26% 0.49 12.26%

0.95

-

(0.46)

Discount rate, % 9% 10% 12.26% 1.47 12.26%

0.95

-

0.53

Government expenditure on primary school per student, US$ per year 189 169 12.06%

0.87 12.26%

0.95

(0.00)

(0.08)

Government expenditure on primary school per student, US$ per year 149 169 12.47%

1.02 12.26%

0.95

0.00

0.08

Income forgone by a student in primary school, US$ per year 90 80 12.16% 0.91 12.26%

0.95

(0.00)

(0.04)

Income forgone by a student in primary school, US$ per year 70 80 12.37% 0.98 12.26%

0.95

0.00

0.04

Decrease in primary school grade repetition rate, % 19.3% 17.5% 13.35% 1.45 12.26%

0.95

0.01

0.50

Decrease in primary school grade repetition rate, % 15.8% 17.5% 11.17% 0.47 12.26%

0.95

(0.01)

(0.47)

"Pessimistic Scenario": combination of Scenario No.: 1.1, 2.2,3.2, 4.2,

5.1, 6.1, 7.1 & 8.2 Base value

for all 7.68% (1.19) 12.26%

0.95

(0.05)

(2.13)

"Optimistic Scenario" : combination of Scenario No.: 1.2, 2.1, 3.1, 4.1,

5.2, 6.2, 7.2 & 8.1 Base value

for all 17.61% 4.20 12.26%

0.95

0.05

3.25

30

Non-financial Benefits and Long-Term Development Impact 90. Although better educational attainment leads to increased private returns, there are a number of socio-economic benefits that are not captured by private financial benefits presented in the form of the IRR and NPV from these two activities. To address such limitations, this Annex presents some non-financial benefits that could have been generated as a result of the program implementation. Exposure to ECE services has been documented to have impacts on improving school outcomes such as lower repetition and dropout rates, higher enrollment rates, better cognitive development, and higher school attainment. In addition, improved educational attainment through the expansion of access to ECE and the provision of learning materials could result in a number of non-quantified socio-economic benefits such as easing socio-economic inequality by addressing inequality in opportunities, improved nutritional outcomes, school readiness, and other benefits. 91. In addition, ECE and the provision of learning materials for disadvantaged children are likely to have long-term development impacts such as addressing inequality by offsetting socio-economic disadvantages, preparing children for a brighter future, and reducing the opportunity costs of attending primary and secondary school for children in families from lower socio-economic strata. ECE lays the foundation for a life of expanded opportunities, and it serves as spring board for success in primary school and reduces language-based disadvantages. UNESCO (2010) documents that being born into the poorest 20 percent of households in Mongolia significantly increases the incidence of ‘education poverty’. Such a gap in education outcomes is the result of a combination of factors like poverty, geographic isolation and cultural discrimination, particularly against women and minorities (ibid). In addition, despite the elimination of fees and charges, the most commonly cited reason for not attending school is high cost. Higher expenses on education supplies and the opportunity cost of attending school are likely to reduce school attendance among disadvantaged children. Financial Analysis 92. The cost design of the project was based on a financial analysis conducted as part of due diligence during appraisal. The analysis identified two major recurrent expenditures generated by the project: (a) additional teacher and staff salaries at newly constructed classrooms to accommodate an expansion in student enrollment; and (b) incremental operating costs and maintenance from additional classrooms. A sustainability analysis of these total incremental costs was conducted to assess the government’s ability to absorb these costs in the budget once program funding came to an end. While the estimates indicated that the annual increments appeared affordable within the current Medium Term Expenditure Framework, the PAD noted that the Bank and development partners would support MECS to engage in close dialogue with the Ministry of Finance to ensure that sufficient resources would be made available to cover longer term recurrent cost implications of the implementation of the project. 93. Recent Trends in the Education Budget. The PAD described increasing government spending in education as a share of GDP, as well as the increasing share of the education budget as a share of total government spending. The PAD estimated that education budget trend was generally upward and would be maintained at the 2006 level.

31

94. The PAD noted that “the low population density of rural Mongolia explains the high costs of education provision on a per-pupil basis as well as the need for public investment in rural education.” This trend continues. The cost of education in Mongolia is still high due to the sparse population and wide geographical spread. The government has spent more than 20 percent of its budget and over 6 percent of its GDP on education since 2006. 95. The PAD also provided trends and projections of education finance and enrollment, which predicted a steady increase in both. The trend of public spending per child from 2006 to 2014 shows consistent increases, even taking into account the rising enrollment and the fluctuation of the exchange rates. In particular, during the recession of 2008 and 2009, there was no budget cut on education. On the contrary, the government provided free textbooks to children in lower grades to ensure that they would not drop out due to financial reasons. 96. The PAD noted an increasing trend of public education spending since the mid-1990s. Since 1998, per student financing has been in place in an attempt to ensure an equitable allocation of resources across all schools. The MECS receives 23 percent of the recurrent budget, a major increase from about 17.5 percent in 2007, and is by far the largest ministry. By a law established in 2002, education is to receive at least 20 percent of the budget. The wage bill accounts for about 69 percent of MECS recurrent expenditures and the Ministry’s share of the investment budget has ranged between 5 percent and 10 percent in recent years.

32

Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/

Specialty Lending/Grant Preparation Harold H. Alderman Consultant GSPDR Education Tungalag Chuluun Operations Officer GSPDR Operations

Badamchimeg Dondog Financial Management Analyst GGODR Financial Management

Feng Ji Senior Environmental Specialist GENDR Safeguards Prateek Tandon Senior Economist GEDDR Education Specialist Sabrina Gail Terry Program Assistant GHNDR Administrative Gerelgua Tserendagva Procurement Specialist GGODR Procurement

Kin Bing Wu Lead Education Specialist EASHE Team Leader

Jun Zeng Senior Social Development Spec GSURR Safeguards Supervision/ICR Prateek Tandon Senior Economist GEDDR Team Leader Rajiv Aggawal Consultant GEDDR Architect Rabia Ali Economist GEDDR Education Specialist Dulguun Byambatsoo Consultant GGODR Operations Tungalag Chuluun Operations Officer GEDDR Operations Anna Coronado Program Assistant GEDDR Administration Batzul Dashdorj Consultant GGODR Operations

Badamchimeg Dondog Financial Management Analyst GGODR Financial Management

Ulziimaa Erdene Consultant EACMF Financial Management

Pagma Genden Consultant GHNDR Operations Jianjun Guo Senior Procurement Specialist GGODR Procurement Takiko Igarashi Jr. Professional Officer GEDDR Operations Atsuko Muroga Consultant GEDDR Education Evelyn Villatoro Senior Procurement Specialist OPSOR Procurement Sandra Beemer Consultant GEDDR ICR Author (b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks US$ Thousands (including travel and consultant costs)

Lending

Total: 50.06 300.97 Supervision/ICR

Total: 85.74 374.87

33

Annex 5. Beneficiary Survey Results Not Applicable

34

Annex 6. Stakeholder Workshop Report and Results Not Applicable

35

Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR 97. The Global Partnership for Education Early Childhood Education Project between the Ministry of Education, Culture and Science (MECS) and the World Bank was signed on March 6, 2012. The project’s overall implementation was coordinated by MECS with the support from the Project Implementation Unit (PIU) comprised of the Project Deputy Director, Monitoring and Evaluation Specialist, Financial Specialist, Procurement Specialist and Construction Consultant. The Grant amount for this project is US$10.0 million targeting 14 aimags and 100 isolated soums and 8 districts with high population density. 98. The objectives of this project were to help the government cope with access to preschool education with the surge in fertility rates; provide access to early childhood education (ECE) for children in disadvantaged communities; and reach and educate children in remote and poverty stricken areas to ensure school readiness. The project also supports construction of new kindergarten facilities in districts with high population density; and provision of mobile ger-kindergartens in rural areas for herder family children in isolated soums 99. The expected key performance indicators were to: (i) increase GER of children ages 3-5 in the urban districts in UB where new kindergartens were built; (ii) increase state-owned kindergarten facilities by 3.4 percent; (iii) increase mobile alternative teaching classes by 14.1 percent; (iv) increase the percentage of beneficiaries from disadvantaged communities; and (v) conduct parental satisfaction survey on the mobile ger-kindergaten services. Component 1: Increasing Kindergarten Capacity in Urban Areas by Building New Kindergarten Facilities and Supplying with Furniture, Equipment and Toys. 100. The project had originally planned to build a total of 37 kindergartens, each with a capacity of 100 children in 21 aimag centers and in 16 districts of Ulaanbaatar. In 2012, the increase in price of gasoline, construction raw materials, and transportation cost resulted in the reduction of 37 kindergartens to 27 (17 from GPE funds and 8 from government funds). 101. The 17 kindergartens supported by GPE funds were completed and supervision was handed over to the principals of the schools. As of August 2015, the total enrollment in 15 khoroos in 8 districts in Ulaanbaatar was 2,813 children ranging from 33-55 students per classroom in some districts. 102. Preparation Work for Building New Kindergarten Facilities. The Construction Client Unit (CCU) of MECS, in collaboration with the Project Implementation Unit (PIU) supervised the progress of construction work. CCU had obtained permission from Ministry of Road, Transportation, Construction and Urban Development (MRTCUD) in December 2012 to build a two story Canadian wood frame kindergartens. According to the World Bank, a third party technical engineer tested the fire sprinkler system of all the 17 Canadian wood framed kindergartens. Fire safety was approved by National Emergency Management Agency in January 2013.

36

103. In order to provide proper fire safety, additional work was done by contactors to ensure a fully functioning fire sprinkler system and smoke detectors by: (i) moving sprinkler system pipes that were above the floor-to-ceiling down to avoid freezing during the winter; (ii) ensuring that they systems had a separate electrical circuit; (iii) installing fire extinguishers; (iv) ensuring waster is circulated from storage tanks to avoid being stagnant, (v) install emergency lights in fire exits with drawing of fire exit schemes; (vi) training school principals and staff on how to run and maintain the fire system; and (vii) marking fire zones on the fire panels. 104. The project stakeholders also identified the Canadian wood framed technology inclusive to the project description. Though this technology was best practice based from other projects previously implemented in Mongolia, the World Bank recommended to develop new technical specifics for each kindergarten due to the local differences in design of fire safety. Six engineering consulting firms were selected to do the design of the 17 wood framed kindergartens. One consulting firm was responsible for the design of 10 kindergartens, two had designed two kindergartens each and the remaining three were designed one kindergarten each. MES funded their services through the Government budget. 105. Additional construction work had to be done, the contractors submitted a letter to MECS requesting additional funds for: (i) using concrete blocks as foundation for new constructions; and (ii) changing the trusses in the building construction to reduce movement. Engineers and MECS approved the additional funding given that the required changes were instructed by the CCU and PIU engineers in preparation for any unforeseen conditions. The Bank has given a no-objection on April 28, 2015 to proceed. Lessons Learned

106. With the support from the World Bank GPE-ECE project, the construction of 17 Canadian wood framed kindergartens in high population density districts was implemented successfully. Coordination with MECS, CCU, construction contractors and state organizations was also excellent. 107. There were however some common issues in the selection of construction companies, it was observed that the contracted kindergarten construction companies did not have their own engineer and technical staff, machinery and equipment, and financial capacity for construction work, all of which were required in the selection. Also, permissions were not obtained to begin construction, drawings were not approved and not done by experts, and lands were not free for construction. These issues caused delays in construction. Component 2: Creating Alternative Pre-school Classes in Rural Areas by Supplying Mobile ger-kindergartens with Furniture, Equipment and Learning Materials 108. Enrollment of Herder Family’s Children in Preschool Education by Supplying 100 Mobile ger-kindergartens. The national competitive bidding for supplying mobile ger-kindergartens with furniture, equipment, and learning materials to 100 remote soums was announced in 2013. Once contracts were awarded, distribution of mobile ger-kindergartens was held between July and November through aimag Education departments. Each mobile ger-

37

kindergarten consisted of two gers, desks, chairs, carpet, toys, refrigerator, solar generator kitchen equipment, and other learning materials. 109. Each year, over 3,000 children of herder families benefit from mobile ger-kindergartens. ECE access was provided, and GER increased in preschool education. 110. Develop and Print of Visual Aids and Children’s Workbooks in Accordance with the Alternative Pre-school Curriculum. A visual aid developed by MECS and a workbook for children developed by "Early Childhood Education for Rural, Nomadic, and Migrant Children", a project funded by the JFPR/ADB in 2012, needed modification according to the alternative preschool curriculum. 111. A working group established by MECS, led by relevant ministry specialists, was formed to assess current preschool education learning materials and to study the possibility of using in core or alternative curriculum to improve visual aid and workbook for children. Kindergartens teachers and ECD specialist had an open discussion on ways to improve these materials – looking for good practices from foreign countries. The discussion outcomes were:

Visual aids improved by studying and learning from 100 visual aids from other countries - 53.4 percent has been newly developed and 4.9 percent improved and 6 annexes has been added.

Workbooks improved by 69 percent - 108 exercises has been assessed, 16 annexes has been added, 75 exercises has been developed, and 19 exercises has been revised.

Both workbook and visual aids were decorated and printed in high quality materials and distribution to 100 mobile ger-kindergartens was completed in November 2013.

Component 3. Grant Management and Monitoring & Evaluation 112. Monitoring & Evaluation on Mobile ger-kindergarten Activities. The GPE-ECE project conducted field trips from February to April 2015 to monitor and evaluate activities, quality and usage of furniture, equipment, toys and learning materials of 100 mobile ger-kindergartens. The results of these evaluations, written in collaboration with Ministry officials, were used to produce Project Completion Reports for the aimags and soums. 113. Parental Satisfaction Survey. A Parental Satisfaction Survey was performed to collect feedback from parents of children attending the 25 mobile ger-kindergartens located in Bayan-Ulgii, Bulgan, Umnugobi, Khentiiaimags and Bayanzurkh, Songinokhairkhan districts of Ulaanbaatar. The sampling, survey design, and reporting methodologies were recognized and approved by World Bank and Ministry of Education and Science.

114. The main questionnaire of the survey seeks to capture the effects of the alternative pre-school program delivery and in particular, it’s impact on the children. The results from the survey was used to monitor and study the effectiveness of mobile ger-kindergartens in children especially the ones coming from disadvantaged communities, the survey focused on physical well-being and motor development; social and emotional development; approach to learning/language development; and cognitive development.

38

115. The survey performed was a pilot of the Parental satisfaction survey data collection which was conducted in two phases – fall 2013 and spring 2014. Beginning of October 2013, parents of children in 6 mobile ger-kindergartens located in Bayanzurkh and Songinokhairkhan districts of UB were selected to participate in the survey. And in March and April 2014, the rest of 19 mobile ger-kindergartens located in isolated rural areas of Bayan-Ulgii, Bulgan, Umnugobi and Khentiiaimag went through the data collection. In total, 517 parents represented the results of the data collected. Mobile ger-kindergarten classroom teacher facilitated the questionnaire. 116. The results of the parental satisfaction survey on the mobile ger-kindergarten services was relatively high and positive, especially on meals provided, children’s furniture, teaching and learning kits and the overall kindergarten facilities (see more detailed key findings from the Survey finding report). 117. Parental Satisfaction Survey for 100 Mobile ger-kindergartens. After completion of the pilot survey, the World bank and PIU decided to conduct parental satisfaction survey to 100 mobile ger-kindergartens under GPE-ECE Project representing approximately 2000 parents in 14 aimags. Once completed, the survey data covering the period of October 2104 to February 2015 was submitted to SPSS program for evaluation. In March 2015, a report on the survey findings was then submitted by SPSS to the World Bank and MECS both in English and Mongolian language respectively. 118. Key findings of the survey:

Percentage of average daily attendance in the new mobile ger-kindergartens: 99.8 percent attended more than five days a week.

Percentage of Parental satisfaction with the ECE services in the new mobile ger-kindergartens: 91.2 percent of the survey rated “good” or above average to all of the service items provided at the mobile ger-kindergartens.

Percentage of beneficiaries from disadvantaged communities: By employment status, 64.5 percent of parents are herders (unstable earnings) and 8.0 percent are unemployed. By household type, 8 percent from female-headed households. By accessibility, 71 percent of the project sites are located more than 100km from the aimag centers (where fix kindergartens are located).

119. Furthermore, 64 percent of the parents reported that their children were not enrolled in kindergarten before the project. In conclusion, all target beneficiaries were reached under the component of creating alternative preschool classes in rural areas. More Activities Implemented by Currency Depreciation Resulting in Project Savings 120. Supply Toys for 500 Public Kindergartens. The recent depreciation of the local currency resulted in project saving and the project’s ability to support more activities to help improve the quality of ECE in Mongolia. The Ministry proposed to use US$970,000 of project savings on innovative toys. 80 percent of the state owned kindergartens toys were already worn-out, and 30

39

percent of the kindergartens have insufficient toys. The Ministry decided to spend some of the project savings for procuring toys for 500 public kindergartens. 121. The procurement of toys for 500 public kindergartens went through an international competitive bidding, contracts were awarded to the selected companies. During the time the contracts were awarded, MECS decided to increase the amount of each contract by 30 percent of the original price. Suppliers successfully distributed high quality toys in a timely manner to the 500 public kindergartens through aimag Education department at the end of July, 2015. 122. Supply Books for 500 Public Kindergartens. In addition to toys, MECS proposed to use US$160,000 of the remaining funds on books for school readiness. There were 100 top books listed for preschool education, 34 of those books were selected by a working group comprised of professional experts and MECS specialists under the framework for "Improving Primary Education Outcomes for the Most Vulnerable Children in Rural Mongolia" – a project funded by the Save the Children Japan and World Bank. 123. Contracts were made to 6 companies and books were distributed to 500 public kindergartens. 124. Consultant Services. In addition to toys and books, MECS proposed to use US$253,066 on consultancy services to evaluate methodology for child development and achievement, teaching materials, preschool teaching environment standard, mapping of primary and secondary school locations and organizations, and preschool, primary and secondary school teachers ‘workload, need and retaining period. The consultancy services will also provide MECS an evaluation of core curriculums for preschool education, and implementation of preschool education reform.

Policy Recommendations 125. Despite the noticeable results in improvement of quality, quantity, and enhancement of the alternative ECE during the last four years, there remains a need for mobile ger-kindergartens for the following reasons: rural nomadic herders comprise 40 percent of total households nationwide; no increase nor substantial decrease of poverty rate; prominent migration continues from rural areas to populated areas; increase in birth rate means rising number of pre-school children; and increase in the demand for ECE. 126. Findings from the parental satisfaction survey resulted in the following policy recommendations:

Budget

Workshop and seminars on skills and capacity building for heads of kindergartens, with recommendation from ECE managers, is needed to provide the proper support to operate the mobile ger-kindergartens. Skills such as planning permanent and alternating costs for the mobile ger-kindergartens and performance analysis need to be upgraded.

40

According to the new Law on Budget, local authorities now control the budget. Local managers should seek funding and investment support for ECE from the local development fund.

Results on the parental survey results showed that kindergartens are needed in local areas. Annual operating cost of these kindergartens should be included in the local budget allocation plans.

Equal Supply of Equipment in Mobile ger-kindergarten in All aimags

Mobile ger-kindergartens under the project are equipped with furniture, carpets, toys, learning materials, workbooks, household items, solar energy generators and other items to meet the children’s developmental and safety needs. It is required to provide similar equipment to other aimags. The budget allocation and lists of items to be supplied should be standard for all mobile ger-kindergarten investments to create a positive environment among all alternative educational programs.

Similarly, children’s workbooks and visual aids should be re-used and re-printed for alternative ECEs in other local areas. These items should be distributed to libraries of teacher training schools for students use.

Summary 127. Mobile ger-kindergartens can operate in the summer for 2-3 months. However, there are social issues on alternative education program teachers that need to be resolved. Bonuses and/or remuneration, in addition to their salary should be offered in order to compensate with the difficult work conditions. Also, additional funds should be provided for transportation cost, which currently is insufficient. A teacher, an assistant teacher and a cook; a guard/fire keeper is necessary to run an effective and efficient mobile ger-kindergartens in the future. 128. The children’s workbook and visual aids developed under the project can be effectively used by all kindergarten teachers and parents as guide for alternative educational programs.

41

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable

42

Annex 9. List of Supporting Documents Aides-memoire and Implementation Status Reports 2012 – 2015 Implementation Completion Report, Borrower Portion, Mongolia Global Partnership for Education Early Childhood Education Project, 2015. Interim Strategy Note 2009-2010, Mongolia, World Bank, Washington, D.C., 2009. Parental Satisfaction Survey, Mongolia Global Partnership for Education Early Childhood Education Project, 2015. Project Appraisal Document, Mongolia Global Partnership for Education Early Childhood Education Project, World Bank, 2012 Project Paper for a Proposed Restructuring, Mongolia Global Partnership for Education Early Childhood Education Project, World Bank, 2013. “On the Road to Education for All: Progress and Challenges, Education for All Global Monitoring Report.” UNESCO, Paris, France. 2010 Validation, Finalization and Adoption of the East Asia-Pacific Early Child Development Scales (EAP-ECDS) - Rao, Nirmala, Jin Sun, Marie Ng, Yvonne Becher, Diana Lee, Patrick Ip, John, Bacon-Shone - UNICEF, New York, New York. 2015 World Bank Country Partnership Strategy, Mongolia, 2013-2017

43

MAP