Document of The World Bank · Comments (incl. % achievement) Following KCA's audit by ILAC in September 2012 several gap areas were identified and addressed. ILAC then noted that

Document of The World Bank

Report No: ICR00002731

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H2470, TF-90641)

ON A

DEVELOPMENT GRANT

IN THE AMOUNT OF SDR3.5 MILLION (US$5.0 MILLION EQUIVALENT)

TO THE

KYRGYZ REPUBLIC

FOR A

REDUCING TECHNICAL BARRIERS FOR

ENTREPRENEURSHIP AND TRADE PROJECT

September 30, 2013

Private and Financial Sectors Development Unit Central Asia Country Unit Europe and Central Asia

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS (Exchange Rate Effective March 31, 2013)

Currency Unit = Kyrgyz Som (KGS) KGS47.2700 = US$1

US$1.500 = SDR1

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

CIS Commonwealth of Independent States CAS Country Assistance Strategy CSM Center for Standards and Metrology (previously NISM) DB Doing Business indicators by the World Bank EU European Union GDP Gross Domestic Product GTZ/GIZ German Technical Cooperation Agency (renamed GIZ) HACCP Hazard Analysis and Critical Control Point (food safety) IDA International Development Association ILAC International Laboratory Accreditation Cooperation ISO International Organization for Standardization ITC International Trade Center KAC Kyrgyz Accreditation Center LFTR Law on the Fundamentals of Regulation MITT Ministry of Industry, Trade and Tourism MoER Ministry of Economic Regulation (previously MITT) MSTQ Metrology, Standards, Testing and Quality M&E Monitoring and Evaluation NISM National Institute of Standards and Metrology (renamed CSM) OIML International Organization of Legal Metrology PAD Project Appraisal Document PDO Project Development Objectives PIU Project Implementation Unit PHRD (Japan) Policy and Human Resources Development (Grant) PTB German Metrology Institute R&D Research and Development RTBET Reducing Technical Barriers for Entrepreneurship and Trade Project SECO Swiss Economic Cooperation Office TBT Technical Barrier to Trade TR Technical Regulations TurKak Turkish Accreditation Body USAID US Agency for International Development WTO World Trade Organization

Vice President: Laura Tuck

Country Director: Saroj Kumar Jha

Sector Director Gerardo Corrochano

Sector Manager: Paloma Anos Casero

Project Team Leader: Sylvie K. Bossoutrot

ICR Team Leader: Tatiana Segal

iii

KYRGYZ REPUBLIC Reducing Technical Barriers for

Entrepreneurship and Trade

Contents Data Sheet .......................................................................................................................... iv

A. Basic Information ...................................................................................................... iv B. Key Dates .................................................................................................................. iv C. Ratings Summary ...................................................................................................... iv D. Sector and Theme Codes ........................................................................................... v E. Bank Staff ................................................................................................................... v F. Results Framework Analysis ..................................................................................... vi G. Ratings of Project Performance in ISRs ................................................................... ix H. Restructuring (if any) ................................................................................................ ix I. Disbursement Profile .................................................................................................. x

1. Project Context, Development Objectives and Design ................................................... 1

2. Key Factors Affecting Implementation and Outcomes .............................................. 7

3. Assessment of Outcomes .......................................................................................... 15

4. Assessment of Risk to Development Outcome ......................................................... 27

5. Assessment of Bank and Borrower Performance ..................................................... 28

6. Lessons Learned ........................................................................................................ 30

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........... 31

Annex 1. Project Costs and Financing .......................................................................... 32 Annex 2. Outputs by Component ................................................................................. 34 Annex 3. Economic and Financial Analysis ................................................................. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 41

Lending ...................................................................... Error! Bookmark not defined.

Annex 5. Beneficiary Survey Results ........................................................................... 43 Annex 6. Stakeholder Workshop Report and Results ................................................... 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 49 Annex 9. List of Supporting Documents ...................................................................... 50

iv

Data Sheet

A. Basic Information

Country: Kyrgyz Republic Project Name:

Reducing Technical Barriers for Entrepreneurship and Trade

Project ID: P087811 L/C/TF Number(s): IDA-H2470,TF-90641

ICR Date: 09/30/2013 ICR Type: Core ICR

Lending Instrument: TAL Borrower: KYRGYZ REPUBLIC

Original Total Commitment:

XDR 3.50M Disbursed Amount: XDR 3.04M

Revised Amount: XDR 3.04M

Environmental Category: C

Implementing Agencies: Ministry of Economic Regulation Kyrgyz Accreditation Center Center for Standardization and Metrology Cofinanciers and Other External Partners: JICA B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 07/19/2004 Effectiveness: 01/16/2007 01/16/2007

Appraisal: 12/15/2005 Restructuring(s):

10/08/2007 12/03/2008 05/03/2010 04/15/2011 09/04/2012 09/09/2012

Approval: 07/13/2006 Mid-term Review: 12/18/2009

Closing: 04/30/2011 03/31/2013 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Substantial

v

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall Bank Performance:

Moderately Satisfactory Overall Borrower Performance:

Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments (if

any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Agro-industry, marketing, and trade 7 7

Central government administration 33 33

General industry and trade sector 12 12

Other industry 48 48

Theme Code (as % of total Bank financing)

Micro, Small and Medium Enterprise support 25 25

Regulation and competition policy 25 25

Trade facilitation and market access 50 50 E. Bank Staff

Positions At ICR At Approval

Vice President: Laura Tuck Shigeo Katsu

Country Director: Saroj Kumar Jha Annette Dixon

Sector Manager: Paloma Anos Casero Gerardo M. Corrochano

Project Team Leader: Sylvie K. Bossoutrot Sylvie K. Bossoutrot

vi

ICR Team Leader: Tatiana Segal

ICR Primary Author: Manuel Lasaga F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)The Project development objectives were to: i) streamline the national technical regulation and standards framework for business; ii) develop systems to enhance product quality and safety; and iii) increase enterprise competitiveness in pilot sectors. Revised Project Development Objectives (as approved by original approving authority) Not Applicable (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Decrease in cost of regulatory compliance

Value quantitative or Qualitative)

DB and BEEPS reports highlighted high cost of regulatory compliance (with the 2005 BEEPS report suggesting a significant deterioration in the number and length of inspections)

Decrease in the cost of regulatory compliance as reported by businesses and improvement of BEEPS and CODB ratings

N/A

Based on intermediate outcomes, the project has streamlined national technical regulations for business to a high extent, developed systems to enhance quality to a substantial extent, and increased enterprise competitiveness in pilot sectors to a modest extent.

Date achieved 01/16/2007 04/11/2011 03/31/2013 03/31/2013

Comments (incl. % achievement)

DB and BEEPS indicators shown to be marginally relevant to achievement of PDOs (see section 3.2) Achievement based on intermediate outcome indicators considered substantial

vii

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Technical Regulations developed and adopted

Value (quantitative or Qualitative)

No Technical Regulations

8 new Technical Regulations in key sectors of the economy

N/A 35

Date achieved 01/16/2007 04/11/2011 03/31/2013 03/31/2013 Comments (incl. % achievement)

437% achievement

Indicator 2 : Reduction in number of products subject to mandatory certification


5500

Further gradual decrease in number of products subject to mandatory certification

N/A 684


100% achievement. The Government is currently considering a further reduction to 673 categories of goods pending approval of additional Technical Regulations (TRs).

Indicator 3 : NISM as a whole certified for Quality Management System (QMS) Value (quantitative or Qualitative)

No Yes N/A Yes


100% achievement

Indicator 4 : Increase in the percentage of national documentary standards harmonized with international standards


38% 46% N/A 46%


100% achievement. The increase in standards harmonization was made possible in part through the purchase on 96 ISO standards during the life of the project.

Indicator 5 : Increase in the number of measurement and calibration services with lower level of associated uncertainties.


500 600 N/A 800

Date achieved 01/16/2007 04/11/2011 03/31/2013 03/31/2013

viii


133% achievement. Several state of the art labs installed under the project increased CSM's ability to provide calibration and metrology services in ranges and accuracy unavailable before the project.

Indicator 6 : Accreditation body compliant with ISO/IEC 17011


Accreditation function carried out by NISM under the old GostStandard system

Accreditation Center compliant with ISO/IEC 17011 opening the road for mutual recognition thereby allowing products and services tested once to be accepted everywhere

N/A No

Date achieved 01/16/2007 04/11/2011 03/31/2013 03/31/2013


Following KCA's audit by ILAC in September 2012 several gap areas were identified and addressed. ILAC then noted that only 2 technical issues remain of concern and requested that they be addressed by its next meeting in October. KCA is currently working on these issues.

Indicator 7 : At least two testing laboratories accredited according to international standards ISO/IEC 17025


0 2 N/A No


0% achievement. Delays in procurement and installation of microbiology testing equipment prevented the full accreditation process.

Indicator 8 : At least 2 certifying bodies accredited according to the relevant standard Value (quantitative or Qualitative)

0 2 N/A No


0% achievement. Two certification bodies with approximate compliance have been identified.

Indicator 9 : Companies certified under ISO 9001, ISO 22000 and/or HACCP Value (quantitative or Qualitative)

6 +10 additional ones N/A +6


60% achievement

ix

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements

(USD millions) 1 06/25/2007 Moderately Satisfactory Satisfactory 0.17 2 03/06/2008 Moderately Satisfactory Satisfactory 0.34 3 03/10/2009 Satisfactory Satisfactory 0.55 4 02/01/2010 Satisfactory Satisfactory 1.54 5 10/09/2010 Moderately Satisfactory Moderately Satisfactory 2.07 6 05/14/2011 Moderately Satisfactory Moderately Satisfactory 2.23 7 12/12/2011 Moderately Satisfactory Moderately Satisfactory 2.30 8 05/25/2012 Moderately Satisfactory Moderately Satisfactory 3.04 9 11/13/2012 Moderately Satisfactory Satisfactory 3.97

10 01/02/2013 Moderately Satisfactory Satisfactory 4.20 11 03/31/2013 Moderately Satisfactory Satisfactory 4.63

H. Restructuring (if any)

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

10/08/2007 MS S 0.24 Reallocation of funds from goods to works

12/03/2008 MS S 0.51 Creation of new category of expenditures: Training

05/03/2010 S S 1.72 Reallocation of funds from goods to operating costs

04/15/2011 MS MS 2.21 Extension of Closing date to complete existing activities

09/04/2012 MS MS 3.20 Extension of Closing date to complete existing activities

09/09/2012 MS MS 3.20 Reallocation of funds from goods to operating costs

x I. D

isbu

rsemen

t Profile

US $ Millions

= ... "' w .,. U1 "' 2007Q1r

2007 Q3 \ I I

2008 Q1 --l l I ~

2008 Q3 --l ·~ l

2009 Q1 --l I \ ' 0 ' I ' ~

' ,Q' I ' ;;· ' I ' ~ 2009 Q3 --l I ' 1 ~

2010 Q1 --l \ ' ., ' "1. ' 0

"' ' ~

' 3 ·~ ' 2010 Q3 --l '\ ' ~

' \I < ·~ "' \I

~

2011 Q1 --l < \I ;:;;· :1 ~

\I ..

2011 Q3 --l \I \I :I ' l li ' 2012 Q1 --l :I ' ' \I ' ' \1 '

2012 Q3 --l '!

' ' 2013 Q1 --l

\, "' ... ' 2013 Q3 --l ' ~

' l ' I\ '

2013 Q4 ' ' I \

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Macroeconomic environment

1. Following the collapse of the Soviet Union, the Kyrgyz Republic’s political and economic transition during the 1990’s was mired by heavy cross-winds and severe economic losses. The immediate impact was a cumulative decline in GDP of 54 percent from 1991 to 1995 and a sharp increase in inflation. After a short recovery in 1996, the Kyrgyz economy was again severely affected by the Russian financial crisis in 1998: annual GDP growth fell to about 2 percent, inflation reached 36 percent and the Kyrgyz Som depreciated against the dollar by 43 percent. The economy started to show signs of recovery in 2000, with GDP growth rebounding to more than 5 percent in 2001. Inflation decreased below the two digits level and the foreign exchange rate stabilized.

2. In response to the tumultuous economic climate of the 1990s, the country embarked on a number of policy reforms to improve governance, enhance economic liberalization, and promote privatization and financial sector development. These efforts resulted in the Kyrgyz Republic accessing the World Trade Organization (WTO) in 1998 and establishing itself as one of the more pro-active economic reformers in the region.

3. Nevertheless, the Kyrgyz Republic continued to face significant challenges. The country lacked natural resources, with the exception of gold and water, and is geographically landlocked. The economy was mostly based on agricultural production, hydropower generation and a declining gold production.

4. In addition, the business environment remained inadequate and was not conducive to foreign investment and private sector growth. As a result, private sector investment was muted and foreign direct investment in the non-gold sectors was negligible. Perhaps more importantly, the Kyrgyz Republic’s trading gap with other Commonwealth of Independent States (CIS) countries widened as the country failed to capitalize on price stability and growing demand from its key traditional regional trading partners, Russia and Kazakhstan.

5. Private sector growth was also dampened by political turmoil. The first elected President of the Republic was forcibly removed from office in 2005. Political and social tensions were in turn aggravated by a lackluster economy with limited opportunities for private sector investment.

6. Kyrgyz policymakers faced numerous challenges on policies to accelerate private sector investment and export diversification. The policy reform agenda included the following areas: (i) lowering administrative barriers to businesses (notably in the areas of taxation and inspections); (ii) enhancing enforcement of property and creditors’ rights; (iii) improving the lending infrastructure and legal framework; (iv) modernizing and streamlining the customs regime and cross-border environment; (v) targeting investment in infrastructure notably to support trade; and

2

(vi) streamlining and aligning the domestic standards and certification regime on international standards.

Reforming the Standards Regime to Support Trade-Led Growth

7. Despite the country’s entry into the WTO in December 1998, the Kyrgyz Republic had not been able to reap the benefits of its WTO membership to access new markets. In addition, the country’s trade position with traditional neighboring CIS markets was deteriorating. For example, while the value of imports from Russia had doubled from 1999 to 2005, the value of exports to Russia declined by 7 percent. Similarly, for the same period, exports to Kazakhstan had increased by 24 percent while imports increased by 156 percent. The principal factors behind the disappointing trade performance ranged from shortage of resources to inadequate infrastructure and lack of marketing capabilities, they were also largely related to regulatory and institutional constraints affecting the competitiveness of domestic companies and products. Among these, constraints stemming from regulatory differences in standards needed to be addressed urgently to allow the country to adapt to WTO requirements to benefit from trade liberalization and to keep pace with trading partners.

8. As Kyrgyz Republic’s traditional regional trading partners were progressively moving towards adoption of international standards, the Kyrgyz Republic recognized the urgency of adopting these standards as a prerequisite for successfully engaging in the increasingly competitive trade arena. The Law on the Fundamentals of Technical Regulation in the Kyrgyz Republic, developed with the assistance of USAID and adopted in April 2004, set an ambitious agenda for the country’s transition to international standards.

9. Furthermore, the absence of an internationally recognized metrology and quality infrastructure was hindering economic development as it raised the country’s vulnerability to non-tariff barriers to trade, that in turn delayed market access and further industrial and economic development. As a result, harmonization of domestic standards with international standards and the establishment of a reliable metrology and quality infrastructure became high priorities on the agenda of policymakers.

1.2 Original Project Development Objectives (PDO) and Key Indicators

10. According to the Project Appraisal Document (PAD) and the Development Grant Agreement, the development objectives of the project were to: i) streamline the national technical regulation and standards framework for business; ii) develop systems to enhance quality and safety of products; and iii) increase enterprise competitiveness in pilot sectors.

11. The key project outcome indicator was to be a decrease in the cost of regulatory compliance for business. The PAD further states that although it may not have been measurable during the life of the project, the project was also expected to result in the longer-term in an increase in the share of non-gold trade with existing WTO partners. However, this is not interpreted as part of the PDO because the results were not intended to be visible during the life of the project.

3

12. Key results indicators of project activities included: (i) development and adoption of Technical Regulations (TRs) in key regulated sectors; (ii) removal of unnecessary mandatory certification requirements; (iii) reduction in the list of products subject to mandatory certification; (iv) increase in the percentage of national documentary standards harmonized with international standards; (v) increase in the number of measurement and calibration services with lower associated uncertainties; (vi) establish an Accreditation Body compliant with ISO/IEC 17011; (vii) at least two testing laboratories and two certifying bodies accredited according to international standards; and (viii) at least ten companies certified under ISO 9001, ISO 22000 and/or HACCP).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

Not Applicable

1.4 Main Beneficiaries

13. By reducing technical barriers to trade such as decreasing the cost of regulatory compliance and enhancing the quality and safety of products the project was expected to boost the competitiveness of local businesses and efficiency of public sector agencies. The targeted focus of Technical Regulations supported by the project on the agribusiness sector was aimed at promoting growth of exports over a longer timeframe, particularly those to non-traditional WTO-member markets thus helping to diversify the country’s export base of agricultural products. The growth in exports was expected to have a ripple effect on other producers along the supply chain such as raw materials suppliers, transport services providers, freight handlers, and ultimately workers in those enterprises that benefited directly and indirectly from the opportunities generated by the activities of this project.

14. The adoption of quality standards through the reinforcement of the Metrology, Standards, Testing and Quality (MSTQ) infrastructure in line with international standards was expected to generate important benefits to consumers and the environment. Better quality and safety of products would protect consumers from potential hazards as well as provide them with goods of higher quality standards. TRs on environmental protection would minimize the adverse impact from production activity on the environment.

15. The institutional improvements engendered by the project were expected to benefit numerous government agencies that were linked to MSTQ. In addition, linkages between Center for Standards and Metrology (CSM, previously NISM) and institutions of higher learning would contribute to research activity at academic institutions that could lead to improvements in industrial productivity.

1.5 Original Components

16. In order to achieve its objectives, the project was designed with the following major output components:

4

Component 1: Technical Regulations (TR) – US$1,482,000

17. The assistance under this component would help facilitate the transition to the new standards framework by building capacity of the Ministry of Industry, Trade and Tourism (MITT/MoER)1 and other agencies involved in the development of technical regulation.

18. Under this component, the project would provide support in the following areas:

i. development of a work plan for implementation of the Law on the Fundamentals of Technical Regulation (LFTR), which had been approved in 2004;

ii. development of technical regulations in critical sectors with high contribution to GDP and export/import substitution potential such as agro-business and food processing;

iii. design and establishment of an integrated information system linking MITT/MoER, the National Institute of Standards and Metrology (NISM/CSM)2 and different line ministries and regulatory agencies responsible for development of sector specific technical regulations; and

iv. establishment of the required WTO Enquiry Point for processing inquiries and notification of adopted technical regulations and related regulatory documents to the WTO and its members.

Component 2: Metrology and Standards - US$2,238,700

19. This component would:

i. provide critical etalons, instrumentation and electronic test measuring equipment to help the metrology institute provide basic measurement needs required by the industry and ensure the accuracy and traceability of measurements performed by the domestic laboratory infrastructure; and

ii. establish procedures for preparation, adoption and application of standards in accordance with the WTO’s Agreement on the Technical Barrier to Trade.

Component 3: Accreditation and Quality Enhancement - US$1,130,900

20. Assistance in this component would focus on three areas as described below.

(i) Accreditation (US$200,000)

21. As part of its technical regulation and standards reform, the Kyrgyz Government was planning to spin-off the accreditation function from NISM/CSM and set up an independent, internationally recognized accreditation scheme. For the new accreditation body to gain international recognition, its structure was to be in compliance with the general requirements of

1 During the course of the Project the MITT was restructured and renamed MoER.

2 During the course of the Project the NISM was restructured and renamed CSM.

5

the applicable standard for accreditation bodies (ISO/IEC 17011) related to governance and structural arrangements.

22. Since the ability of the new national accreditation body to attain international recognition would largely depend on its institutional design, governance arrangements, staff skills and expertise, this sub-component would provide institutional development assistance to help structure and build-up the accreditation body in accordance with international standards.

(ii) Testing Laboratories and Certifying bodies (US$830,900)

23. This sub-component would provide support on a pilot basis for upgrading and accreditation of selected laboratories operating in the agro-business and food processing sectors as well as selected certifying bodies in quality management systems and products. The laboratory and certifying bodies involved in this demonstration pilot would be selected based on the assessment to be carried out by a technical consultant.

(iii) Quality at enterprise level (US$100,000)

24. This sub-component would provide assistance on a pilot basis to enterprises willing to introduce quality in their production lines and management systems (International Organization for Standardization, ISO and Hazard Analysis and Critical Control Point, HACCP certifications). Procedures, guidelines and eligibility criteria for selecting beneficiaries and administrating the sub-grant facility would be set forth in the Guidelines for Operation of Sub-grants to be developed by the Recipient and agreed with the World Bank.

25. Although the grant scheme was envisioned to be relatively small (US$100,000), it was expected to be extremely useful for its demonstration effect and could also help mobilize additional resources from donors, notably in the form of commercial credit lines.

Component 4: Project Management (US$217,500)

26. This component would provide support to the Project Implementation Unit (PIU). The start-up cost of the PIU would be funded by a Project Preparation Facility.

27. PIU staff would include an independent Project Manager with basic knowledge of technical regulation and MSTQ-related issues, a procurement specialist, and a disbursement specialist selected on a competitive basis. The project would also set up a Steering Committee consisting of representatives of the Government, various relevant agencies and ministries as well as private sector representatives. The Steering Committee would provide policy guidance and ensure that Project activities are implemented in accordance with the project development objectives.

1.6 Revised Components

Not Applicable

1.7 Other significant changes

6

PHRD Grant

28. Given the complexity and scope of the TR reforms, the Bank applied for a Japan Policy and Human Resources Development (PHRD) Technical Assistance Grant (TF090641) in the amount of US$299,600 on behalf of the Government. The Grant Agreement was signed on March 10, 2008. Grant proceeds were used to: (i) help address implementation capacity issues in the area of legal metrology, conformity assessment and market surveillance; (ii) strengthen overall Project management and coordination; (iii) support implementation agencies’ outreach efforts to increase public awareness of the benefits of the TR reforms to businesses and consumers, and (iv) launch and manage the Matching Grant Facility for adoption of product and quality system at enterprise level.

Changes to IDA Grant Agreement

29. Several changes to the Project in terms of the Closing Date, implementation arrangements, and reallocation of funds were approved. Each of these modifications to the Grant Agreement was considered Level 2 restructuring that did not affect the Project’s PDOs. These changes are summarized below:

Extensions of Project Closing Date

First extension: due to the technical complexity of the MSTQ reform agenda and to the disruptive political events that transpired during the first half of 2010, the Bank management approved an extension of the closing date to October 31, 2012 on April 15, 2011.

Second extension: a five month extension was requested to complete an important ongoing equipment contract delivery consisting of high precision metrology equipment as well as to allow project stakeholders to make more progress towards international accreditation of laboratories (one of the intermediate indicators). The Bank management approved the extension of the closing date to March 31, 2013 on September 4, 2012.

Creation of a new expenditure category

30. Creation of expenditure category “training:” During the early stages of implementation it became clear that staff from the agencies in charge of MSTQ – CSM, Kyrgyz Accreditation Center (KAC) and the TR Department needed to participate and network with their counterparts in other countries by participating in international congresses and fora. These activities were expected to deepen the knowledge and understanding of how other long-established internationally-known counterpart agencies worked as well as increase the visibility of the Kyrgyz Republic’s MSTQ infrastructure. Bank management approved the amendment dated December 3, 2008 introducing a new expenditure category “training” defined as Project-related study tours, training course, seminars, workshops and other training activities not included under goods or services.

7

Reallocation of funds

31. Reallocation of funds to category “works:” At the time of project preparation the Bank was not aware of some issues concerning the physical conditions of the locations for the CSM reference laboratories. As a result, it became necessary to retrofit the facilities that would house the etalons or national references of basic measures. Thus, the Bank approved an amendment dated October 8, 2007 to reallocate funds from the “goods” category to “works” that was to be used exclusively for construction to bring the facilities that would house the metrology labs up to international standards. The reallocated amount was supported by savings in the actual costs of equipment as originally planned.

32. Reallocation of funds to category “operating costs:” this reallocation was needed to re-inject money into operating costs for the PIU in view of an underestimate of salary expenses resulting from the extended implementation period. An amount of SDR75,338 was shifted from “Re-financing of funds for the project preparation” to the Operating Costs category. The Bank management approved an amendment dated May 13, 2010

33. Reallocation of funds to category “operating costs:” this reallocation was needed to re-inject additional funds into operating costs for the PIU to account to the second project extension of five months. An amount of SDR23,000 was reallocated to Operating Costs from the Sub-Grants category. The Bank approved an amendment dated September 9, 2012

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

Soundness of the background analysis

34. The RTBET project was designed to provide assistance to streamline the technical regulation framework and develop and strengthen the Metrology, Standards, Testing and Quality infrastructure. In the design of the project the Bank benefitted from prior support from multilateral and bilateral donors, in particular the EU, the Swiss and USAID a series of initiatives had been undertaken to provide assistance in those areas, which culminated with the passage of the Law on the Fundamentals of Technical Regulation in 2004. The Project had a well-placed emphasis on building capacity and culture change through the introduction of best practices. The skill mix of the team appropriately covered numerous aspects of MSTQ.

35. In the design of the project the Bank focused on key challenges associated with Technical Barriers to Trade that had not yet been addressed by any specific donor – mainly SECO/ITC (Swiss), USAID/Pragma, GTZ (Germany) and EU TACIS, as most of the donor focus had been more on Administrative Barriers to Trade. Project design thus addressed key gaps the Bank had identified to help the transition to an effective modern standards regime and quality infrastructure.

36. The analysis underpinning the enterprise competitiveness objective was limited in scope. The project’s support of enterprise competitiveness was part of the third component which dealt

8

with the upgrading of selected laboratories operating in the agri-business and food processing sectors and by a small quality enhancement sub-component that provided financial support to companies seeking to introduce quality in their production process through ISO certification. However, increased competitiveness hinges on multiple factors, among which those targeted by the Project were important components. Production efficiency and product quality are necessary elements of competitive advantage. Achieving lower costs than competitors involves numerous elements of the production process of which the regulatory cost of compliance is only one albeit an important one. The comparative costs of other resources, which in the case of the Kyrgyz Republic involve labor and natural resources, are also key determinants of competitive advantage. At the same time the productivity of the resources used in production, which in turn depends on the technology used in the production process, will determine the degree of cost advantages. But sustainable competitive advantage also depends on the extent to which producers can differentiate themselves from competitors in other markets. During preparation it might have been useful to have discussed in greater detail the benefits of the project relative to any other barriers to competitiveness, which may have existed at the time and which could have been taken into consideration in identifying expected project outcomes or in broadening the scope of the Project. Greater analysis of the demand for ISO certification by companies seeking to enhance their product quality may have helped to identify the degree of funding that was in fact needed.

37. At the time of preparation, other Bank operations were tackling non-MSTQ related competitiveness issues. One of these projects was the Agribusiness and Marketing Project (P049724) whose Development Objective was to expand the level of activity of processing, marketing, and trade enterprises downstream of the farm-gate, to increase the number and economic importance of producer organizations, and to improve the functioning of markets and trade linkages between producers, and primary and secondary level trade organizations. The project became effective in 2005 and closed in 2013, for a total budget of US$13 million. When preparing the RTBET Project, the Bank’s team interacted greatly with their counterparts in the agribusiness project. Similar efforts during appraisal in addressing other key challenges and opportunities faced by the Project might have been helpful.

38. The linkage between outputs and outcomes in the case of this project was also dependent on the expected response by Kyrgyz entrepreneurs and by foreign buyers. To what extent would exporters invest in quality improvements in their production process; and what would have been the acceptance by foreign buyers of Kyrgyz product standards? The answers to these questions might have helped set realistic expectations for outcomes related to the matching grant sub-component.

Assessment of the project design

39. The objective of the RTBET project in terms of streamlining national technical regulations, enhancing quality and safety of products and increasing competitiveness was consistent with the priority areas established in the Country Assistance Strategy (CAS) of May 2003, which identified three priority areas of intervention: (i) promoting private sector-led growth; (ii) preventing the deterioration of key infrastructure and social services; and (iii) addressing governance constraints to growth and poverty reduction. The project’s three PDOs covered a broad agenda for quality standards and competitiveness comprised of technically complex components. Recognizing that quality and safety standards affect all sectors of the

9

economy, which would have been unwieldy for a small project like the RTBET, the Bank decided to focus on selected industries, principally agribusiness and food processing industries because of their export potential and their importance to consumer safety, as well as the medical/pharmaceutical industry among others.

40. Since the beneficiaries included private sector enterprises there was some measure of external oversight of project activities. Industry representatives were invited to participate in the Project’s Steering Committee, which was seen as essential to ensuring that the new system works for the benefit of entrepreneurs.

41. The PDOs focused on the transition from the old Soviet standards to the new international standards that have become the pillars of globalizing markets. The three main project components addressed the essential elements of the strategy to enhance the quality and safety of production, which would eventually be expected to strengthen the international competitiveness of selected sectors. Due to its technical nature however, the RTBET project became a complex undertaking that called for reforming the legal, regulatory and institutional framework with a new quality infrastructure that replaced the earlier compulsory Soviet era system prone to rent seeking. Institutional adjustments and “mind-change” required to enable the transition from the old Soviet-style standards system to a modern WTO compliant one were going to be challenging tasks.

42. The design of the three project objectives was somewhat ambitious given the institutional development requirements that were necessary to achieve the overall objectives even if only focused on selected sectors. The connection between the project components of i) improving the quality and safety of products in targeted sectors through new technical regulations; ii) upgrading the national network of testing laboratories, and iii) increasing enterprise competitiveness in pilot sectors to the PDOs was not as strong or clearly justified as implied in the PAD. With regards to increasing non-gold trade with existing WTO partners, the design aimed a bit high in terms of development outcomes during such a short time with limited funds.

43. The PDO of increasing enterprise competitiveness by upgrading testing laboratories that would be used by exporters in the agri-business/food processing sectors and a US$100,000 facility to support companies seeking quality enhancements in their production process was not realistic. It fell short of adequately addressing such a broad area as enterprise competitiveness and the multiple factors that contribute to competitiveness.

44. Given the small size of the project and its focus on the MSTQ infrastructure a narrower scope of objectives could have been considered, for example, just reducing the cost of regulatory compliance and upgrading the quality of the production process based on micro-indicators of outcome pertaining to the agri-business/food processing sector, and eliminate the objective of increasing enterprise competitiveness.

45. A Quality Assessment (QALP-2) by the Bank performed in June 2010 rated the quality of the design as Satisfactory.

10

Adequacy of Government commitment

46. As the first CIS country to enter the WTO in 1998, the Government of the Kyrgyz Republic was fully committed to the elimination of technical barriers to trade which was demonstrated by the passage of the 2004 Law. Country ownership for the Project was constant at the highest levels in the country including the Presidency, and relevant ministries, notably MITT/MoER.

Assessment of risks

47. The highest risk factor for the Project as identified in the PAD was the country political risk and possible instability of the Government and to a lesser degree, a lack of cooperation between the implementing agencies. Going into this project the Bank was aware that because of the high risk factors and technical complexity of project activities the key to its success was going to be close supervision by an experienced team that would be committed to managing the project until its completion. Greater consideration of the financial risks such as insufficient support from donor agencies or Government budgetary restrictions that could challenge the sustainability of the project would have been helpful in determining alternative strategies to address these contingencies.

48. Given the high risk factor and technical complexity of the project perhaps a longer period of implementation could have been considered as well as a larger amount of funding that would have provided the flexibility to finance additional equipment for laboratories that may not have been foreseen at the time of preparation.

2.2 Implementation

49. Successful implementation of this project can be attributed to Government commitment, management effectiveness by the implementing agencies and proactive supervision by Bank staff and the PIU. A Quality Assessment (QALP-2) performed in June 2010 rated the Implementation Progress as Satisfactory.

50. The following factors are considered instrumental to the Project’s implementation and outcomes:

51. Government commitment: Government commitment to the project was instrumental in overcoming some serious obstacles during its implementation. Difficult political circumstances in 2010-2011 on the other hand affected negatively project implementation and contributed to the need for extension of the closing date by 23 months (Section 1.7). While the availability of budgetary resources was always an issue reflecting the broader poor state of public finance in the country, the overall commitment of the Borrower manifested itself through the life of the project that underwent frequent social and political upheavals in the country. Despite high staff turnover at the Ministry level – nine ministers of economy from project negotiations to closure, the independent Kyrgyz Accreditation Center (KAC) was allowed to develop and its creation which had disrupted earlier vested interests was never seriously challenged.

11

52. Continuity of Bank staff team: This is a project whose small dollar amount could have rendered it easily vulnerable to being crowded out by larger issues and projects in the country dialogue. The fact that it did not suffer this fate is due greatly to the dedication and to the lasting continuity of the staff team – both field office staff and the TTL, who remained with the Project from the beginning.

53. Quality of experts and consultants: International and local consultants provided valuable input in the development of new TRs. A highly experienced international advisor on technical regulations also supported capacity building. The work by these experts contributed to the project exceeding significantly the original target for approval of technical regulations and the reduction in the number of products subject to mandatory certification. Other consultants who had been involved in the development of metrology institutes in other countries assisted with the identification of critical instrumentation for the development of the metrology infrastructure.

54. Twinning arrangements involving CSM and KAC and their counterparts overseas was a very effective tool for strengthening institutional development within a short timeframe. The brokered twinning arrangements with internationally recognized peer technical agencies such as the one between CSM and the German Metrology Institute (PTB) and KAC with the Turkish Accreditation body (TurKak) led to the establishment of excellent cooperation between these institutions. In the case of CSM their relationship with PTB also facilitated the challenging task of selecting the type of metrology equipment best suited to the specific needs of the country. TurKak was instrumental in preparing KAC’s full membership in ILAC.

55. Coordination with Donors: The RTBET project became a privileged platform for donor coordination on activities of projects funded by various donors active in the area of MSTQ, innovation, trade facilitation and SME development. The nature of collaboration with other donors was purely operational and not formalized in specific co-financing agreements. The RTBET project was able to provide funding for investment in the procurement and installation of equipment, while most other donors had only marginal resources for the purchase of goods. At the same time other donors’ projects complemented the RTBET investment by contributing with resources devoted to technical assistance, training and consulting.

56. For example, cooperation with the German bilateral donor GTZ/GIZ in the area of technical regulations supported the development of Technical Regulations by retaining local consultants to actively participate to the Technical Committees for the definition of draft TRs. In the area of metrology GIZ provided an international consultant on the International Standards Regime. GIZ also supported the renovation of the facilities to house KAC, while the project financed the equipment and the creation of the institutional website. The ITC/SECO project retained an international consultant to provide technical assistance to KAC. This type of collaboration is expected to maintain donor engagement in supporting the sustainability of the PDOs.

57. Disruptive country risk factors: Political instability contributed to delays in Project implementation that were resolved through two extensions of the closing date of 23 months in total (Section 1.7). The first elected president of the Republic was forcibly removed from office in 2005 after nearly 15 years in power following a disputed re-election. These events occurred during project preparation. Then in April 2010, anti-government political demonstrations took

12

place against the authoritarian tendencies of his successor, who had centralized power within the presidency. The April 2010 events left a power vacuum in the south with a mosaic of ethnic groups, which were mobilized by politicians. In June 2010, political and social tensions climaxed into violent inter-ethnic clashes resulting in 300 deaths and large scale destruction of public and private property.

58. The events of April and June 2010 jeopardized project implementation in several ways:

The lack of a functioning Parliament halted the adoption of a number or pending Technical Regulations;

The volatile situation limited incoming travel of experts required to assist the beneficiary in the definition of technical specifications for both high-precision equipment and laboratory renovation;

The lack of a permanent Government and the lack of clarity in the institutional structure of the Government made it impossible to deploy some of the additional required technical assistance; and

The purchasing of metrology and laboratory equipment was also adversely impacted as suppliers increased the costs of guarantees, insurance, shipping costs and installation charges, particularly to account for greater security risks for their staff.

59. Complexity of Institutional Development: Implementation delays were also experienced due to the complexity of the institution building process associated with the reforms of technical regulations and standards. Because these required extensive revamping of an institutional standards framework that had prevailed in the region for decades, the Kyrgyz Republic as with many of the CIS countries initially encountered difficulties and resistance from within the Government and implementing agencies in the transition to the new regulatory and quality standards system, and thus achievement of a number of project activities was slow to materialize such as those dealing with metrology and standards.

60. Purchasing of complex calibrating and laboratory equipment led to project delays: The complexity of MSTQ equipment was another factor that contributed to the project delays. While procurement under the project was well managed, the technical specifications of some of the equipment purchases had to be revised in order to attract qualified bids. The preparation of technical specifications for the procurement of high precision metrology equipment was delayed at inception due to the difficulties in identifying and retaining an appropriate international technical expert with the required set of skills. The announcement for the hiring of an international consultant was advertised three times. This issue was subsequently resolved through the organization of the above-mentioned twinning arrangement with the German PTB through which the PTB provided the technical resources to define technical specifications for the equipment that was subsequently acquired and installed in refurbished facilities. The tender for the Integrated Information System (IIS) linking various regulatory and supervisory agencies in support of TR development was canceled as the two offers received did not comply with the specifications. An international IT technical specialist was then retained under the project to assist in the revision of the technical specifications. The tender for the purchase of testing equipment for the three laboratories under the Ministry of Health was also cancelled in January

13

2011 due to issues with the technical specifications. These issues were resolved with the hiring of an international consultant who revised the specifications and the next bidding round resulted in the award of the contracts.

61. As implementation of the Project gathered momentum and the implementing agencies became more experienced in the acquisition of equipment new opportunities for upgrading of laboratories were identified. For example, with more money and time the project could have supported the establishment of a testing laboratory for water and bottled water, a sector with high growth and export potential. Funding for the water laboratory was originally to come from savings from other project components. Discussions on the water lab started in 2010 and envisioned an arrangement based on a public-private sector venture, where the Government would build the lab and then turn over its management to a private sector company. However, the actual savings from other components was not sufficient to move forward with this activity.

62. Support of private sector firms in acquiring quality standards: Utilization of the matching grant facility for adoption of quality standards by private sector firms was much less than anticipated. The trend in acquisition of voluntary quality standards (ISO) by firms was slow suggesting that companies were not yet investing in quality or aiming to expand into new (quality sensitive) WTO markets. Many Kyrgyz companies appeared to be much more focused on maintaining their internal market shares and keeping afloat as going concerns rather than investing in quality systems in support of market expansion and exports in what many perceived as a mostly unstable political and international economic environment. In addition, smaller yet high export potential enterprises (e.g., textile/apparel) tended to shy away from exploring new markets given their limited production capacity. While some of these issues could have been anticipated at the time of project design, the major challenge facing potential users of the facility was the high degree of market uncertainty and financial disruption after 2010; a development that could not have been anticipated at the time of project design.

63. Due to the technical complexity of project activities and other factors mentioned above, project implementation was delayed and the closing date was extended by 23 months. Accreditation of targeted testing laboratories and certifying bodies was advanced at the time of closing and are expected to be achieved in the near future.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

M&E design

64. Per the PAD, there was one main outcome (PDO) indicator: decrease in the cost of regulatory compliance for business. By definition, such an indicator would be calculated as the ratio of the cost of regulatory compliance to total operating expenses of businesses that benefitted from the project. In the absence of such information the PAD relied on “macro” information drawn from various indexes such as the Doing business (DB) and the Business Environment and Enterprise Performance Survey (BEEPS). With respect to enterprise competitiveness, there was no quantifiable indicator that could have been utilized to measure outcomes. The reference in the PAD to non-gold exports to WTO member countries was not intended to be measurable during the life of the project.

14

65. The Doing Business (DB) index that was used as the indicator represents the overall ranking for the Kyrgyz Republic. While there is a correlation between the cost of regulatory compliance and the overall level of the DB ranking, there are other country indicators that are included in the overall index: starting a business, dealing with construction permits, getting electricity, registering property, getting credit, protecting investors, paying taxes, trading across borders, enforcing contracts, and resolving insolvency. Not all of these indicators are directly related to the outcomes of the project. In fact the majority of these sub-indexes are not directly related to the PDOs.

66. Perhaps some consideration could have been given to the definition of a short-term indicator of increased competitiveness. An additional trade-related indicator could have been reduced import dependence, since this would reflect the impact from the adoption of new quality standards by local businesses that are focused primarily on the domestic market. Local consumers may have switched to higher quality local products that were more price competitive than imports.

67. At the time of project preparation it may have also been useful to develop information through surveys of businesses in a targeted industry such as food processing by compiling a “dashboard” of financial and business indicators that could have provided better focus on the relationship of reduced cost of regulatory compliance and enhanced quality standards on the competitiveness of companies in that industry.

68. In view of the challenges of measuring the key outcome indicators, this analysis has relied on the intermediate indicators of outcome as proxies of project effectiveness. Intermediate results indicators were maintained up-to-date throughout the course of the project and covered all of the project components. Perhaps some additional intermediate indicators of the level of activity by CSM, KAC, the laboratories and certifying bodies might have been useful as a measure of how businesses were incorporating the new standards in their operations.

M&E Implementation and utilization

69. Project information was monitored through the Implementation Status and Results (ISRs). The Bank supervision missions and the PIU kept the project information up-to-date with detailed records on each of the components. Nevertheless, the main data sources were comprised of the intermediate indicators of outcome.

2.4 Safeguard and Fiduciary Compliance

70. No deviations were found with respect to the Bank’s financial management, procurement, and disbursement policies. All annual audits performed by external auditors confirmed that the financial reports for the project gave a true view of actual expenditures and that the PIU provided the auditors with all the relevant information to arrive at their opinions.

2.5 Post-completion Operation/Next Phase

71. The Ministry of Economy requested a follow-up World Bank operation to build on the foundation established under the RTBET project and provide further support to the National

15

Quality Infrastructure (NQI) of the Kyrgyz Republic. However, the current available IDA envelope is limited, and the World Bank may not be in a position to accommodate the Ministry’s request. The RTBET project team will explore ways to continue supporting this reform agenda. The MSTQ and NQI space remain important areas of interest to other donors, who will continue to provide technical assistance support to selected agencies. PTB, building on the twinning arrangement brokered under the RTBET project, will continue to involve technical experts from the CSM in their regional capacity building TA project in the areas of metrology and calibration. TurKAK is continuing to provide technical assistance in the area of accreditation to the KCA in continuation to the excellent working relationships established through a similar twinning arrangement brokered between the two peer institutions under the project. GIZ will remain engaged in the area of administrative barriers to trade while ITC/SECO is expected to continue support to SMEs in the area of export and trade promotion. All of these donor activities will help provide valuable continued support to the MSTQ reform agenda and individual agencies involved in the system. The RTBET project team is also currently preparing a results movie to document and disseminate the project results and provide information on the benefits of MSTQ reforms, notably to neighboring countries as MSTQ reforms have much to gain from enhanced regional cooperation.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

Relevance of objectives

72. The development objectives of the project, streamlining TRs and standards framework and developing a system to promote the quality and safety of products is consistent with the current Country and Bank development goals. The overall goal of the current Country Partnership Strategy (CPS: 2014-2017) is rooted in a single pillar of improved governance, which in the case of the RTBET project is reflected in accountability, transparency and efficiency in utilization of resources. The current CPS’s support for improved governance focuses on three broad areas of engagement—public administration and public service delivery, business environment and investment climate, and natural resources and physical infrastructure. An improved business environment and investment climate is seen as a prerequisite for faster growth and job creation. These CPS objectives are reflected in the RTBET’s PDOs of streamlining the national technical regulations and increasing enterprise competitiveness. Project activities support the Country’s efforts to harmonize its regulatory and institutional framework with WTO and international trade system requirements. As highlighted by the QALP review of June 2010, the “project was highly relevant and consisted of the strategic building blocks to assist the Government in tackling priority issues in trade and business environment”. The Project remains relevant for the current Government national development strategy that makes direct reference to MSTQ as a key element for the diversification of the economy and enterprise competitiveness.

73. The development of a system to enhance quality and safety of products – including the environmental impact of production activities, and by doing so increasing enterprise

16

competitiveness – directly addresses the current core objective of improved governance as it applies to the private sector.

Relevance of design and implementation

74. The project design is deemed highly relevant for achieving the two main dimensions of the PDO (namely, reducing the cost of regulatory compliance and developing systems for product quality and safety) because the component definitions and intermediate outcomes were closely related to the PDO level outcomes. New TRs and the transition from mandatory to voluntary certifications supported by a risk-based inspections regime contributed to the reduction in the cost of regulatory compliance. The PDO indicator “cost of regulatory compliance” reflected a relevant project outcome generally speaking, but using the overall ranking in cost of doing business survey did not reflect closely the results achieved through project interventions (Section 2.3). The intermediate outcomes were very relevant for measuring the project outcomes. Project activities and results proved instrumental in advancing reforms towards streamlining of the business environment (introduction of Risk-Based Inspections regime), and provided continued support to the various governments and relevant ministries in tackling the challenges of the country’s MSTQ reform. Project activities not only helped to streamline business regulations but they also supported the Country’s efforts to harmonize its regulatory and institutional framework with WTO and international trading system requirements. The project’s targeting of agribusiness also supports the overall country strategy to improve the management of natural resources in a way that optimizes job growth and thus improve the standard of living.

75. However, the design of the component to support quality enhancements was not as relevant to the third dimension of the PDO (increasing enterprise competitiveness in pilot sectors). Component 3 (ii) was to upgrade laboratories that provided testing services utilized by agribusinesses/food processing companies and the grant facility in Component 3 (iii) was to provide co-financing for companies seeking ISO quality certification. Improving the quality of producers’ output through improved measurement capabilities, testing of product quality and ISO quality certification are necessary but not sufficient conditions for increasing enterprise competitiveness. In addition, the matching grant set up in support of enterprise acquisition of certification was small and limited in scope.

76. Based on the reasons discussed above, the overall relevance of this project is deemed to be High.

3.2 Achievement of Project Development Objectives

77. The Project has achieved substantive results in the two first dimensions of the PDO. It streamlined the national technical regulation and standards framework for businesses, reducing the cost of regulatory compliance to a high extent. It also developed systems to enhance quality and safety of products to a substantial extent. However, the project achieved modest results in the third dimension of the PDO - enhanced enterprise competitiveness in pilot sectors. The overall project efficacy is considered substantial (see section 3.3).

78. This section assesses to what extent each dimension of the PDO was achieved through component outcomes and how these outcomes contributed to reducing the cost of regulatory

17

compliance for business (see Table 1). The efficacy of the PDO is assessed through the analysis of the Intermediate Indicators that were identified in the PAD (shown on Table 1) and additional documentary evidence of outcomes captured in selected indicators. Indeed, while the overall project design was good, the M&E framework had some weaknesses and for that reason the assessment of the PDO achievement in the ICR is based on intermediary outcome indicators. Each of the Intermediate Indicators was supported by project outputs that are summarized in Annex 2 of this report.

79. The degree of achievement of project outcomes is accurately measured by the intermediate indicators. While the PDO indicator of reduced cost of regulatory compliance reflects the overall achievement of the three PDOs, even though the indicator chosen for this project was only marginally relevant, the intermediate indicators do reflect the degree of achievement of each of the three PDOs. The new TRs incorporated the latest international practices in product quality and safety guided by risk-based principles and did away with the excessive mandatory requirements that prevailed under the old GostStandard. The reduction in the number of products subject to mandatory certification contributed directly to easing the regulatory burden for producers, including exporters, of products that are not considered to represent a risk to consumers and that are manufactured according to fairly standardized processes. A notable increase in the number of documentary standards harmonized with international standards enhanced the system of quality and safety of products since it has increasingly been guided by practices that have produced positive outcomes worldwide. The increase in the number of measurement and calibration services provided by the labs equipped under the project will also help raise the quality of output by local producers, who supply inputs to other businesses, sell directly to consumers, and export.

80. Key achievements of the Project by Component are summarized below:

The Technical Regulations component exceeded its expected performance, although the enforcement and surveillance of the new regulations is still in the early stages of implementation. Key achievements include:

o The target for the number of new Technical Regulations issued under the project was eight, while the actual value to date is 35. These 35 new Technical Regulations are now in force in various sectors, ranging from food and agricultural products to construction and transportation services.

o The list of products subject to mandatory certification has been reduced significantly to 684 at project completion from 5,500 at project design. This led to a significant decrease in the number of mandatory standards for safety and quality requirements from 22,000 old Soviet GostStandards to just 100 WTO compliant modern standards.

o The issuance of the new Technical Regulations also led to the simplification of procedures for product certification, lowering the number of days required for certification from 15 to seven.

o By introducing product classification on the basis of their inherent risks, these new Technical Regulations are at the root of the Risk Based Inspections regime that is now being implemented in the country. The introduction of this reform led to a

18

threefold decrease in the number of business inspections to 7,247 in 2012 from 20,629 in 2011.

The Metrology and Standards component has achieved its expected impact with newly equipped metrology laboratories. The procurement of state of the art metrological and laboratory equipment expanded the range and accuracy of tests performed in the Kyrgyz Republic.

The Accreditation and Quality Standards component has made substantial progress particularly in the establishment of the Kyrgyz Accreditation Center (KAC), but is has not yet reached the critical mass needed to achieve its intermediate impact. Qualified international auditors carried out an assessment of the international accreditation readiness level of selected testing laboratories and certification bodies. In the case of the laboratories, the readiness ranged from 90 percent to 95 percent depending on the type of testing equipment and procedures. In the case of the certification bodies, the readiness level was assessed at 75 percent. ILAC is expected to review KAC readiness for full membership in October 2013. Notwithstanding the progress in the development of the Kyrgyz Republic’s quality of infrastructure under the project, the adoption of quality systems by enterprises remains slow, due to several factors, ranging from low sophistication of local companies and competing investment priorities.

Enterprise competitiveness in pilot sectors. With respect to the third dimension of the PDO –enhanced enterprise competitiveness in pilot sectors, the results are modest. Improved enterprise competitiveness was to be achieved through enterprise acquisition of quality certification. The small matching grant facility that was set up under the project was designed as a pilot to facilitate enterprise acquisition of quality certification in a few sectors. At project closure, 12 companies had acquired international certification of quality management systems (i.e. ISO). These companies operate in a variety of sectors, ranging from agribusiness and foodstuff to manufacturing (cement, bulbs) and services. Of these companies, only two benefited from the support of the pilot matching grant facility set up under the project. While the result is modest, interest in quality certification has been growing. Agro-industry associations (fruit and vegetable association) are promoting the importance of quality for competitiveness due to pressures from destinations markets such as Russia, which recently joined the WTO and remains a key trading partner for the Kyrgyz Republic. The establishment of the WTO inquiry point is expected to help channel information from and to trading partners of the Kyrgyz Republic.

Table 1: Assessment of PDOs based on achievement of project Intermediate Indicators At time of Baseline ICR Target

PDO #1: Streamlining the national technical regulation and standards framework for business - High Component 1: Technical Regulations

Technical Regulations developed & approved 0 35 8 Reduction in products subject to mandatory certification 5,500 684 Reduction

19

PDO #2: Developing systems to enhance quality and safety of products – Substantial

Component 2: Metrology and Standards NISM/CSM QMS accredited No Yes Yes

% of natl. documentary standards harmonized with intl. Standards

38% 46% 46%

No. of calibration measurement & calibration services with lower associated uncertainty

500 800 600

Component 3: Accreditation and Quality Enhancement

KAC compliant with ISO17011 No Yes No

2 plus labs accredited under intl. standards (ISO) 0 0 2

2 plus certifying bodies accredited under intl. standards (ISO) 0 0 2 PDO #3: Increasing enterprise competitiveness in pilot sectors

Companies certified under ISO 9001, 22000, or HACCP 6 12 16

Contribution of intermediate outcomes to the PDOs

PDO #1: Streamlining the national technical regulation and standards framework for business through Component 1 “Technical Regulations”

81. The project streamlined the national technical regulation and standards framework for business to a high extent. Subsequently, the introduction of new standards through TRs has established a solid foundation for lower costs of regulatory compliance and for increased competitiveness of enterprises. The reduction in the number of products subject to mandatory certification (down to 684 from 5,500 at project design) led to a significant decrease in the number of mandatory standards identified for safety and quality requirements from the old Soviet GostStandard. This has resulted in a decrease in the regulatory cost of compliance for many producers. The introduction of Technical Regulations also simplified the procedure for certification of products, lowering the average number of days required for certification from 15 to seven. The project thus helped to transform the MSTQ from a “control system” to a “compliance risk-based” system based on the new TRs and a market surveillance infrastructure.

82. The development of new TRs outperformed the targets set out in the PAD. As shown in Table A2-1 in Annex 2, there is an ongoing pipeline of new TRs aimed at improving the quality and safety of products as well as reducing the cost of regulatory compliance. While the activities in MSTQ supported by this Project targeted certain industries such as food production, there is a significant pipeline of new TRs in other industries that are in various stages of preparation. The workflow of new TRs was facilitated by the acquisition of an IT system, also financed by the project, for collaboration and monitoring by different agencies and other stakeholders involved in development of new TRs. The establishment of the WTO Enquiry Point supported by the project will also serve as a channel to support the harmonization of the domestic regulatory system with international standards.

83. By introducing product classification on the basis of their inherent risks, the Technical Regulations reform is also at the root of the Risk-Based Inspections regime (RBI) that is now being implemented in the country. The Legal Metrology Inspectorate started to operate in 2012

20

although during its initial phase of operations it has limited its supervision and surveillance activities to public sector entities.

84. An important micro-indicator of PDO’s is the effectiveness of the new TR’s in achieving greater safety and higher quality of products and services. While the quality of the technical work in the development of the TRs has been in line with international standards, their enforcement through the work of the Legal Metrology Inspectorate is only at the beginning stages. The key to achieving internationally recognized quality standards, and thus strengthening the competitiveness of goods and services which will in turn contribute to the expansion of exports to other WTO member countries will hinge on the effectiveness of surveillance and enforcement actions in engendering a culture of quality and safety in the business community. This is an important outcome that will require additional time for achievement.

PDO #2: Developing systems to enhance quality and safety of products

85. The project developed metrology and standards and accreditation and testing laboratories/certifying bodies to enhance quality and safety of products to a substantial extent. The institutional development impact on KAC and on the testing and laboratory labs and certification bodies and their pending accreditation bodes well for the eventual effectiveness of this component.

Through Component 2 “Metrology and Standards”

86. Through the project’s support, CSM exceeded the targets for calibration measurement & calibration services with lower associated uncertainty and continues to provide more and better measurement and calibration services with tangible positive impact on Kyrgyz companies. Two state-of-the-art reference laboratories are fully operational and allow CSM to provide more reliable and precise measurement and calibration services to the Kyrgyz public and private sectors.

Table 2: CSM Calibration Activities

2010 2011 2012 Mass - 29 32

Temperature - 25 29

87. The availability of reliable calibration services in the Kyrgyz republic makes it cheaper and understandably more convenient for Kyrgyz companies to now calibrate in Bishkek. Kyrgyz entrepreneurs also benefit from lower fees and an increased range of services. New services now available to users thanks to the installation of the new high precision metrology laboratories procured under the project allow companies to calibrate in Bishkek instead of taking their equipment to Almaty which resulted in lower costs (see Table 3 below). Additional laboratories in six areas (volume, pH-meter, density, pressure, time & frequency, and viscosity) were installed by the time of project closure and are expected to further enhance the availability of calibration services to the broad spectrum of production activities in the country (see Annex 2 for more details).

21

Table 3: Calibration Services and Prices in Bishkek and Almaty

CSM NaCEC

Calibration Services Bishkek Almaty Calibration of weights:

E2 class of accuracy 156 KGS 2,000 KGS

For calculation of uncertainty of Measurements 1,420 KGS 2,500 KGS

Calibration of liquid-in-glass Thermometers

600 KGS Service unavailable

KGS are Kyrgyz Soms. Source: CSM

88. The approximation to international standards has also contributed to raising the quality of goods and services produced in the Kyrgyz Republic to international ISO/IEC standards. At the time of this ICR about 46 percent of national standards were fully in line with international WTO standards, as targeted at appraisal, and this percentage is expected to increase.

89. At project inception the Kyrgyz Republic was not represented in a systematic way in international and regional fora on metrology, standards and accreditation; RTBET helped put the Kyrgyz Republic “on the map” in the Quality infrastructure area. As a result, today Kyrgyz technical delegations are tasked with technical responsibilities in multilateral fora; the most recent example is from the COOMET meeting in September 2012 where CSM was tasked with the definition of guidelines for calibration of weights and calculation of uncertainty to be adopted by the COOMET countries3.

And through sub-components 3 (i) and (ii) “Accreditation and Testing Laboratories and Certifying Bodies”

Accreditation

90. The project managed to centralize the accreditation functions under one umbrella that were previously housed in different government agencies, and spin it off from metrology, as these are two separate and incompatible technical functions. By consolidating the accreditation function from various agencies under one independent agency, the project lowered opportunities for rent seeking, and possible conflicts of interest.

91. A well-functioning KAC was established and equipped under the RTBET project. As shown by Table 4 below, KAC has been active in accrediting testing and calibration laboratories

3 COOMET is the organization for regional cooperation on metrology, associating 18 countries: Armenia,

Azerbaijan, Belarus, Bulgaria, Georgia, Germany (Assoc. Member), Kazakhstan, Kyrgyz Rep., DPR of Korea (Assoc. Member), Cuba (Assoc. Member), Lithuania, Moldova, Russia, Romania, Slovakia, Tajikistan, Uzbekistan and Ukraine.

22

as well as certification bodies. It also organized laboratory inter-comparison activities for testing of water and geological materials.

Table 4: KAC Accreditation Activity

2008 2009 2010 2011 2012 Testing & calibration Labs

National standards 16 2

Intl. standards 52 65 80 89 80

Conforming Assessment Bodies

National standards

Intl. standards 19 18 16 14 14

92. One of the critical steps for global recognition of KAC was its full acceptance into the ILAC (International Laboratory Accreditation Cooperation). The application for full ILAC membership was initiated in 2009 and at the time of project closure KAC was at the final stage of the process. Full ILAC membership was one of the key Project Intermediate Indicators that establishes the technical competence of KAC in the accreditation of laboratories and certification bodies compliant with the highest international standards. Once accredited, KAC will be in a position to apply all relevant ISO standards in their accreditation activities of domestic laboratories and certification bodies.

Testing Laboratories and Certifying bodies

93. The RTBET supported the upgrading of three laboratories in the Epidemiology center under the Ministry of Health: microbiology, chemicals and radiology. These laboratories provide services to the agri-business and food processing sectors. As shown in Table 5 below, the newly equipped labs have been providing testing services for numerous companies. The new equipment allows the lab to perform tests in the area of pesticides and residues that were not available before the Project.

Table 5: Laboratories Activity

2010 2011 2012

Microbiology Lab

Tests (#) 7,242 9,507 9,838

Chemicals Lab

Tests (#) 30,038 28,228 24,421

94. The increase in the number of products tested has been made possible through the expanded capabilities of the labs through the acquisition of state-of-the-art equipment. Table 6 below shows the larger list of tests available in the case of the radiology lab.

23

Table 6: Availability of Tests Before and After the RTBET

Before RTBET New Lab installed under RTBET

Testing method based on volt-ampere-metric inversion Lead Zinc Copper Cadmium Arsenic

Testing method based on nuclear absorption Lead Zinc Copper Cadmium Arsenic Tin Iron Nickel Antibiotics Melanin Vitamin content Caloric content Aspartame Sodium benzoate Benzapyrene Saccharin

95. On international accreditation of selected testing laboratories under ISO 17025 and certification bodies under ISO 17065, the delays related to the installation of testing laboratories accumulated throughout implementation in turn delayed the process of international accreditation against the relevant ISO standards. At the time of project closure two international experts had assessed the readiness of the testing laboratories at 90 percent to 95 percent with the gap attributed to the need to perform laboratory inter-comparisons. In the case of certification bodies the main gap is in the translation into English of guidelines and internal documentation. When fully accredited to international standards these labs and accreditation bodies will serve as the new standard for other labs and accreditation bodies.

PDO #3: Increasing enterprise competitiveness in pilot sectors through sub-component 1 (c) “Quality at enterprise level”

96. The project increased enterprise competitiveness in pilot sectors to a modest extent. The adoption of quality management systems by enterprises remains slow, due to a combination of factors ranging from competing investment priorities for companies that continue to face difficult economic conditions to the relatively low level of sophistication of local companies and buyers’ markets in which they operate. As explained in Annex 2, only two companies received support under the project for certification of ISO standards. However, 10 other companies have received ISO certification that did not tap the resources of the Grant Facility. One of the critical linkages between the project and the PDO of increased enterprise competitiveness is its effectiveness in instilling a culture of quality at the enterprise level. Achievement of this objective may still take some time.

Other Evidence

97. This section presents additional indicators of outcome that complement the analysis based on Intermediate Indicators. As discussed in section 2.3 the DB indicator was not a realistic

24

outcome indicator because it was not directly linked to the PDOs of the RTBET. However, there are other “macro” indicators that can be used as evidence of outcomes such as the World Economic Forum’s Global Competitiveness Index and the Enterprise Performance Survey (BEEPS). The relevant information contained in each of these indicators can be considered as proxies for the main outcome indicator of “reduced cost of regulatory compliance.”

98. The Global Economic Forum’s Global Competitiveness Index which is shown in Table 7 demonstrates a myriad of factors that comprise the overall characterization of a country’s competitiveness. Two of the sub-indexes are considered as relevance to the Project’s PDO of reducing the regulatory cost of compliance. The number of procedures to start a business may have been positively impacted by the introduction of technical regulations for products and for the safety of the facilities housing the business requires that simplified the process. The sub-index of local supplier quality has also been positively impacted by the upgrading of testing laboratories in the agri-business/food processing sector. For example, a producer of agricultural inputs sold to a food processor has access to calibration and testing services that enhances the quality of the products sold to the food manufacturer.

Table 7: Impact of Reduced Cost of Regulatory Compliance on Enterprise Competitiveness

(Ranking relative to all countries)

2006 2007 2008 2009 2010 2011 2012

Global Competitiveness Index 110 119 122 123 121 126 127

Sub-Indices for Competitiveness:

No. of procedures to start a business

31 37 44 8 6 3 3

Local supplier quality 110 121 125 124 122 93 107

No. of countries ranked: 122 131 134 133 139 142 144

Ranking is based on total number of countries ranked, where 1 is the highest-ranked country. For each index the table shows the total number of countries ranked per year. Sources: Ease of Doing Business - World Bank; Competitiveness Index - World Economic Forum

99. The latest available survey for the BEEPS is for 2009. One of the findings of the 2009 survey was that concerns about business licenses and permits were noted by only 2.2 percent of the firms responding to the survey. These results indicate that relative to other concerns, for businesses the problems of licenses and permits are not as burdensome. The low percentage for business licenses and permits may have already captured the initiatives supported by the project of reducing the number of products subject to mandatory certification.

3.3 Efficiency

100. RTBET was efficient in its utilization of limited project resources (US$5 million in total), as testified by the high level of disbursements, which reached 95 percent of the original Grant amount, that led to substantial achievements. Notwithstanding the lack of economic analysis

25

during project appraisal, the project achieved substantial results during its implementation, including a reduction in the cost of regulatory compliance for businesses and improvements in quality systems for product safety and quality. On the other hand the impact of increased competitiveness on growth of non-gold exports as explained in the PAD was established as a medium-to-long-term outcome not intended to be captured within the duration of the project.

101. The net benefits of the project based on its three main components consist of the reduced cost of regulatory compliance, although it is not possible to quantify the monetary amount of cost reductions from available information. For example, the number of type products subject to mandatory certification was reduced from 5,500 to 684 and the number of days required for certification was reduced from 15 to seven; thus the financial benefits of these two outcomes are considered significant. By building on the work previously done by other donors and which had led to the passage of the Law on the Fundamentals of Technical Regulation, the project was able to engage in a more efficient way by focusing on the development of the TRs already prescribed in the new legislation. The twinning arrangements involving PTB and TurKak as well as the quality of experts and consultants with extensive prior experience in the field of MSTQ contributed to the more cost effective purchasing of scientific equipment that incorporated the latest technologies available worldwide. By upgrading labs that provided calibration and measurement services oriented towards agro-businesses/food processing companies the project achieved a higher level of efficiency given the importance of this sector to the economy. By increasing the accuracy of measurements, the high precision metrology equipment procured under the project will also help reduce potentially costly measurement errors across sectors ranging from electricity distribution to manufacturing and provision of health services.

102. The project however suffered from significant implementation delays due to the highly technical nature thus complexity of activities (metrology, calibration, accreditation and testing) as well as frequent disruptions from social and political events that affected the country as a whole (upheavals in April and June 2010). These delays led to two project extensions of 23 months in total; however, without generating any cost overruns. By delaying the net benefits from the project as explained in the case of KAC’s Full Membership in ILAC, the laboratories and certification bodies, the longer duration of the project and still pending completion of accreditation and supervision activities the present value of those benefits have accordingly been materially reduced.

103. Another limitation on the Project’s efficiency is the small size of the operation. With only US$5 million for support of the MSTQ infrastructure, the multiplier effect may fall short of its targeted growth of exports. At the same time, in addition to the noticeable decrease in regulatory burden on enterprises, the project introduced new Quality related services (broader range and reliability of measurements in calibration and testing services) critical to production processes and consumer safety that were not available in the country before.

104. Based on the experience of the project, particularly the longer than expected implementation period, efficiency is considered to have been substantial.

26

3.4 Justification of Overall Outcome Rating

105. The overall outcome rating, which incorporates the three rating sub-categories is thus considered moderately satisfactory. The relevance of the project is considered high and both the achievement of the PDOs and efficiency are considered as substantial.

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

Not Applicable

(b) Institutional Change/Strengthening

106. The project had a strong institutional building dimension as it supported the establishment of an independent accreditation body (KAC) and also strengthened the capacity of key institutions involved in the MSTQ reform, namely the Ministry of Economy and the Center for Standards and Metrology (CSM). Specifically, the project supported these target institutions through tailored technical assistance aimed at building internal capacity. In addition, the project brokered twinning arrangements with internationally recognized peer technical agencies such as the German Metrology Institute (PTB) and the Turkish Accreditation body (TurKak). These twinning arrangements led to the establishment of excellent cooperation between these institutions. The project enhanced the capacity for regulatory reform: RTBET provided extensive capacity building support to the Ministry of Economy and other ministries and agencies in Technical Regulation development. Throughout project implementation, approximately 200 public officials and technical specialists from various ministries and agencies were trained on the various aspects of Technical Regulations and MSTQ. RTBET also increased the international visibility and recognition of the Kyrgyz Republic as an active member of the WTO arena: at project inception the Kyrgyz Republic was not represented in a systematic way in international and regional fora on metrology, standards and accreditation; RTBET helped put the Kyrgyz Republic “on the map” in the Quality infrastructure area.

(c) Other Unintended Outcomes and Impacts (positive or negative)

Not Applicable

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

107. While no beneficiary survey was conducted, on April 15, 2013 to mark the closure of the RTBET project, a high-level workshop was organized in the presidential palace gathering all project beneficiaries including all concerned government and MSTQ agencies, laboratory operators, private companies that had acquired ISO certification in the water and confectionary sectors, project implementation unit staff, Bank management, donors and the media. The workshop presented the main project achievements and discussed the remaining agenda ahead to strengthen the domestic Quality infrastructure and boost competitiveness. The workshop was moderated by the Minister of Economy and was addressed by both the President of the Kyrgyz Republic, who had served as Minister of Economy during project preparation and was a strong

27

supporter of this operation, and the Prime Minister4. The project team is also currently preparing a results story movie to better capture and disseminate the results of the project as well as broad benefits of MSTQ reforms.

4. Assessment of Risk to Development Outcome

108. Risks to Development Objectives are considered as significant for the following reasons.

109. Political changes and volatility are still considered the principal risk factor as they were at the time of Project preparation; although the likelihood of internal political conflict has diminished. While the constitutional reforms implemented in 2010 bode well for continuity in the political process, long-standing tensions still persist. Nevertheless, the possibility of political changes that could adversely affect Government policies in those areas supported by the project continues to be a significant challenge to sustainability of development outcomes.

110. Risks to the economic outlook either from a weaker global economy or from internal factors that impede growth of domestic markets, such as other constraints to competitiveness as demonstrated by some of the Doing Business and in the Competitiveness indicators as well as in the Enterprise Surveys that were not addressed by the RTBET, and by other Bank and donor projects may limit the ability of local enterprises to invest in quality improvements.

111. Since the Kyrgyz Republic's main trading partners in the Eurasian Customs Union are either already members (Russia) or aspiring to become a member of the WTO (Kazakhstan, which in anticipation also recently embarked on reforming its Technical Regulations framework), the proposed adoption of the Customs Union in the Kyrgyz Republic does not represent an obvious risk to the project outcome. However, it should be noted that the difference in the pace of alignment of the various trading partners on WTO non-tariff barriers to trade regimes may have an impact on the ease of cross-border trading as regulatory requirements and standards differences persist.

112. The financial viability of the project could be compromised if budgetary support from the Government is curtailed. Fee-generating opportunities from the principal agencies such as CSM, KAC and the laboratories in the form of certification, inspection and other user fees will reduce but not eliminate budgetary dependence on Government resources.

113. Proper follow-on operational and financial support from the Government is necessary in order to continue building on the achievements of the Technical Regulation and Quality infrastructure reforms. The MSTQ system still needs hand-holding in terms of financial support and institutional development. The technical agencies - KAC and CSM - will require continued support from the Kyrgyz Government in order to sustain their achievements in the sphere of accreditation and metrology. Much of the progress achieved by these agencies to date has been motivated by the existence of donor support channeled directly or through the RTBET project. And as a result of the close collaboration between the Bank and the main donor agencies, their

4 The press release covering the closing event can be found at the following link: http://www.worldbank.org/en/news/press-release/2013/04/15/world-bank-project-increases-international-visibility-of-kyrgyz-republic-as-wto-member

28

support is expected to continue post-RTBET. If donor support were to wind down, the Government would need to allocate additional resources to ensure the sustainability of its MSTQ infrastructure and to maintain the human and technical capacity developed in these agencies over the past six years of project implementation.

114. An important element of sustainability in the short- to medium-term following the closure of the RTBET project will be the role played by donor assistance. Due to Government budgetary constraints continuation of the key planned activities in supervision – Legal Metrology Inspectorate, recalibration of metrology equipment, accreditation and upgrading of more laboratories – may well hinge on the degree of donor support and on the ability of some of the agencies to find revenue generating opportunities. The Center for Standards and Metrology is currently being assisted by GTZ; but at this critical juncture in the modernization of the MSTQ infrastructure continued donor support is essential.

115. Nevertheless, irrespective of any continued donor engagement, the sustainability of the reforms will rest on the shoulders of the Government. While recent history showed that the Kyrgyz Republic can be prone to political and social instability, the current Government and President are supporters of the work accomplished to date which gives reasonable indications that MSTQ reforms could be sustained over the medium term although foregoing potential concerns about funding and capacity need to be kept in mind.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Rating: Moderately Satisfactory

116. As asserted by the QALP-2 assessment of June 2010, the preparation and appraisal process appears to have been satisfactory on all counts, as were the management inputs. The financial management aspects were assessed comprehensively and incorporated into the overall project design. The task team considered all the pertinent risks and opportunities in designing the project.

117. With respect to the project outcomes, Bank staff were somewhat ambitious in framing the objectives. As discussed previously in this report, the third major objective of increasing enterprise competitiveness based on the scope of the project was not realistic. The linkages of the third component to the PDO of increased competitiveness were marginally relevant. The goal of increasing exports was not expected to materialize until sometime in the medium-to-long-run, however, by then it may be very difficult to identify the direct linkages to the project from among all other possible factors that may impinge on the development of the country’s exports. As described in the previous sections, the PDO indicator identified in the PAD – cost of regulatory compliance based on the Doing Business data – cannot be closely linked to results from the project interventions and thus provided an insufficient indicator by itself for measuring

29

the outcomes. On the other hand, the intermediate outcome indicators measured appropriately the project interventions.

(b) Quality of Supervision

Rating: Satisfactory

118. The Bank team brought special value to the borrower by bringing in internationally recognized experts who had held top positions in the field of metrology and quality standards in other countries and had considerable experience in the respective fields of expertise, to recommend solutions drawing upon external experience and to persuade counterparts who were resistant to reform. Also, the team arranged for twinning of CSM with the German Metrology Institute (PTB), which led to a considerable transfer of know-how, and of KAC with TurKak to twin the Kyrgyz agency for Accreditation. The team members in the field in particular made exceptional efforts to deal with a complex undertaking that called for revamping the legal, regulatory and institutional framework to move away from the mandatory approach, which was susceptible to rent seeking to a demand-driven approach.

119. The continuity of a well-qualified Task Team leader and team members in place since the beginning was instrumental in keeping the project on track in the face of the resistance to reforms. Management attention was very good at the design stage and remained satisfactory thereafter. Supervision reports and documents covered the situation adequately and regularly reflected progress towards intermediate indicators. The Bank addressed implementation delays through relevant actions, such as extensions and innovative ways to resolve procurement and bidding problems.

120. Some concerns were expressed by the Borrower in terms of delays in the approval of project expenditures on equipment and with the reallocation of funds. This partly was a reflection of the highly technical nature of the type of equipment purchased. Nevertheless, the implementing agencies were complementary of the prompt responses they received from the project team.

(c) Justification of Rating for Overall Bank Performance

121. Based on the two components of Bank performance as rated above, the overall rating is deemed as Moderately Satisfactory.

5.2 Borrower Performance

(a) Government Performance

Rating: Moderately satisfactory

122. The request for Bank financing of this project was initiated by the Government – recognizing the need for external expertise. The project had strong champions at the highest levels in the country including the Presidency, and relevant ministries, notably MITT / MoER. However, significant staff turnover at high policy levels in the Ministry of Economic Regulation presented a challenge to Project implementation.

30

123. During the political events of 2010 the lack of a permanent Government and the lack of clarity in the institutional structure of the Government disrupted the implementation of the Project. The lack of a functioning Parliament halted the adoption of a number or pending Technical Regulations; the volatile situation limited incoming travel of experts working on the Project; and made it impossible to deploy some of the additional required technical assistance.

(b) Implementing Agency or Agencies Performance

Rating: Satisfactory

124. After a slow start, the implementing agencies became more proactive in the implementation of the project. All Directors of the Implementing Agencies were very vocal and the technical staff was very well prepared.

125. Despite a high turnover of upper-level officials at MoER, the TR Department made good progress with the regulatory framework. During project implementation CSM had three directors. Initially, there was some resistance to the reforms supported by the project from CSM; however, these differences of opinion by the Director were subsequently resolved and this project component made good progress. The KAC Director was involved with the project from the start and the Agency was very committed to supporting the accreditation component.

126. The PIU, which was established under the MoER developed excellent knowledge of Bank procedures and acquired extensive practical experience in managing large and technical complex contracts.

(c) Justification of Rating for Overall Borrower Performance

127. Based on the two components of Borrower performance as rated above, the overall rating is deemed as Moderately Satisfactory.

6. Lessons Learned

Continuity of Bank staff in charge of project supervision is critical to the success of projects that operate in a challenging environment. The RTBET was a project whose small dollar size could have rendered it easily vulnerable to being crowded out by larger issues and projects in the country dialogue. The fact that it did not suffer this fate is due greatly to the dedication of the staff team. But the real systemic lesson is that it was the lasting continuity in the staff team – both the field office staff and the TTL who have remained with the project from the beginning – that made this dedication possible. At the same time, the high risk of the project makes it essential to have a strong field presence in such projects.

The linkage between project components, outputs and outcomes needs to be clearly identified at project design stage to arrive at more realistic Project Development Objectives. The connection between the project components of reducing the cost of regulatory compliance, improving the MSQT infrastructure and an increase in the share of non-gold trade with existing WTO partners may not have been as strong or clearly

31

justified as implied in the PAD. In this regard the design may have also aimed a bit high in terms of development outcomes. Given the small size of the project and its focus on MSTQ a narrower scope of objectives could have been considered, for example, just reducing the regulatory cost of compliance.

PDOs need to be supported by project components that are consistent with the results framework. In the case of the RTBET, the objective of enhancing enterprise competitiveness was supported by relatively small sub-components: the upgrading of laboratories that provided services to the agri-business/processed foods sector and establishment of a US$100,000 grant facility for companies that were seeking ISO certification to enhance the quality of their production process. Improving enterprise competitiveness is a very broad topic which did not fit with the very specific objectives of streamlining technical regulations and developing systems to enhance quality and safety of products.

An effective mechanism to jump-start institutional development of newly-formed entities and to speed up the learning curve is to organize twinning arrangements with well-known institutions that are committed to the sponsorship of best practices. The brokered twinning arrangements with internationally recognized peer technical agencies such as the one between CSM and German Metrology Institute (PTB) and KAC with the Turkish Accreditation body (TurKak) led to the establishment of excellent cooperation between these institutions.

Projects with complex technical components should be allowed more time for implementation or its principal components structured as separate consecutive projects. Given the high risk factor and technical complexity of the RTBET perhaps a longer period of implementation could have been considered as well as a larger amount of funding that would have provided the flexibility to finance additional equipment for laboratories that may not have been foreseen at the time of preparation such as in the case of the water laboratory, a natural resource with strong export potential, which was not considered until the second half of the Project.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies - Not applicable

(b) Co-financiers - Not applicable

(c) Other partners and stakeholders - Not applicable

32

Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$)

Components Appraisal Estimate

(US$ )5

Actual/Latest Estimate (US$)

Percentage of Appraisal

.1. Technical Regulations 1,482,00 928,278 63% Consulting Services 342,000 321,437 94% Goods 1,090,000 558,548 51% Training 50,000 48,293 96%.2. Metrology 2,238,700 2,34,426 105% Consulting Services 68,700 96,170 139% Goods 2,005,000 2,183,896 109% Training 165,000 154,360 94%.3. Accreditation 1,130,900 1,060,121 94%

Consulting Services6 250,900 127,230 51%

Goods 850,000 901,419 106% Training 30,000 31,472 105%.4.Project Management 207,700 373,478 180%

Consulting Services7 149,700 342,603 229%

Goods 31,000 10,727 35% Training 27,000 20,148 75% Total Baseline Cost 5,059,300 4.796,303 95%

Consulting Services 784,800 887,448 113%

Goods 3,976,000 3,654,589 92% Training 272,000 254,274 93%Physical Contingencies 0.00 0.00 0.00Price Contingencies 0.00 0.00 0.00Total Project Costs 0.00 0.00 Front-end fee PPF 0.00 0.00 .00Front-end fee IBRD 0.00 0.00 .00Total Financing Required 0.00 0.00

5 including co-financing of Kyrgyz Republic 6 including matching grant category 7including Operational Expenses category 8 this amount includes Consulting Services, Matching Grant and Operational Expenses categories and PPF funds

33

(b) Financing

Source of Funds Type of Co-financingAppraisal Estimate

(US$) Actual/Latest Estimate

(US$) Percentage of

Appraisal Borrower 58,600 82,400 102% IDA Grant 5,000,000 4,796,303 94%

34

Annex 2. Outputs by Component

1. This annex provides a summary of the main activities financed by the RTBET.

Component 1: Technical Regulations

2. This component strengthened the capacity of the public sector to develop new technical regulations based on the Law of the Fundamentals of Technical Regulations approved in 2004 to bring the country’s standards system in compliance with WTO standards.

3. The principal activities financed by the Grant were:

Technical Regulations (TR):

RTBET provided extensive capacity building support to the Ministry of Economy and other ministries and agencies in TR development. Throughout project implementation, approximately 200 public officials and technical specialists from various ministries and agencies were trained on the various aspects of Technical Regulations and MSTQ.

An international consultant was retained as a long-term resource to advise the government on how to set up an institutional framework and procedures to develop Technical Regulations.

Local consultants developed technical regulations (TRs) with emphasis on the food and agriculture sector. Based on the input provided by these consultants and other international experts the following TRs were produced:

Table A1: Status of Technical Regulations

Gov. / Technical

Industry Approved Parliament Drafting Review OtherAgriculture 2

Chemical 1 1 5

Construction 3 1

Electric power industry 3 1 1

Environmental protection 2

Fire Safety 1

Food 15 3 11

Fuel and gas 1 2 1

Machinery & Equip. Safety 2 1 2 1

Medical/Pharmaceutical 4 3

Metallurgy 1

Mining 2

Textile 1 2

Transport vehicles 2 1 1

Woodwork 1 1

Total 35 5 10 25 4

35

Note: each column depicts the status of the TRs across each industry. In addition to those approved, numerous TRs are in various stages of preparation and approval: those submitted to Government and/or Parliament for approval, those in the drafting stage; those under technical review; and those in other preliminary stages of preparation.

4. These regulations address multiple technical issues associated with quality and safety such as those noted in the examples below:

Food Requirements for production, storage, transportation, distribution and utilization of drinks

made from fruits and vegetables, bread, bakery products, confectionary, pasta, and alcoholic beverages.

Requirements for safety of raw materials for production of sugar, technological standards and environmental protection in the production of sugar.

General requirements for safety of drinking water. Requirements regarding the safety of products made by public catering companies.

Medical/Pharmaceutical Establish obligatory standards for medicines produced in drugstores. Safety requirements for medicines produced by pharmaceutical companies and other

healthcare providers operating in the Kyrgyz Republic.

Construction Minimum safety requirements for construction materials. General requirements for safety of design and construction of homes, buildings and other

commercial and industrial structures.

Chemicals Safety requirements for production of chemicals. Safety requirements for production, storage, transportation, and utilization of poisonous

substances and of toxic and highly toxic substances.

Legal Metrology

5. Provided assistance with the spin-off of Legal Metrology Inspectorate from the previous NISM into an independent regulatory agency.

6. Introduction of Risk Based Inspection regime. Financed local and international consultants to develop the basic principles of risk-based inspections and training in surveillance techniques.

Integrated Information System

7. Installed an Information System linking various regulatory and supervisory agencies in support of TR development and enforcement. The IT system supports the documentation flow between government agencies and ministries involved in the preparation and implementation of TRs. It allows multiple agencies and stakeholders to collaborate on the drafting of TRs providing

36

a clear track of what was sent to whom and when. The IT system also consolidates data and information on the implementation of the TRs such as supervisory activity and enforcement actions.

WTO Enquiry Point

8. Establishment of a fully functional WTO Enquiry Point to channel requests and information to and from potential trading partners including purchase of hardware, software and telecommunications equipment. Project resources facilitated the development of a website for the Technical Regulation Authorized body with an interface with and support for the Technical Barrier to Trade (TBT) Enquiry Point.

9. The Ministry of Economy is now in full compliance with the TBT WTO notification requirements thanks to the establishment of a WTO Enquiry Point designed to facilitate information exchange and transparency amongst international WTO trading partners.

Training

10. Participation of Experts in Relevant International and Regional Fora (e.g.: WTO, ISO, FAO). Kyrgyz officials participated in meetings of the working groups of UNECE on policy issues in the field of standardization and norm regulation, in special meetings of the TBT WTO, and also in the various Commissions of EurAsEC, ISC, CIS, the scientific and technical commissions on technical regulation, sanitary, veterinary and phyto-sanitary measures, surveillance, accreditation, metrology and certification.

PHRD Grant

11. The proceeds of the TA Grant were utilized to retain international experts to perform needs assessment against international standards as well as training in the following areas:

Legal Metrology Market Surveillance (inspection system) in consideration of the new Technical

Regulations Microbiology: provided training to testing laboratories and industry on the microbiology

and scientific aspects of food safety Information Technology: requirements and technical specification for the WTO Enquiry

point Accreditation of Testing Laboratories: TA to testing laboratories and initial assessment of

testing laboratories against international standards for accreditation.

Component 2: Metrology and Standards

12. This component strengthened the capacity of the Center for Standards and Metrology (CSM) to perform its functions in the area of metrology, standards and calibration.

37

13. Developed twinning arrangements between internationally recognized organizations such as the German Metrology Institute (PTB) and CSM in order to maximize ongoing professional coaching, training, and peer exchanges. PTB provided critical support to CSM in the definition of technical specifications for the procurement of laboratory facilities. For example, PTB presented to CSM the specification of the metrological equipment on temperature and mass, the specification on construction works and the control system of the laboratory environment.

14. A total of 56 standards were procured and translated in support of harmonization of the Kyrgyz domestic standards system with international standards.

15. Installation of metrology equipment for seven laboratories in the following areas: length, volume, pH-meter, density, pressure, time & frequency, and viscosity. These laboratories will house the national etalons or reference measurement units critically needed to develop the country’s export potential. Resources accounting for about 40 percent of Grant funding were used to purchase metrology and calibration equipment for the laboratories and to upgrade the physical location to house the etalons.

Training

16. Technical experts from CSM participated in meetings of СООМЕТ Committee on etalons, in the various technical committees on questions of mass, flow-metry, standard samples, physics-chemistry, thermometry and thermo-physics, and a quality forum.

17. Technical experts also participated in laboratory inter-comparisons of national etalons.

Component 3: Accreditation and Quality Enhancement

18. Development of a modern domestic accreditation, testing and certification infrastructure capable of providing reliable and trusted services to companies and consumers.

Kyrgyz Accreditation Center

19. An independent national accreditation body, the Kyrgyz Accreditation Center (KAC) was established in November 2006. From its start, KAC drew on the technical competence and knowhow of the former Accreditation staff of the NISM. Resources from the project were utilized to establish the new entity, to purchase IT equipment and furniture for the KAA office and development of its web-site, and to acquire Etalons and materials for inter-laboratory comparisons.

20. In order to facilitate its institutional development, KAC implemented a Twinning Arrangement with TURKAK, the Turkish Accreditation Body. This arrangement provided technical assistance in a number of areas including gap analysis and performance assessment; training on various ISO procedures and requirements; and assisting KAC with the preparation of internal guidelines and procedures.

38

21. In July 2009, KAC applied for full membership in the International Laboratory Accreditation Cooperation (ILAC)9. After completion of a pre-peer review process, an ILAC expert delegation visited Bishkek in September 2012 for a full audit to assess the strengths and weaknesses of the Kyrgyz Accreditation system. The KCA has been responding to the initial findings of the ILAC mission and the results of the final ILAC audit are expected to be received by the end of 2013.

Laboratories

22. Out of approximately 80 laboratories that have been accredited according to national requirements the project worked only with the Epidemiology center under the Ministry of Health, where the project equipped three laboratories: microbiology, chemicals and radiology. The selection of these labs was based on the list of targeted industries identified in the PAD. In addition, at inception, the RTBET Project retained two international experts on testing laboratories and certification bodies to run a nation-wide assessment so as to select those testing laboratories and certification bodies that would have benefited the most from any technical and financial assistance to be provided under the RTBET Project. The labs provide services to agribusinesses such as those involved in production of fruits and vegetables, juices and drinks, meat and meat-products, milk and dairy products.

23. On international accreditation of selected testing laboratories and certification bodies, two qualified international auditors were retained under the Project to carry out a pre-audit readiness assessment against the relevant ISO standards for international accreditation, namely ISO 17025 for testing laboratories and the new ISO 17065 for certification bodies. The international experts assessed the readiness level of testing laboratories ranging from 90 percent to 95 percent depending on the type of testing equipment and procedure. The identified gap lies in the need to perform laboratory inter-comparisons to ensure consistency of test results.

Certification Bodies

24. One certification body has applied for international accreditation. The project financed the services of an international expert who assessed the readiness level to be at 75 percent with the main gap lying in the translation into English of guidelines and internal documentation.

Matching Grant Facility for Quality Improvement by Private Sector Companies

25. At present, 12 companies in a wide range of sectors did acquire international certification of quality management systems (i.e. ISO). Of these, three only are in the agribusiness / foodstuff sector, with the remainder in manufacturing (cement, bulbs) and services. Of these companies, just two benefitted from the project’s Matching Grant Facility.

9 Accreditation bodies around the world, which have been evaluated by peers as competent, have signed an arrangement that enhances the acceptance of products and services across national borders. The purpose of this arrangement, the ILAC Arrangement, is to create an international framework to support international trade through the removal of technical barriers. The ultimate aim of the ILAC Arrangement is the increased use and acceptance by industry as well as regulators of the results from accredited laboratories and inspection bodies, including results from laboratories in other countries. In this way, the free-trade goal of ’product tested once and accepted everywhere' can be realized.

39

26. The trend in acquisition of voluntary quality standards (ISO) by firms has been slow suggesting that companies are not yet investing in quality or aiming to expand into new (quality sensitive) WTO markets. Many Kyrgyz companies indeed appeared to have competing investment priorities as the international economic crisis and domestic social upheavals required companies, especially smaller ones, to focus on maintaining their internal market shares and keep afloat as going concerns rather than investing in quality systems in support of market expansion and exports. In addition, smaller yet high export potential enterprises (e.g., textile/apparel) tend to shy away from exploring new markets given their limited production capacity.

40

Annex 3. Economic and Financial Analysis Selected Indicators 2006 2007 2008 2009 2010 2011 2012*

GDP and Prices

GDP growth (annual %) 3.1 8.5 8.4 2.9 -0.5 6 -0.9

Inflation, CPI (annual %, end of year) 5.1 20.1 20.1 0 19.2 5.7 7.5

External sector

Current account bal. as % of GDP -3.1 -6.2 -15.5 -2.5 -6.4 -6 -10.2

Exchange Rate/USD (average) 40,16 37,31 36,57 42,89 45,96 46,14 47,0

Intl. reserves: months of imports 3,0 3,2 4,0 4,9 4,1 3,8 3,8

Public Sector Deficit

Overall Fiscal Balance (% of GDP) -2.1 -0.3 0 -3.5 -6.3 -4.6 -6.1* preliminary

Source: Government of the Kyrgyz Republic and World Bank

41

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Hormoz Aghdaey Sector Manager EASFP Anarkan Akerova Counsel LEGCF Sylvie K. Bossoutrot Senior Operations Officer ECSPF Rochelle Hilton Consultant EASPR Carlo Segni ET Consultant ECSPF Lorenzo Costantino Consultant ECSF2 Mauricio Frota MSTQ Specialist ECSPF Naushad A. Khan Consultant ECSO2 Hannah M. Koilpillai Senior Finance Officer CTRFC-His Talaibek Torokulovich Koshmatov

Senior Rural Development Specialist ECSAR

Kathy Lalazarian Senior Public Sector Specialist LCSPS John Otieno Ogallo Sr Financial Management Specialist ECSO3

Supervision/ICR Lorenzo Costantino Consultant ECSF2 Irina Goncharova Procurement Specialist ECSO2 Anarkan Akerova Legal Counsel LEGEC Suerkul Abdybaly Tegin ET Consultant ECSPF Nurbek Kurmanaliev Procurement Specialist ECSO2 John Otieno Ogallo Sr Financial Management Specialist ECSO3 Nurlanbek Tynaev Consultant ECSPF Shirin Imanbaeva Team Assistant ECCKG (b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks US$ thousands (including travel and consultant costs)

Lending FY04 0.88 13.21 FY05 15.23 144.21 FY05 (TF funding)* 0.0 31.30 FY06 31.93 90.42 FY06 (TF funding)* 0.0 5.83 FY07 1.9 8.16

FY07 (TF funding)* 0.0 2.33 Total: 49.94 295.46

42

Supervision/ICR FY07 (TF funding)* 0.0 29.88 FY07 26.14 69.33 FY08 17.43 66.22 FY09 26.58 88.92 FY10 29.04 99.28 FY11 12.56 105.07 FY12 5.36 83.42 FY13 0.33 71.94

FY14 0.0 20.55 Total: 117.44 634.60

*The TF funding includes financing by the following grants: TF030614 , TF030761, TF030843

43

Annex 5. Beneficiary Survey Results

Not Applicable

44

Annex 6. Stakeholder Workshop Report and Results See link: http://www.worldbank.org/en/news/press-release/2013/04/15/world-bank-project-increases-international-visibility-of-kyrgyz-republic-as-wto-member

45

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

Kbi.PrbiJ PECHYGJIJ(KACbllfhlll

:)KOHOMlDV\

MU RliCTP JJJinl

~CTEPCfBO

3KOH0Mlfl0i Kb[Pri>IJCKOii

PECUY&.IlliKH

MJJ\1STRY OF~ ECONOMY OP TEIF. KYRGYZ R.£Pl/BU C

72COQ2.. K .. n..3 P«<r)~ l>~~nll~C~e~~: 111_ t.fyiii npoene~m~. 106 TV!.: 1"996(_) J2)62-0.S..90, +atcc: +4)916(] 12)66.-11-37; ti.p,A~ MO*fll!.: +996(}12)62.0.S.. )j d<*'lr. -w.mineoooOm.k& Uftail: mailgmillc«:!nc:m.t • S..1t: 129051. i1HH:Ot 10ol200?10102

20IJ ,.. ;JO .---"~~~x. /s-t(9tt.fJ 20 __ ,._ . • )(, ____ _

120002. Xwpl"bll('CCJJ Pec:ft)'6.n..x:. r, Muuu.. lllf)!X'or«T 'ly~. 106 TU.: +996 (li2)62·0S.90. +axe: - 996 () 12)66-1$·)7; cAMtU.til! -+996() 12}62.o:S·l5 at:6-ca.rtr. - mincC>CllnOal.kg e-fnlit qil@m~ 6d: l290S2, H.HH=Otll)4200710 102

r ,usc o<jlu ca Bcc>mpaoro OauKa s Kbtpn.13CKon P«ny6.1JIKC

r-ny Anet<cau; tpy KpcMcpy

Yaa~CMiolii rocnO}tUU KpeMe.p,

HaCTOJIIltHM, OTHOCIITCJJbHO Or'iCTa o 3ascp weHHH peamt:JaUH« 11 pe3ym>raTax npoeKTa

«CoKpSUICHHe TeXHHqCCJ<IIX 6apbepo8 Ope.!lflpiiHIIMaTeflbCTIIY H TOpf08Jlel>, coOOlUlUO CJlCJ1}'10UlCC.

B pa."l<3x npoek-ra nposc.nellhl pa6oTbJ no HHCTHTYUHOH8Jlbso-crpy~<cypllbJM npeo6p3306aHHliM B CllCTCMe TCXHHqCCKOrO pery!1Hp01131iHJI ll MeTpo.~OrH H, 8 TaK>t<e 8

ycuneHfUi noTeEIUH3Jia fOCyAapcroeHliOrO CCh.lOpa n p33pa6oTKe TeXRH4e<:KMX pern3MeHTOB,

K3K HOBblX OCHOB TeXHH'!eCKOrO p eryJJllpo113HHJI 8 YCJ10811JIX pbiHO'iHOil 3KOHOMJIJ01 8

COOTBeTCTBnu c 3SKOHOM KP «06 ocaoll!lx TeXHJJ'iecKoro perynupo118HHJI e KP». C UCJlbiO p838JJTI1• HHcj>pSC'TJlYI<TYPhl 1<3qCCT88 8 l(eHTJlC CTaHJl3pTH38Uil M H MCTpOJJOnUJ

n pu MDHHCTepcTIIe 3KOHOMHKH KblprbocJ<o<i Pecny6JnJKe npose.neHbl pa6<rn.1 no

nepe(>CH8UICHHIO H peMOHTY Jl300paTOpHil cj>H3HXH-XI!MIIll, .!l38.~CHliR, JUUIHbl, pa.!li!O II

3!1enpuqecTBa. Opno6pereubl u yCTSHOsncHbl nep 8H.,HbJe :na.•o""' If Me..-ponorn .. cc~<oe o6opy,noii8Hue .!l!IR na6opaTopn1i M3CCbl If TeMnepazypbl. Bnepnble 8 pecny6mJKe co3,naHa

JJa6op8TOpHJI no nepe,naqe 11 CIIHXpoHH33UIJH CIITHaJ108 TO'iHOfO 8peMeHII. TaJ<"&e 33J<)'JIJJeHbl

>tClK,I\)Ili3J)OJJJ1biC CTaH.!l3pTbl H3 pyCCKO,_. Jl3b1 KC.

YcuneH DOTCHUH8JJ UeuTpa aKKpe.!lHTaUJJH n pH MIJHHCTepcTse 3KOHOMHKH l<blprb13CJ<Oii

Pecny6nnKe (KllA npH M3 KP) 11 nposc.neubt pa6oTbl no eczynneH>ftO 8 MCl<GlfllSpo.!lHytO

o pral UI3aUH!O no 3KKpeAHT8UHJt rLAC. B OI<TR6pe 2012 rOIUl npoaeneu MClK.!lYH3po,!lH blii

ay,!lHT, H 8 IJ3CTORUlCC BpeM• O>t<ll}laeTCR peWCH JIC MelK.!lYil3potlHOil OpraJU133UHII n o

aKKpe.!lHTaUIIH !LAC o npHHJITil>t KUA n p H M3 KP KaK nonaonpa8uoro 'L1eua ILAC, 'iTO

no3110nt!T npJOH8118Tb pe3yJJhT8Tbl OJlCHKH COOTilCTCTIHIR, noJJY'ICHHLie n Khlpn.ocKoil

Pecny6mtKe, Ha Japy6e>t<Hbtx pbDIKax.

Tax>~~" o 6p330M, npu co.neiiCTBnll npoeKTa Bce,mpHoro 6311<3 «COKpauteHue

TCXHJI'ieciOIX 6apbep08 8 Dpe.!lflpHHH!of8TCJJ bCTBe H TOpf08Jle>> 6bUJ3 npo,nenaua 6 0JJbWaJI

pa60Ta no pe4Jop.1e CJICTeMbl TCJ<HHqCCKOTO pery!1Hp088HIIR II p83BHTJIIO H81.1HOH<l<lbHOil

IIHcj>paCTpyKzyphl K8'iCCTBa.

npoei<T 3aBCpWHJJCJJ ycneWHO H nO 11pe.,nCT3U.1CIHIOMY 0rqery 0 3311CpWeH11H

pe8!1H33UHH U pe:lynhT8Tax npoexra «CoKp3WCH.IJC TCX HJI .. CCKI!X 6apbepoo

npe.nOpiiHHMSTenl.cTily JoJ TOproiiJle» 'UlHHCTepcTBO 38MC'iaHltil I! Dpe.!l!IO>KCHliH He HMCCT.

46

BMecre c TeM, na"aTbie 8 c<!Jepe TeJ<Im"ecKoro perynnpooannll pe<!JopMbi -rpe{)y!OT CROrO llpQAOfl)f(CHHSI, B ttaCTH:

• npHusmur TeXHH\feCKI1_X perttaMCHiOll no 8HCJI.peHtuo CHCTCMbl MCHC.lliKMCHTa t<8'1CCTBa

u t5ewnacHocnt npOJIYKU!!ii ua npe.anp!1J!nt8X: pecny6llHK_ij; • ).lOOCHau:(CHHCM MCTpOJlOflf'tCCKifX }I HCDb!TaTenbH.blX na6opaTOpltil;

3KKpe.ZU1TBUHH OTo6p31lllbiX HCllbl'r3TCJlbllhi X na60p3T0pHH II Oprauoo DO

cepm<jmKBUHH na Mell<.!lynapOAHOM ypo8n e .

B CUII3H c <JCM, MHHHCTepCTBO 3KOIIOMHK11 KLlpfbi3CKOH l'ecny6ni1KH npoCIIT

p3CCII•10rpeTb BOlMOlKIIOCTb OTKpbiTlle HODOro npoeKTa JlH60 DJlOJ.IOJlli<CHHJI n poeKTa

«COJ\pall.lCHHC TCXHII'JCCKHX 6apbCpOB OpeJIJl pHHifMaTCJibCTB)I H TOprOKJlC>>.

nonOli<"TCJlhHOC pCUJCHHC ll3HH()f()o BOllJlOC3 J.laCT B03MOlKI!OoCTb JlMbiiCfiWCfO

)IJI}'"CUeHJUI UH<!JpaCTp}'KT}'pbl K3,ecTBa 8 pecny6JUtKe, 'f'l'O 6yACT COOC06CTBOB3Tb OOBbWlCHJIIO

3KCOOpTHOfO OOTCUUHM3 II palBIITI!IO 3KOHO~fHKII CTp3Hf>l.

B••paJKaiO cooro npnlHaTenbHOCTb n 11:1.1\CIOCb Ita AMhneiiwee rutOJ.IOTBopuoe

CDrp)IJ.IHli'ICCTBO.

c YB3.it\CHU CM,

CraTC-cct<perapb

ucn.:C. Op.'tf()fffl(J(J cmiJeJI ?COC mu. :610JJS4-SOJ9

A. KypM3H083

47

Unofficial translation

Ministry of Economy of the Kyrgyz Republic

September 30, 2013 # 15-2/9453

Mr. Alexander Kremer Country Manager World Bank Country Office in the Kyrgyz Republic

Dear Mr. Kremer,

Let me use this letter to share details about draft Implementation Completion Report produced under Reducing Technical Barriers for Entrepreneurship and Trade Project.

Project implemented the number of structural and institutional adjustment activities in the technical regulation and metrology area, and improved government’s capacity to contemplate technical regulations that serve as the basis of technical regulation in the market economy as per the Law of the Kyrgyz Republic “On Basics of Technical Regulation in the Kyrgyz Republic.”

To refine existing quality assurance infrastructure of Center for Standardization and Metrology (CSM), Project refurbished and re-equipped laboratories performing physical, chemical, pressure, length, radio, and electricity tests. In addition, Project procured and installed primary standards and metrological equipment for mass and temperature laboratories. For the first time ever the country commenced a laboratory for transmission and synchronization of standard time signals. As a part of procurements, CSM acquired the Russian translation of International Standards.

Among other things, Project strengthened capacity of Kyrgyz Accreditation Center under Ministry of Economy and assisted with accession to International Laboratory Accreditation Cooperation (ILAC). An international audit was held in October 2012 and presently ILAC is deciding on the full membership of Kyrgyz Accreditation Center under Ministry of Economy of the Kyrgyz Republic that would allow foreign markets to recognize KR conformity assessments.

Therefore, World Bank’s Reducing Technical Barriers for Entrepreneurship and Trade Project facilitated outstanding reforms in technical regulation and development of national quality assurance infrastructure.

Project was successfully completed and Ministry of Economy has no comments or proposals with regard to Implementation Completion Report.

Please note that MSTQ reforms require a follow-up in:

48

- Adoption of technical regulations on implementing quality and safety product management systems by KR enterprises;

- Further equipping metrological and testing laboratories;

- Modernization of quality infrastructure in the regions of the country;

- International accreditation of selected testing laboratories and certification bodies.

That being said, Ministry of Economy is cordially requesting to consider a new project or RTBET project continuation.

Continuation of the project efforts would help further improvement of the quality infrastructure in Kyrgyzstan and reinforce its export and economic capacities.

Let me express my warmest regards and hope to carry on our fruitful cooperation.

Sincerely,

Aidai Kurmanova,

Permanent Secretary of Ministry of Economy.

49

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

Not Applicable

50

Annex 9. List of Supporting Documents

1. Project Appraisal Document

2. Development Grant Agreement and amendments

3. CAS and CPS

4. ISRs from 2005 - 2013

5. Supervision mission back-to-office reports / Aide Memoires

6. Miscellaneous project documents (posted on the WBDocs)

TT ii aa nn SS hh aa nn

KK uu nn gg ee yy -- AA ll aa tt aa uu MM tt ss ..

Peak PobedyPeak Pobedy7439 m7439 m

B AB AT K E NT K E N

TTA L A SA L A S

J A L A L - A B A DJ A L A L - A B A D

C H U I C H U I

I S S Y K - K U L I S S Y K - K U L

O S H O S H

N A RN A R Y NY N

SulyuktaSulyukta

Kyzyl-KiyaKyzyl-Kiya

Tash-KumyrTash-Kumyr

Kara-Kul Kara-Kul

At-BashyAt-Bashy

KirovKirovKara-BaltaKara-Balta Tokmok Tokmok

ChaekChaek

Barskaun Barskaun

Kara-Say Kara-Say

Sary-TashSary-Tash

Daraut-Daraut-KorganKorgan

Gul'chaGul'cha

ToktogulToktogul

TunukTunuk

TyupTyup

EnilchekEnilchek

ShyirakShyirak

OshOsh

Jalal-AbadJalal-Abad

BatkenBatken

NarNarynyn

TTalasalasKarakolKarakol

B AT K E N

TA L A S

J A L A L - A B A D

C H U I

I S S Y K - K U L

O S H

N A R Y N

BISHKEKBISHKEK

Sulyukta

Kyzyl-Kiya

Tash-Kumyr

Kara-Kul

At-Bashy

KirovKara-Balta Tokmok

Balykchi

Chaek

Cholpon-Ata

Barskaun

Kara-Say

Sary-Tash

Daraut-Korgan

Gul'cha

Toktogul

Tunuk

Tyup

Enilchek

Shyirak

Osh

Jalal-Abad

Batken

Naryn

TalasKarakol

BISHKEK

T A J I K I S T A N

UZBEKISTAN

C H I N A

K A Z A K H S T A N

Chatkal

Chu

Naryn

Aksay

Kyzyl Suu

Kurshab

Lake Issyk-Kul

ToktogulReservoir

Lake Sonkul

LakeCharyi-Kel'

To Panfilov

To UshtobeTo

Burylbaytal

To Shymkent

To Tashkent

To Bukhoro

To Murghab

To Hotan

T i a n S h a n

K u n g e y - A l a t a u M t s .

Peak Pobedy7439 m

72E 74E 76E 78E 80E

70E 72E 74E 76E

78E 80E

44N

42N42N

40N40N

KYRGYZREPUBLIC

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

0 25 50 75

0 25 50 75 Miles

100 Kilometers

IBRD 33430

SEPTEMBER 2004

KYRGYZ REPUBLICSELECTED CITIES AND TOWNS

OBLAST CAPITALS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

OBLAST BOUNDARIES

INTERNATIONAL BOUNDARIES