Document of The World...

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004177

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IDA 58180, 52420, 43500)

CREDIT

IN THE AMOUNT OF SDR 45.4 MILLION

(US$ 68.1 MILLION EQUIVALENT)

TO THE

PEOPLE’S REPUBLIC OF BANGLADESH

PUBLIC PROCUREMENT REFORM PROJECT II ( P098146 )

December 19, 2017

Governance Global Practice

South Asia Region

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

CURRENCY EQUIVALENTS

(Exchange Rate Effective November 20, 16, 2017)

Currency Unit = = Bangladesh Taka (BDT) BDT 78 = US$1

US$1.50 = SDR 1

FISCAL YEAR

July 1 ‐ June 30

ABBREVIATIONS AND ACRONYMS

AF Additional Financing AF II BCCP

Additional Financing II Bangladesh Center for Communication Programs

BIM Bangladesh Institute of Management BIGD BRAC University Institute of Governance and Development BREB Bangladesh Rural Electrification Board BWDB Bangladesh Water Development Board CAS Country Assistance Strategy CIPS CPF

Chartered Institute of Procurement and Supply/ U.K. Country Partnership Framework

CPI CPTU

Corruption Perception Index Central Procurement Technical Unit

CSM DLI

Civil Society Members Disbursement‐Linked Indicator

e‐GP Electronic‐Government Procurement e‐PMIS Electronic‐Procurement Management Information System ECoP Environmental Code of Practice ESCB Engineering Staff College Bangladesh FI FM

Financial Institutions Financial Management

GCF Government Contractors Forum GOB Government of Bangladesh HQ Headquarters ICR Implementation Completion and Results Report ICT Information and Communication Technology IMED Implementation, Monitoring, and Evaluation Division ISM Implementation Support Mission ISR Implementation Status and Results Report LGED Local Government Engineering Department M&E Monitoring and Evaluation MCIPS Member of The Charted Institute of Procurement and Supply MIS Management Information System

NCB National Competitive Bidding NSAPR National Strategy for Accelerated Poverty Reduction PAD Project Appraisal Document PDO Project Development Objective PE PPA

Procuring Entity Public Procurement Act

PPR Public Procurement Rules/Regulation PPRP Public Procurement Reform Project PPRPII Public Procurement Reform Project II PPSC Public‐Private Stakeholders Committee PROMIS Procurement Management Information System RHD Roads and Highways Department SRGB Survey Research Group Bangladesh

Regional Vice President: Annette Dixon

Country Director: Qimiao Fan

Senior Global Practice Director: Deborah L. Wetzel

Practice Manager: Felipe Goya

Task Team Leader(s): Zafrul Islam

ICR Main Contributor: Paul J. Kaiser

TABLE OF CONTENTS

DATA SHEET ............................................................................................................................ 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................ 6

A. CONTEXT AT APPRAISAL ........................................................................................................... 6

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................. 9

II. OUTCOME ...................................................................................................................... 12

A. RELEVANCE OF PDOs .............................................................................................................. 12

B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................ 13

C. EFFICIENCY ............................................................................................................................. 19

D. JUSTIFICATION OF OVERALL OUTCOME RATING ..................................................................... 20

E. OTHER OUTCOMES AND IMPACTS .......................................................................................... 20

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21

A. KEY FACTORS DURING PREPARATION..................................................................................... 21

B. KEY FACTORS DURING IMPLEMENTATION .............................................................................. 22

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................. 23

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25

C. BANK PERFORMANCE ............................................................................................................. 26

Bank Performance Rating: Satisfactory ....................................................................................... 28

D. RISK TO DEVELOPMENT OUTCOME ........................................................................................ 28

V. LESSONS AND RECOMMENDATIONS .............................................................................. 28

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................ 33

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42

ANNEX 3. PROJECT COST BY COMPONENT............................................................................. 44

ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................ 45

ANNEX 5. DISBURSEMENT LINKED INDICATORS..................................................................... 47

ANNEX 6. BORROWER, CO‐FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 50

ANNEX 7. SUPPORTING DOCUMENTS .................................................................................... 52

The World Bank Public Procurement Reform Project II ( P098146 )

DATA SHEET

BASIC INFORMATION Product Information

Project ID Project Name

P098146 PUBLIC PROCUREMENT REFORM PROJECT II ( P098146 )

Country Financing Instrument

Bangladesh Technical Assistance Loan

Original EA Category Revised EA Category

Not Required (C) Not Required (C)

Related Projects

Relationship Project Approval Product Line

Additional Financing P132743‐Additional Finaning of Public Procurement Reform Project II

09‐May‐2013 IBRD/IDA

Additional Financing P158783‐Second Additional Financing to Public Procurement Reform Project II

06‐Jun‐2016 IBRD/IDA

Organizations

Borrower Implementing Agency

People's Republic of Bangladesh, Economic Relations

Division (ERD)

Implementation Monitoring and Evaluation Division

(IMED), Central Procurement Technical Unit (CPTU)

Project Development Objective (PDO) Original PDO

The Project Development Objective (PDO) is to improve performance of the public procurement system progressively in Bangladesh, focusing largely on the target agencies. The PDO would be achieved by strengthening the ongoing reform process and moving it further along with the following outputs: (i) enhanced capacity in creating a sustained program to develop skilled procurement professionals, (ii) strengthened management and monitoring of procurement in target agencies, (iii) introduction of e‐government procurement in CPTU and the target agencies on a pilot basis, and (iv) creation of greater public awareness of a well functioning public procurement system by engaging civil society, think tanks, beneficiaries, and the private sector. All these actions are key elements in effective implementation of the procurement law/ regulations.

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing IDA‐43500

23,600,000 23,600,000 24,041,237

IDA‐52420

34,500,000 34,500,000 32,446,437

IDA‐58180

10,000,000 10,000,000 9,638,638

Total 68,100,000 68,100,000 66,126,312

Non‐World Bank Financing

Borrower 1,300,000 4,800,000 3,610,000

Total 1,300,000 4,800,000 3,610,000

Total Project Cost 69,400,000 72,900,000 69,736,312

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

05‐Jul‐2007 12‐Sep‐2007 09‐Dec‐2010 31‐Mar‐2013 30‐Jun‐2017

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

09‐May‐2013 23.62 Additional Financing

29‐Sep‐2015 39.94 Change in Disbursements Arrangements Other Change(s)

06‐Jun‐2016 45.26 Additional Financing

KEY RATINGS

Outcome Bank Performance M&E Quality

Satisfactory Satisfactory Substantial

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 28‐Mar‐2008 Moderately

Unsatisfactory Moderately Unsatisfactory 0

02 30‐Sep‐2008 Moderately Satisfactory Moderately Satisfactory 1.67

03 26‐Mar‐2009 Moderately Satisfactory Moderately Satisfactory 2.11

04 01‐Oct‐2009 Moderately Satisfactory Moderately Satisfactory 3.46

05 23‐Dec‐2009 Moderately Satisfactory Satisfactory 3.69

06 14‐Jun‐2010 Moderately Satisfactory Moderately Satisfactory 4.73

07 30‐Apr‐2011 Satisfactory Satisfactory 8.43

08 20‐Aug‐2011 Satisfactory Satisfactory 9.61

09 07‐Jan‐2012 Satisfactory Satisfactory 10.51

10 07‐Jul‐2012 Satisfactory Satisfactory 12.49

11 18‐Jun‐2013 Satisfactory Satisfactory 15.22

12 29‐Dec‐2013 Satisfactory Satisfactory 18.10

13 27‐Jun‐2014 Moderately Satisfactory Moderately Satisfactory 19.66

14 22‐Dec‐2014 Moderately Satisfactory Moderately Satisfactory 22.10

15 22‐Jun‐2015 Satisfactory Satisfactory 23.36

16 01‐Feb‐2016 Satisfactory Satisfactory 26.77

17 06‐Jul‐2016 Satisfactory Satisfactory 31.77

18 30‐Dec‐2016 Satisfactory Satisfactory 39.63

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Public Administration 100

Central Government (Central Agencies) 80

Sub‐National Government 10

Information and Communications Technologies 100

Other Information and Communications Technologies 5

Social Protection 100

Social Protection 5

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 10

Public Private Partnerships 10

Public Sector Management 87

Public Finance Management 15

Public Expenditure Management 15

Public Administration 72

Administrative and Civil Service Reform 14

Transparency, Accountability and Good Governance

Social Development and Protection 14

Social Inclusion 14

Participation and Civic Engagement 14

ADM STAFF

Role At Approval At ICR

Regional Vice President: Annette Dixon Annette Dixon

Country Director: Johannes C.M. Zutt Qimiao Fan

Senior Global Practice Director: Deborah L. Wetzel Deborah L. Wetzel

Practice Manager: Felipe Goya Felipe Goya

Task Team Leader(s): Zafrul Islam Zafrul Islam

ICR Contributing Author: Paul J. Kaiser

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context

1. At the time of appraisal of the Public Procurement Reform Project II (PPRPII) in 2007, Bangladesh was one of the least developed countries in the world, with 40 percent of the population living below the national poverty line.1 Despite considerable progress in reducing poverty since the country became independent in 1971, inefficient and ineffective governance practices prevented Bangladesh from achieving its full economic potential and accelerating the pace of poverty reduction. Inefficiencies in public procurement practices, poor financial management (FM), and corrupt practices were identified as key challenges that undermined good governance in the country. Vested interests in maintaining the status quo were strong, and there was initial reluctance by the political leadership to address these issues.

2. The World Bank 2002 Country Procurement Assessment Report, which was broadly endorsed by the Government of Bangladesh (GOB), identified several deficiencies in public procurement practices. They included the absence of a sound legal framework governing public sector procurement, protracted bureaucratic procedures, a lack of professionals to manage public procurement, and the absence of transparency and accountability mechanisms. With World Bank support through a credit Public Procurement Reform Project (PPRP), the GOB began addressing these issues in 2002 by establishing the Central Procurement Technical Unit (CPTU), issuing new public procurement regulations in 2003, and enacting a Public Procurement Law in 2006 based on international best practices. Despite these initial accomplishments, there was limited progress in achieving improved procurement outcomes. Ongoing challenges included inconsistent implementation of procurement regulations across government agencies, substantial delays in the issuance of contracts, and inappropriate bidding practices.

3. The World Bank’s 2006 Country Assistance Strategy (CAS), which was aligned with the GOB’s 2005 National Strategy for Accelerated Poverty Reduction (NSAPR), identified governance as the major constraint preventing Bangladesh from improving its investment climate and empowering the poor. The NSAPR specifically highlighted the need for enhancing the efficient use of resources, promoting transparency and accountability in resource use, and tackling corruption. PPRPII was designed to address these issues, specifically focusing on strengthening the public procurement system at the key sector ministries/agencies, a key element of good governance.

Theory of Change (Results Chain)

4. The four clusters of activities and outputs in figure 1 correspond to the four project components described in the original PPRPII, Additional Financing (AF), and AF II Project Appraisal Documents (PADs), including the intermediate results/outcomes presented in the Results Frameworks and Monitoring Plans. The project focuses mainly on the four target agencies of the key ministries that spend about 50 percent of the annual development program of the GOB: Roads and Highways Department (RHD), Local Government Engineering Department (LGED), Bangladesh Water Development Board (BWDB), and

1 See: https://data.worldbank.org/indicator/SI.POV.NAHC?locations=BD.

Bangladesh Rural Electrification Board (BREB). The theory of change in the original and subsequent projects identified the following assumptions for project success: (a) capacity of the CPTU, the implementing agency, to implement this operationally complex and politically challenging project; (b) sufficient Internet connectivity across the country to ensure fair and equal access by bidders/contractors to electronic‐government Procurement (e‐GP); and (c) political will of GOB leadership to support the public procurement reforms required for improving the use of public resources with greater efficiency, transparency, and accountability.

5. During implementation, the CPTU continually enhanced its operational capacity, the GOB effectively rolled out Internet connectivity across the country, and the highest levels of GOB leadership supported the reform agenda associated with PPRPII after initial reluctance. This was the evolving context for the theory of change described in Figure 1.

Figure 1. PPRPII Theory of Change/Results Chain at Appraisal

Note: BIM = Bangladesh Institute of Management; CIPS = Chartered Institute of Procurement and Supply; ESCB = Engineering Staff College Bangladesh; GCF = Government Contractors’ Firm; PDO = Project Development Objective; PROMIS = Procurement Management Information System.

Project Development Objectives (PDOs)

6. The original PDO, as stated in the Legal Agreement, was “to strengthen the Recipient’s public procurement system, with a particular focus on improving the performance of Target Agencies, through

(a) enhancing capacity by creating a sustained program to develop skilled procurement professionals; (b) strengthening management and monitoring of the procurement process; (c) introducing electronic government procurement; and (d) creating greater public awareness of a well‐functioning public procurement system through mobilization of civil society, think tanks, beneficiaries, and the private sector. All these actions are key elements in effective implementation of the procurement law/regulations.” The PDO, as stated in the PAD, was much shorter and included the first part of the original PDO in the Legal Agreement, that is, “to improve performance of the public procurement system progressively in Bangladesh, focusing largely on the key sectoral ministries and targeting their implementing agencies.”

Key Expected Outcomes and Outcome Indicators

7. There were three expected outcomes, based on the original PDO (see Components 1–4 below derived from the PDO) and reflected in the original PDO indicators: (a) about 60 percent contracts in at least three target agencies reduce delays by making contract awards within the initial bid validity period, (b) e‐GP introduced in at least two target agencies at the headquarter (HQ) level for international procurement, and (c) increased stakeholder engagement in following procurement issues in at least two target agencies. The project identified four agencies of the key sectoral ministries as ‘target agencies’: RHD; LGED; BREB; and the BWDB. The CPTU, housed in the Ministry of Planning and under the direct authority of the Implementation, Monitoring, and Evaluation Division (IMED), was designated as the primary implementing agency.

Components

8. The project comprised four interrelated components that are summarized below, including resource allocations and actual costs. Also, project financing is provided in Figure 2.

(a) Component 1: Furthering Policy Reform and Institutionalizing Capacity Development (Total estimated cost: US$21.4 million; Actual cost: US$21.4 million)

Support the public procurement reform legal framework based on the 2003 Public Procurement Regulations and the 2006 Public Procurement Act (PPA)

Enhance the capacity development program that provides core procurement skills required to institutionalize procurement capacity

(b) Component 2: Strengthening Procurement Management at Sectoral Level and CPTU/IMED (Total estimated cost: US$15.9 million; Actual cost: US$10.35 million)

Strengthen the four key sectoral target agencies, along with 20 additional agencies in seven different ministries (added in the AF),2 for improved management and monitoring of procurement

2 The seven ministries/divisions were Ministry of Education, Power Division, Ministry of Primary and Mass Education, Energy and Mineral Resources Divisions, Ministry of Health and Family Welfare, Local Government Division, and the Ministry of Housing and Public Works.

(c) Component 3: Introducing e‐Government Procurement (e‐GP) (Total estimated cost: US$30.2 million; Actual cost: US$33.1 million)

Pilot e‐GP on a phased approach in the targeted agencies to increase procurement transparency and lower transaction costs and build the physical capacity of the e‐GP system to sustain a countrywide rollout (added in the AF and AF II)

(d) Component 4: Communication, Behavioral Change, and Social Accountability (Total estimated cost: US$5.4 million; Actual cost: US$4.87 million)

Create greater public awareness of the public procurement system and e‐GP by engaging civil society, think tanks, beneficiaries, and the private sector with the expectation of initiating a long‐term process of procurement monitoring.

Figure 2: Project Financing (US$ Million)

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION

Revised PDOs and Outcome Targets

9. Original PDO revisions. The original PDO was clarified in 2013, but the intent of the original PDO remained unchanged. The AF PDO was to “improve performance of the public procurement system progressively in Bangladesh, largely focusing on the Target Agencies.” This adjusted PDO was shorter, similar to the original PAD, and focused to reinforce the project ‘s four target agencies (RHD, LGED, BREB, and BWDB) while expanding the scope to provide capacity development assistance to 20 additional agencies in several sectors, including health, education, power, and public works. The more detailed content of the longer PDO at appraisal was further refined in the AF and AF II as project components and intermediate results in the Results Frameworks and Monitoring Plans but not formally included in the PDOs in the AF and AF II Legal Agreements.

10. Outcome revisions. In the AF, PDO indicator targets (related to the reduction in delays in making contract awards, the rollout of e‐GP, and expanded capacity development) were scaled up to reflect expanded scope, additional funds, and the extended closing date. Two new indicators were added to reflect assistance to the 20 additional agencies and the publication of quarterly reports by the target

agencies for monitoring of procurement performance. In AF II in 2016, AF PDO targets were again scaled up to reflect expanded scope, additional funds, and the extended closing date, and a new indicator related to the rollout of e‐GP was added.

Revised PDO Indicators

11. PDO indicators were revised and added to reflect the expanded scope, additional funds, and extended closing dates of the AF and AF II. Table 1 summarizes the PDO revisions in the AF and AF II.

Table 1. Revised PDO Indicators

Original Additional Finance Additional Finance II Comments

About 60% contracts in at least 3 target agencies reduce delays by making contract awards within the initial bid validity period

About 80% contracts awarded within the initial bid validity period by the 4 target agencies

About 82% contracts awarded within the initial bid validity period by the 4 target agencies

AF: Number of target agencies for PDO measurement increased from 3 to 4 and end target increased; AF II: End target increased

— 4 target agencies publish PROMIS quarterly report for monitoring of procurement performance covering 90% of bids/invited contracts awarded

4 target agencies publish PROMIS quarterly report for monitoring of procurement performance covering 95% of bids/invited contracts awarded

AF: Number of target agencies for PDO measurement increased from 3 to 4 and end target increased; AF II: End target increased

e‐GP introduced in at least 2 target agencies at their HQ level for international procurement (80% in 2016)

4 target agencies expand e‐GP to all 64 districts using National Competitive Bidding (NCB) procurement (80% in 2016)

4 target agencies expand e‐GP to all 64 districts using NCB procurement (83% in 2017)

AF: Number of target agencies increased and expansion from HQ to districts; AF II: End target increased

Increased stakeholder engagement following procurement issues in at least 2 target agencies

Dropped — Target achieved during original project

— Procuring entities of 20 additional agencies with one trained/certified procurement staff (70% in 2016)

Procuring entities of 20 additional agencies with one trained/certified procurement staff (78% in 2016)

AF: Introduced to reflect measurement of new outcome; AF II: End target increased

— — Number of public sector organizations registered in the e‐GP system

AF II: Introduced to reflect measurement of new outcome in the e‐GP system with enhanced capacity

Revised Components

12. All four components were revised in the AF, and two components were revised in AF II. These revisions focused on adding new and enhancing existing activities with additional funds and an extended closing date. Table 2 summarizes project component revisions in the AF and AF II.

Table 2. Project Component Changes

Original Component Additional Finance Additional Finance II

1: Furthering Policy Reform and Institutionalizing Capacity Development

Scope of the capacity development program expanded to 20 additional agencies beyond the 4 target agencies, with further targeting and refinement of training programs that include procurement compliance and core competence training, along with master’s level training in procurement.

Additional training provided for bidders and master's students

2: Strengthening Procurement Management

The organizational capacity for procurement performance management further strengthened at CPTU and the four target agencies, with full implementation of online performance monitoring (PROMIS) using indicators and an M&E framework.

3: Introducing e‐Government Procurement (e‐GP)

The transaction levels of the four target agencies substantially increased and the e‐GP network was initiated in 20 additional agencies.

Expansion of CPTU data center to manage e‐GP growth

4: Communication, Behavioral Change, and Social Accountability

Efforts to raise awareness regarding public procurement reform and e‐GP through behavior change communications and social accountability mechanisms strengthened

Other Changes

13. Revised closing dates. At the request of the GOB, the original closing date of March 31, 2013, was extended by six months to September 30, 2013, to ensure implementation continuity and enable timely processing of the AF, including its effectiveness within the project’s implementation time frame.

14. Additional Financing. There were also two AF agreements that expanded the scope, extended closing dates, and added funds to the project. The original total estimated project cost was US$24.9 million, of which the IDA contribution was US$23.6 million. In the AF, this was increased to US$60.9 million, with an added financing of US$34.5 million from IDA. The closing date was extended from September 30, 2013, to December 31, 2016 (for both the original project and the AF). In AF II, the total added project cost was US$12 million, with IDA financing of US$10 million. With AF II, the total project cost increased to US$72.9 million, with IDA financing of US$68.1 million. The closing date for all three credits was extended six additional months from December 31, 2016, to June 30, 2017.

Rationale for Changes and Implications for the Original Theory of Change

15. Rationale for changes. The changes to the project reflected in the AF and AF II were in response to the achievement of targets in the original project, as political will began to evolve in support of

procurement reform, target agencies were beginning to see improved results, and the bidding community increasingly supported World Bank‐supported procurement reform efforts. At the time of the original project appraisal, the GOB and the World Bank established modest and realistic project expectations in specific reference to e‐GP, recognizing the sensitive nature of procurement reform, specifically the vested interests that needed to be overcome to improve the use of public resources with greater efficiency, transparency, and accountability. Over time, the CPTU successfully built on the stakeholders’ ownership, both public and private, by extensive consultations and ground‐level engagements with behavioral change communication program, followed by political support at a later stage for procurement reform to achieve the targets set in the original, AF, and AF II projects. The key rationale for AF was to introduce e‐GP with an adjustment in the financing arrangement (DLI‐based financing) and expand the capacity development program to additional 20 agencies. Subsequently, due to the quicker‐than‐expected rollout of e‐GP, AF II supported the development of enhanced capacity data center to accommodate increased demand across the country so that all public‐sector organizations are covered.

16. Impact on theory of change. The original theory of change remained relevant throughout the life of the project. The slight adjustments to the PDO, PDO indicators and targets, and project components in the AF and AF II further refined project activities, outputs, and outcomes and strengthened their relationship to the PDO. Providing additional, targeted training for key procurement stakeholders (including procurement entities and the bidder/contractor communities), adding 20 additional agency beneficiaries across a wide range of sectors, and further refining communication and stakeholder engagement were three concrete changes to the project that contributed to the PDO by strengthening the public procurement system while specifically improving the performance of the 4 target agencies.

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating 17. The current Country Partnership Framework (CPF) for Bangladesh (FY16–20) “identifies governance reforms as a long‐term agenda that demands sustained effort as well as downstream and upstream interventions” by supporting policies and systems that improve transparency and efficiency in service delivery. The current CPF specifically states the World Bank’s commitment to support Bangladesh’s efforts to strengthen its public procurement system, including e‐GP.

18. Throughout the project appraisal process and implementation (2007–2017), the PDO’s focus on strengthening the public procurement system and improving the performance of the four target agencies, remained relevant to, and clearly supported, the World Bank’s objectives in Bangladesh. In addition to the current CPF, there are also references to the preceding Country Assistance Strategies (FY06–09; FY11–14) since the project was originally developed in 2007. The CAS FY11–14 specifically had a core governance pillar (enhance accountability and promote inclusion) under which enhanced transparency and accessibility of public services through information technology came in prominently.

Relevance Rating: Substantial

B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 19. The achievement of the PDO is assessed by reviewing the achievement of the activities, outputs, and outcomes associated with the four project components, which are reflected as intermediate results in the Results Framework. Throughout implementation, the GOB and the World Bank worked together closely to assess the relevance of Intermediate Results Indicators used to monitor key activities and assess the achievement of results associated with each component, along with the higher‐level PDO indicators’ relevance to the PDO. There were slight adjustments to the PDO and Intermediate Outcome Indicators at the time of the AF and AF II, with some dropped, added, and revised, retaining the main objective. Several also remained unchanged for AF II.

20. All PDO indicators were achieved fully, with some of them substantially exceeding the targets (for example, e‐GP). This has contributed to bringing about a systemic change in the public procurement environment (see annex 1 for a complete list of indicators). One PDO and 10 Intermediate Results Indicators were dropped in the AF because the targets were met in the original project. The indicators that were revised in the AF (two PDO and four Intermediate Results Indicators) and AF II (four PDO and six Intermediate Results Indicators) also achieved indicator targets fully and were adjusted upward to reflect this progress. The original PDO and Intermediate Results Indicator targets of e‐GP were modest given the prevailing challenging environment at that time with inadequate infrastructure, inadequate knowledge of e‐GP, limited willingness of implementing agencies, and unsure political commitment. Also, it was not possible to accurately measure the political will of key decision makers, whose proactive and public support for public procurement reform came at a later stage and their support of the rollout of e‐GP was essential to project success. Extensive stakeholders’ consultation and broad ownership of both the procuring organizations and the bidding community helped substantially in the successful implementation of e‐GP because improved transparency in e‐GP attracted large numbers of the beneficiaries. This was reflected in the original theory of change for the project. The improvements to the overall public procurement system, resulting from the capacity development and e‐GP initiatives, are summarized in Figure 3 in terms of efficiency, transparency, and competitiveness.3 Similarly, Bangladesh improved its ranking in the Corruption Perception Index (CPI). In 2007, the CPI ranking was 162 out of 179 countries, with a score of 2 (out of 10), while in 2016 it ranked 145 out of 176 countries, with a score of 26 (out of 100).

Figure 3. System Improvements Indicator Results

3 Efficiency is measured in terms of timeliness of awarding tenders, transparency is related to the publication of awards, and competition is based on the average number of bidders. Improvement in these three measures correlated with the rollout of e‐GP.

21. Over time, the Prime Minister, Minister of Finance, and the Minister for Planning became strong supporters of the project by observing its contribution to improving the procurement environment with drastically reduced inappropriate and/or coercive bidding practices at the decentralized level. They clearly communicated this support publicly and to key project stakeholders. In addition, IMED also strongly supported the project and the CPTU’s status as implementing agency. CPTU leadership managed to capitalize on this political support while effectively managing the project, despite the ongoing staffing challenges described in section IIIB.

22. While the PDO and Intermediate Results Indicators were organized in accordance with the four project components that guided implementation, many of them were cross‐cutting and mutually reinforcing across components/Intermediate Results. This contributed to the achievement of the PDO and supported the longer‐term goal of improving the use of public resources with greater efficiency, transparency, and accountability.

23. A brief analysis of relevant project data, including Results Framework Indicator results (both at the PDO and Intermediate Results levels), provides further empirical basis for an accurate assessment of project efficacy.

Intermediate Result/Component 1: Furthering Policy Reform and Institutionalizing Capacity Development

24. The project’s comprehensive capacity development program effectively provided core procurement skills required to institutionalize procurement capacity at the 4 target and 20 additional public agencies. There were five categories of training offered by the project: (a) 1–5‐day short courses for 17 different types of audiences covering policy makers to journalists; (b) three ‐week training sessions on the procurement of goods, works, and consultant services along with e‐GP training for five days; (c) a Procurement Core Competence Program offered by the Chartered Institute of Procurement and Supply, U.K. (CIPS) in collaboration with BRAC University Institute of Governance and Development (BIGD) for top performers of the three‐week training sessions; (d) a top‐up Master’s Program in Procurement and Supply Management offered by BIGD; and (e) a one‐year Master’s Program in Public Procurement Management for Sustainable Development in the Turin School of Development at the University of Turin in Italy.

25. The establishment of this comprehensive capacity development program, with five discrete but interrelated training opportunities (from one‐day courses to master’s level study), provided much‐needed capacity for a wide array of procurement professionals. By focusing on building the procurement training capacity of two local training institutes (the ESCB and BIM) and BRAC University, the project also developed a sustainable approach to training procurement professionals. Table 3 provides a final tally of capacity development project beneficiaries.

Table 3. Capacity Development Program Beneficiaries

Capacity Development Program Number of Beneficiaries Comments

Short courses (1–5 days) 14,083 17 topics, multiple stakeholders

3‐week training sessions 6,414 4 target agencies, 20 additional agencies

MCIPS (CIPS‐UK and BRAC‐IGD) 143 Top performers in 3‐week training

Top‐up Master's Program (BRAC‐IGS) 107 Potential MCIPS participants

Master's (University of Turin) 25 Potential participants from target agencies Note: MCIPS = Master’s Charted Institute of Purchasing and Supply.

26. There were two indicators directly related to capacity development when the project closed: (a) building procurement expertise in the procurement entities of the 20 additional agencies (PDO and Intermediate Results) and (b) building the capacity of the ESCB to deliver procurement training (Intermediate Results) from its own resources beyond PPRPII. Both were added in the AF.

27. Three additional Intermediate Results Indicators focusing on capacity development were dropped in the AF because targets were already achieved: (a) curriculum and training materials, (b) faculty capacity at a local training institute, and (c) institutional capacity at a local training institute or university to deliver core competence skill certification and accreditation in procurement.

28. There were indicators fully achieved in the Results Framework for the second part of this component, which focused on furthering policy reform, along with one indicator in the Monitoring Plan. In the original and AF PADs, reference is made to supporting secondary legislation resulting from the 2003 Public Procurement Regulations and the Public Procurement Act (PPA). Policy reform activities focused on preparing and updating policy documents, standard bidding documents, request for proposals, bid evaluation and post review guidelines, and ensuring for transparency and accountability by posting relevant public procurement information on appropriate GOB websites. This work was done by the CPTU, with the assistance of FINEUROP and ESCB.

Intermediate Result/Component 2: Strengthening Procurement Management

29. The original project focused on improving procurement management and monitoring in the 4 target agencies, with an additional 20 agencies receiving procurement management support in the AF and AF II. Additional Component 2 activities included (a) initial development of M&E frameworks for the four target agencies, (b) implementation of an online procurement performance monitoring system as part of e‐GP, and (c) provision of additional space for the CPTU with the vertical extension of a third floor in its building. AF II enabled the CPTU to meet the unexpected rapid growth in e‐GP demand across the country, with resources for additional processing and storage capacity of the e‐GP system.

30. There were four indicators that focused on strengthening procurement management when the project closed: (a) reducing delays by making contract awards within initial bid validity period (PDO), (b) publishing online PROMIS quarterly reports for monitoring of procurement performance (PDO and Intermediate Results), (c) publishing all contract awards above Public Procurement Regulation (PPR) thresholds on the CPTU website (Intermediate Results), and (d) the satisfactory handling of complaints (Intermediate Results).

31. Two additional Intermediate Results Indicators were dropped in the AF because targets were already achieved in the original project: (a) procuring computer hardware and software and (b) the submission of annual implementation plans.

32. Because the four target agencies embraced and implemented original project activities, indicators were added and targets expanded in the AF and AF II to reflect this progress. In addition, indicators were also dropped that were achieved before the AF Agreement in June 2013. When PPRPII ended on June 30, 2017, all indicator targets were fully achieved, with some substantially exceeding these targets. For example, e‐GP achievements far exceeded the target, which contributed to transform the procurement environment, and PROMIS, which tracked procurement entity performance related to efficiency,

transparency, and competitiveness, was effectively rolled out by the CPTU and implemented by all four target agencies, eventually producing quarterly reports on a regular basis. This system was integrated into e‐GP as the e‐Procurement Management Information System (e‐PMIS) module, with the CPTU and the four target agencies publishing quarterly procurement performance reports on their respective websites.

33. To create incentives for the target agencies to implement e‐GP and publish quarterly procurement performance reports on a regular basis, Disbursement‐Linked Indicator (DLI) with performance‐based financing, approximately valued at US$7.5 million, was introduced in the AF. There were two DLIs relating to e‐GP introduction and procurement performance measurement of the four target agencies. Disbursement was contingent on each of the four agencies meeting the above two DLIs. Three of the four target agencies spent 100 percent of their available DLI funds with the fourth, the BREB, spending 87 percent.

Intermediate Result/Component 3: Introducing e‐Government Procurement (e‐GP)

34. The project initially, and realistically, piloted e‐GP in the four target agencies but subsequently supported the rollout of this platform nationally for these four target agencies in the AF and AF II, once initial targets were achieved. It started with a pilot in four PEs only, then to 50 PEs, followed by 308 PEs, and eventually 1000s of PEs in four target agencies. The broadening of the scope of the e‐GP rollout in the AF and AF II reflect the rapid and growing acceptance and use of e‐GP by procuring entities and the bidding community, with the support of national‐level stakeholders and political leaders. Currently, over 1,000 other implementing agencies are registered in the e‐GP system and many of them are using it. Figure 4 elucidates the expanded rollout and use of e‐GP across Bangladesh by tracking the number of registered bidders/suppliers, the number of bid invitations, and the value of these bid invitations from e‐GP’s launching in 2011 until the end of PPRPII in June 2017.

Figure 4. Implementation of e‐GP Status

35. The milestone of exceeding 100,000 e‐GP tenders by nearly 34,000 registered bidders over the past five years demonstrates the widespread acceptance and sustainability of electronic public procurement throughout Bangladesh.

36. There were two indicators focusing on e‐GP when the project closed: (a) rollout of e‐GP in all 64 districts using NCB procurement by four target agencies (PDO and Intermediate Results) and (b) number of public sector organizations registered in the e‐GP system (PDO).

37. Two indicators were dropped in AF because targets were already achieved in the original project: (a) establishment of network and connectivity for management information systems (MIS) at the district level at two target agencies and (b) development and installation of e‐GP at the CPTU and all four target agency HQs.

38. In a survey that analyzed the impact of e‐GP implementation in cost savings for agencies in Bangladesh, the World Bank collected comprehensive procurement contract‐level data from implementing agencies. Based on an analysis of this data, e‐GP reduced the cost ratio between 13 percent and 20 percent, resulting in cost savings of more than US$ 446 million that includes transaction costs, advertising cost, and record‐keeping costs. There was also evidence that e‐GP improved competition by limiting political influence, because the e‐GP system allowed bidders to place bids from anywhere there was Internet access. This limited rent‐seeking opportunity at various stages of the procurement process that migrated to a more insulated, online environment in e‐GP. Overall, e‐GP promoted more competition among bidders, resulting in more competitive pricing and lowered transaction costs through improved access for bidders across the country. This resulted in cost savings, as shown in Figure 5.

Figure 5. Cost Savings in e‐GP

Intermediate Result/Component 4: Communication, Behavioral Change, and Social Accountability

39. While the first three components built government capacity, this supply‐side approach was complemented by the fourth component’s attention to the demand for public procurement reform. It created greater public awareness of the importance of public procurement to local development by engaging civil society, think tanks, beneficiaries, and the private sector, along with key public officials at the local and national levels.

40. The CPTU undertook a massive behavioral change and communication campaign to demystify procurement at the grassroots level and build ownership of stakeholders. The Bangladesh Center for Communication Programs (BCCP) assisted the CPTU on promoting behavior change communication by developing and implementing a multi‐faceted communications strategy. The key activities included: conducting 99 events on procurement reforms covering over 5,700 participants that includes future 64 search conference and e‐GP launching; convening 64 e‐GP awareness workshops at the district level with over 2,400 participants; establishing 64 GCFs covering over 3,300 Pes and tenderers; and organizing four e‐GP workshops including one at the national level for 44 registered banks in the e‐GP system. The GCF provided an informal platform for GOB officials and bidders to share information and experience,

including issues and resolutions about procurement and the e‐GP system. The BCCP also developed a mobile app; produced an e‐GP theme song, videos, radio and television commercials, TV talk shows, success story videos, infographics, email and on‐line marketing; and created two digital billboards in Dhaka that display online live procurement data with a direct feed from the e‐GP platform. CPTU produced newsletters, leading articles in media.

41. The CPTU pursued citizen engagement at the policy level by creating a Public Private Stakeholders Committee (PPSC), headed by the Planning Minister, and comprising representatives of key GOB agencies, leading think tanks, civil society, and business apex bodies. The PPSC was designed as a platform to discuss procurement policy guidance and provide opportunity for the public and private sector to share updates and focus on challenges and opportunities associated with ongoing procurement reforms, including the e‐GP rollout across the country. The BIGD under BRAC University provided technical support to the CPTU for the PPSC. Concurrently, the BIGD also piloted citizens’ monitoring of procurement contracts at the decentralized level through the establishment of citizens’ committees in four sub‐districts in two northern districts. This pilot program brought together community members, private contractors, and local procurement entities around a specific public works project, providing opportunities for joint oversight and problem solving. The BIGD held four divisional workshops, produced a documentary of social accountability, and shared its findings in a national citizens’ engagement forum. Three case studies on school construction and textbook delivery and three policy notes on citizen engagement and e‐GP were also produced.

42. These activities demonstrated a clear recognition by the GOB and the World Bank that successful procurement reform requires improved supply and informed demand by key stakeholders at the local and national levels. The BCCP and BIGD proactively reached out to key stakeholders across the country, sensitizing them to the ongoing procurement reform process with a range of activities at the community, district, and national levels. Many of these activities were attended by CPTU officials and World Bank staff, strengthening relationships and increasing communications between stakeholders regarding procurement reforms implemented by the CPTU. Toward the end of the project, BCCP conducted a qualitative perception opinion research (end‐line evaluation) of procurement reforms covering a target group of tenderers, procuring entities, civil society members, media, and financial institutions. Overall, 79.3 percent of respondents were positive about improved transparency in public procurement; 76.6 percent expressed procurement reforms as effective; 72.3 percent recognized the system as efficient; and 81.9 percent mentioned improved accountability. Variations among audiences are summarized in Figures 6 to 9.

Figure 6: Opinion on Transparency of Procurement Reforms Figure 7: Opinion on Effectiveness of Procurement Reforms

Figure 8: Opinion on Efficiency of Procurement Reforms Figure 9: Opinion on Accountability of Procurement Reforms

43. There were three Intermediate Results Indicators focusing on behavioral change communication, and social accountability when the project closed: (a) GCFs held in all 64 districts, holding semiannual meetings; (b) e‐GP awareness workshops held in all 64 districts; and (c) public‐private stakeholders committee workshops held. There were no PDO Indicators.

44. Four indicators were dropped in the AF because targets were already achieved: (a) increasing stakeholder engagement on procurement issues in at least two target agencies (PDO), (b) developing a government communications strategy (Intermediate Results), (c) organizing communications and advocacy workshops in all 64 districts (Intermediate Results), and (d) convening workshops on public procurement and social accountability for key stakeholders (Intermediate Results).

Justification of Overall Efficacy Rating 45. The project strengthened the public procurement system and improved the performance of the four target agencies based on the achievement of PDO and Intermediate Results Indicator targets and the project data presented earlier. Both parts of the PDO were weighted equally in the original, AF, and AF II PADs, and indicators and targets used were appropriate measures of project efficacy.

Efficacy Rating: Substantial

C. EFFICIENCY

Assessment of Efficiency and Rating 46. The project was designed to strengthen the public procurement system and improve the performance of the four target agencies by supporting sustainable procurement reforms to improve competition in the competitive bidding process, reduce procurement delays, and improve procurement quality through better management and monitoring at the target agencies. To gauge the impact of project activities, a set of well‐defined procurement performance indicators was developed and improved in the AF and AF II. The existing procurement management information systems at the CPTU and the four target agencies were enhanced and expanded to monitor their performance against established benchmarks

throughout implementation. Because project results could not be fully quantified in monetary terms, there was no calculation of an economic rate of return.

47. The business model that was developed for the e‐GP system included revenue generation from end‐user fees (bidders’ registration fee, renewal of registration, and sale of bidding documents). This model was designed to ensure the sustainability of the e‐GP system beyond the life of PPRPII. At the end of the project, the e‐GP system was generating revenues that exceeded initial targets. Earnings forecasts/actuals are as follows: FY14‐ US$0.55 million/US$1.8 million; FY15‐ US$0.95 million/US$4 million; FY16‐ US$1.25 million/US$5 million; and FY17‐ US$2 million/US$ US$8.8 million. As opposed to these earnings, on average, the annual operation and maintenance cost of the e‐GP system is US$2.5 million. This exemplifies that the system is already generating higher revenues than expenses and is becoming self‐sustainable. The CPTU also fully funded all staff salaries throughout the project and implemented the e‐GP operation and management (O&M) contract including 24 x 7 helpdesk services with substantial financing (60 percent) from the GOB.

48. Overall, the project costs incurred to achieve its broader objectives were reasonable in terms of the tangible benefits achieved (as described in the last section) and ‘value for money’ given the sustainability of these benefits.

Efficiency Rating: Substantial D. JUSTIFICATION OF OVERALL OUTCOME RATING

49. The PDO remained relevant to the GOB’s and the World Bank’s plans and priorities throughout implementation. Project objectives measured by PDO and Intermediate Results Indicators and related project data were consistently achieved or exceeded, and project resources were efficiently used to achieve sustainable results. Overall Outcome Rating: Satisfactory E. OTHER OUTCOMES AND IMPACTS

Gender

50. AF II specifically addressed the impact of Components/Intermediate Results 1 and 3 on closing the gender gap in public procurement. The e‐GP created more professional opportunities for computer‐literate women, particularly working in computer cafes given the increased Internet access demands by the bidding community across the country and at procuring entities. Women also participated in training programs, providing them with opportunities for professional advancement. Furthermore, women participated effectively in the citizen engagement component with specific reference to contract monitoring and e‐GP workshops at the decentralized level.

Institutional Strengthening

51. The project contributed to institutional strengthening of the public procuring entities, civil society organizations, and associations along with the public and private sectors, through the training, technical

assistance, and outreach initiatives described. Key public sector beneficiaries included the focal implementing agency, CPTU, along with the 4 target and additional 20 agencies. Representatives from the bidding community also received various levels of procurement training—in Dhaka and at the district and sub‐district levels. Civil society and educational organizations were also beneficiaries, including the BCCP, BIDG, BIM, and ESCB.

Mobilizing Private Sector Financing

52. Not applicable.

Poverty Reduction and Shared Prosperity

53. Throughout implementation, there was a growing recognition by the GOB regarding the importance of enhanced accountability and improved governance to reducing poverty, improving the investment climate, and accelerating private sector‐led growth. Because the annual government expenditure on the procurement of goods, works, and services accounted for 20 to 40 percent of the annual national budget and 60 to 80 percent of the annual development budget at appraisal, improving procurement was—and continues to be—a foundational aspect of good governance and enhanced accountability.

54. PPRPII addressed accountability and governance issues directly, improving the public procurement system by developing and successfully rolling out e‐GP. During consultations by the Implementation Completion and Results Report (ICR) Team with key GOB stakeholders and representatives of the bidding community, there was a clear consensus among public and private sector beneficiaries that the project improved accountability and governance in public procurement by limiting opportunities for corrupt practices when e‐GP became operational.4

55. The e‐GP also leveled the playing field in the procurement process for the bidding community by limiting the impact of personal connections, informal relationships, and rent‐seeking behaviors of public officials at the national and local levels. This contributed to a broader, shared prosperity in the bidding community, empowering contractors previously unable to successfully navigate the daunting challenges of the pre‐e‐GP paper‐based procurement process.

Other Unintended Outcomes and Impacts

56. Not applicable.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

57. Project design. The project was well designed at the outset, aligning the original PDO with the GOB’s and World Bank’s priorities (see section IA and IB), identifying appropriate activities relevant to the PDO (see section IIB), and applying lessons learned from the PPRP, specifically regarding the importance

4 The ICR team conducted two missions to Dhaka in June and July 2017, meeting with the World Bank and CPTU teams, along with a wide range of project stakeholders.

of including the supply and demand dimensions of procurement policy reform. There were four clearly structured and sequenced components with an appropriate array of interrelated activities. In addition, there was a clear operational logic with a motivated focal implementing agency (CPTU), 4 targeted agencies, and in the AF, an additional 20 agencies. There were also 19 legal covenants that were established for the project focusing on a range of operational issues related to the CPTU and target agency staffing, installation of e‐GP systems, and performance and financial reporting. Most covenants were fully in compliance, with a few partially, at the end of the project. Compliance was delayed for nine of these covenants.

58. Results Framework. There were five PDO and nine Intermediate Results Indicators well distributed across components when the project closed. This included three indicators that were common as PDO and Intermediate Results Indicators. Eleven additional indicators were dropped in the AF because targets were already achieved in the original project.

59. Monitoring Plan. The project’s Monitoring Plan placed the responsibility for data collection on the CPTU and the four target agencies, with the assistance of the firm, Survey Research Group Bangladesh (SRGB), which reported to the CPTU. For the indicators that monitored incremental progress in achieving specific results, target values were set annually for the anticipated five‐year duration. Overall, appropriate baselines and target were set, given the working assumptions of the GOB and the World Bank at the time of appraisal, as reflected in the theory of change described in Section IA.

60. Stakeholder selection. The project identified a subset of public procurement stakeholders to serve as implementers and beneficiaries. This included the CPTU, the four target agencies, BIM, and ESCB. The CPTU and the four target agencies were the key drivers for developing and rolling out e‐GP, with BIM and ESCB contributing to capacity building. Additional project beneficiaries included the bidding community (including banks), along with civil society organizations, institutions of higher learning, and professional associations representing bidders with common expertise and focus.

61. Risk mitigation measures. There were three risks to the PDO and six risks to component‐level results identified at preparation, all rated Modest or Substantial. The overall risk to project completion was rated Substantial. Lack of capacity, political will for reform, and willingness to embrace change were three main drivers for these risks. Project approach and planned activities were designed to mitigate these risks when feasible. Component/Intermediate Result 1 focused on capacity, while political will and willingness to change were addressed by empowering and equipping motivated change agents in implementer and beneficiary agencies. A comprehensive stakeholder engagement and strategic behavior change communication program was integrated throughout the life of the project to neutralize and/or reduce the possibility of perceived risks from non‐PE and bidding community stakeholders such as civil society organizations (CSOs), media, and other advocacy groups.

B. KEY FACTORS DURING IMPLEMENTATION

62. GOB/implementing agency. With the initial encouraging results on the ground, the GOB strongly supported the rollout and increased use of e‐GP by government procurement entities and the bidding community. This was conveyed by the highest levels of political leadership (see section IIB). There were, however, issues related to human resources and the organizational capacity of the CPTU during the early stages of implementation. A few positions were not filled, with this challenge compounded by periodic

staff transfers. This contributed to the slow mobilization of project activities, but this was subsequently resolved when the positions were eventually filled.

63. PPR 2008 and PPA 2006 served as the legal and regulatory foundation for project activities. There were—and continue to be—challenges associated with subsequent amendments to the PPA that were not favored by some bidders and development partners, including the World Bank. Some amendment requirements also did not conform with international best practices and World Bank procurement requirements. This included the following changes to provisions for contracts valued at US$300,000: lottery for works contracts in case there was a tie in price because of an artificial price cap, no experience/qualification requirement for small‐value works contracts, and the rejection of bids above or below a fixed percent of the estimate for small works contracts. None of these amendments were or continue to be acceptable for World Bank‐financed contracts.

64. There were no major issues related to basic logistics, M&E, or fiduciary, social, and environmental practices.

65. World Bank PPRPII team. The World Bank Team proactively supervised all four project components and worked closely and collaboratively with the primary implementers of the project: CPTU, the four target agencies, FINEUROP, ESCB, BCCP, and BIGD. Continuity of project leadership (there was one Task Team Leader throughout appraisal and implementation) contributed to building strong relationships with stakeholders and facilitating open and constructive dialogue that focused on resolving issues as they arose.

66. Project reporting was conducted regularly and predictably, ensuring that project data were routinely collected and analyzed by the GOB and used to effectively monitor implementation progress. There were some challenges with the collection and analysis of data for the project, but the team carefully monitored this and worked closely with the CPTU and SRGB to resolve these challenges.

67. There were 18 implementation supervision missions that were conducted on a regular basis, with detailed and well‐documented Aide Memoires. In addition, 17 Implementation Status and Results Reports (ISRs) documented implementation progress according to the Results Framework and Monitoring Plan up to December 2016. The 18th and final Implementation Support Mission (ISM) findings and conclusions are recorded in the ICR.

Outside Project’s Manageable Control: Not applicable

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

68. Project M&E was conducted by the CPTU and target agencies. The SRGB assisted the CPTU by preparing semiannual reports on the overall project M&E, including progress on the Results Frameworks, for review by the World Bank.

69. Results monitoring for the original project covered (a) procurement activities by the target agencies to determine compliance with the PPR, (b) procurement performance by the target agencies, (c) a capacity development program and its impact, (d) behavioral change communication and social accountability subcomponents, (e) the status of procurement and FM, and (f) overall project performance and its impact. For each component, a separate monitoring system was developed.

70. AF results monitoring was enhanced by addressing (a) procurement performance by the target agencies, (b) capacity development by the additional agencies, (3) procurement monitoring by the target agencies using PROMIS, and (d) conducting of procurement using e‐GP.

71. Due to quicker‐than‐expected progress in developing and rolling out e‐GP in the AF and AF II, and strengthening civic engagement during the original and AF projects, the M&E design in AF II was further refined. The GOB established e‐PMIS for tracking procurement performance with a set of indicators to measure efficiency, transparency, and competitiveness of the system, with all four target agencies publishing e‐PMIS indicator reports on their respective organization websites and on the CPTU website. During AF, e‐GP system information became available to the public, including bid invitations, contract awards, and quarterly online procurement performance monitoring reports with 45 system‐generated indicators.

72. To meet the demand side of governance with civic engagement as the ‘third eye’ on overall procurement, the model of third‐party monitoring was piloted by the BIGD, with two field‐level nongovernmental organizations using feedback from the citizen committee at the subdistrict level. The PPSC, which included high‐level representatives of different key stakeholders including leading think tanks, regularly convened to provide strategic support.

73. Overall, project M&E covered all components and subcomponents of the project, with M&E data used to evaluate the project in three stages: baseline at preparation and progress made at midterm and end of project implementation. The indicators selected for all the three projects adequately reflected PPRPII’s theory of change, tracking progress made for all four project components and, more generally, for the PDO. Two project indicators were repeated as PDO and Intermediate Results Indicators without any explanation in the AF II PAD.

M&E Implementation

74. During implementation of the original, AF, and AF II projects, M&E implementation gradually improved after a challenging start. The SRGB initially did not collect, analyze, and report the data needed for the Results Frameworks and Monitoring Plans, requiring the World Bank and CPTU to clarify expectations. This was well documented in many Aide Memoires, proactively addressed by the Task Team Leader and the team, and further discussed during ISMs. The e‐GP system offered systematic data collection which enabled the SRGB, along with the World Bank and CPTU, to access much of the data required for the Results Framework and Monitoring Plan regularly during the AF and AF II.

75. The target agencies and the CPTU provided required information to the SRGB, including direct data from the agencies; published data in the MIS of the target agencies’ and CPTU (e‐PMIS); and documentation of field visits, consultations with official sources (including the bidding community, civil society, and beneficiary groups), and interviews with key informants.

M&E Utilization

76. The project effectively used the original and AF Results Frameworks and Monitoring Plans to update and improve subsequent iterations for both AF agreements. Both PDO and Intermediate Results Indicators were dropped and added based on progress achieving indicator targets, with some indicator targets increased to reflect strong progress in implementation. The methodology for one indicator was revised to improve accuracy based on available data. No targets were lowered in the original, AF, and AF II projects.

Justification of Overall Rating of Quality of M&E 77. Despite initial challenges with the M&E firm (SRGB) and the unexplained repetition of indicators in the PDO and Intermediate Results categories, overall the project Results Framework and Monitoring Plan did a commendable job capturing results, adapting to the rapid acceptance of e‐GP by procuring entities and the bidder community. The original and AF indicators also appropriately informed the content and form of both AF agreements. M&E Quality Rating: Substantial B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

78. Environmental compliance. The project was assigned Category ‘C’ at preparation, since no environmental impacts were expected to arise due to project implementation. This designation remained for the AF and AF II. Overall, the project did not involve any acquisition of land and displacement of population, and there were no project‐affected persons issue on this account. The AF supported the vertical extension of the CPTU building (from two to three stories), but no significant and irreversible environmental impacts arose during implementation, and the construction was subject to proper adoption of an Environmental Code of Practice (ECoP), including occupational health and safety issues. The bidding document included environmental clauses and kept provisions for better occupational health and safety for the workers. The Public Works Department supervised and ensured proper implementation of the ECoP.

79. Social compliance. Extensive efforts under PPRPII contributed positively to changing the ‘mindset’ of stakeholders, including public procurement officials and the bidding community. Under the AF, the involvement of civil society organizations and project beneficiaries further contributed to procurement monitoring, with specific reference to e‐GP awareness and bringing the reform agenda at the grassroots. The framework for stakeholder participation (a) initiated a gradual approach to the concept of social accountability in procurement, (b) conducted public awareness campaigns in the beneficiary/user communities, and (c) instituted a communications strategy to build support procurement reform.

80. The social accountability and citizen engagement subcomponent of the project included PPSCs, GCFs, and third‐party monitoring by citizen/beneficiary groups on a pilot basis in two districts covering four sub‐districts. This approach generated deep interest in the community to take the reforms to the grassroots and is concurrently making the public procurement officials more accountable to the citizens.

81. The project also contributed positively to improved gender inclusion. The e‐GP initiative (Component 3) specially created more opportunities for the computer‐literate female workforce at computer cafes and in data processing specifically in the bidding community and procurement entities. Furthermore, women benefitted from capacity development (Component 1), and increased participation in the monitoring of procurement contracts through citizen engagement.

82. Fiduciary compliance. The CPTU, with the assistance of two FM consultants, implemented PPRPII in the absence of its own FM organization. The target agencies received project funds through the CPTU in their dedicated project bank accounts. In the AF, funds were disbursed to the agencies based on achievement of targeted results related to DLIs. After initial challenges establishing the disbursement modalities related to DLIs, they served their purpose and three of the four target agencies received 100 percent of available DLI funds, with the BREB receiving 87 percent.

C. BANK PERFORMANCE

Quality at Entry

83. Strategic relevance and approach. The World Bank leveraged the GOB’s evolving understanding of the importance of procurement reform to good governance at entry, building on, and learning from the PPRP, the previous operation. The 2003 PPR, followed by subsequent passage of the PPA 2006 and issuing the PPR 2008, provided the legal framework for developing a strategic approach to public procurement reform in Bangladesh.

84. Technical, financial, and economic aspects. The World Bank team that prepared PPRPII adequately considered the technical, financial, and economic aspects of the proposed project. The PPRP provided some helpful lessons learned, including nontechnical elements such as political economy analyses to inform and improve implementation and operationalizing technical insights regarding the importance of addressing both the supply and demand aspects of supporting and institutionalizing sustainable reform. The IDA funds allocated for the original and two AF projects correctly reflected absorption capacity for the implementing agency and four target agencies.

85. Poverty, gender, and social development aspects. The World Bank team focused on the importance of good governance to economic growth and poverty reduction, based in part on the GOB and World Bank strategic planning documents. While gender was not an explicit focus of the original, AF, and AF II projects, there was a recognition that the project could promote gender inclusion, even if indirectly. Social development was directly related to Component 4 activities related to behavior change communication and stakeholder engagement (see section IVB).

86. Environmental and fiduciary aspects. The World Bank assessed that the project was a Category ‘C’ at entry, because no environmental impacts were expected during implementation. Category ‘C’ status remained for the AF and AF II as well. Given limited FM capacity of the CPTU at entry (see section IIIB), the World Bank supported hiring of two FM consultants to address fiduciary requirements and build the CPTU’s capacity in this area.

87. Policy and institutional aspects. The World Bank Team and GOB recognized at preparation that the 2003 PPR along with the PPA required follow‐up assistance, including the revision of bidding

documents and English language translations of key documents. Component 1 specifically addressed this requirement.

88. Implementation arrangements. The CPTU was identified as the lead implementing agency, given its prominent role as the public procurement focal point for the GOB, and to ensure implementation continuity from the PPRP to PPRPII. The four target agencies were also identified as important implementing partners given the project’s focus in building the capacity of procurement entities to improve public procurement across sectors and throughout the country. Three of the four target agencies had a physical presence outside Dhaka.

89. M&E arrangements. The M&E framework at entry for the original, AF, and AF II projects evolved to reflect implementation progress, additional funds, and expanded scope. This was done by adding indicators, increasing targets for existing indicators, and dropping indicators when targets were met. The GOB and the World Bank recognized the needed for additional M&E support at preparation and hired a consultant to assist the CPTU with data collection and analysis required for the project’s Results Frameworks and Monitoring Plans.

90. Risk assessment. The Bank Team assessed the overall risk to PDO achievement and component results as substantial. The main assessed risk to PDO was inadequate support and commitment from target agency upper management at the initial phases due to the sensitive nature of procurement reform, and the important role they needed to play in initially piloting, and subsequently rolling out e‐GP. Component #4 activities related to civil society monitoring of procurement processes were assessed as the most controversial and highest risk to achieving results due to the politically sensitive nature of monitoring in sub‐districts outside of Dhaka.

Quality of Supervision

91. The World Bank conducted 18 ISMs which carefully monitored implementation progress and identified key challenges to development impact that required additional attention by the GOB and the World Bank. The World Bank team was a multi‐sectoral team with fiduciary, urban, water, transport, energy, communication, and information and communication technology (ICT) expertise. Key staff and consultants with required technical expertise at the component level participated in these missions regularly. The project Task Team Leader and most team members were based in Dhaka, with a few international consultants hired on an as‐needed basis. The team developed an intimate knowledge of the project and maintained a strong working relationship with the GOB.

92. The PDO and longer‐term outcomes in the theory of change informed the prioritization of the challenges identified during ISMs and routine oversight protocols. The Aide Memoires for ISMs clearly identified these challenges and documented general recommendations for the GOB, along with key stakeholders including the CPTU, the four target agencies, and others involved in implementing activities for each component. The ISR also documented progress against Results Framework indicators and legal covenants.

93. The World Bank team prioritized sustainable project outcomes for all four components and for the PDO more generally. Building the capacity of the ESCB to deliver procurement training without project funds and developing a business model for e‐GP that prioritized cost‐recovery from user fees were two

innovative approaches used by the GOB and the World Bank to prepare for project sustainability after the project closes.

Justification of Overall Rating of Bank Performance

94. Despite the highly challenging environment, the World Bank team’s continuous efforts to pursue the reform agenda, by consistently maneuvering and adjusting its approach to implementation, is commendable. Most ISMs were held on time, fully and transparently recorded in Aide Memoires, and followed by quality ISRs. This formal ISM is beyond the day‐to‐day field‐based supervision of project implementation by the World Bank team. Generally, the team was realistic and, at times, conservative in the ratings included in the ISRs. Overall, the team demonstrated its passion for reform and was able to form a critical mass within the GOB’s leading agencies who themselves now own the agenda. It is rare for politically sensitive and challenging reforms to be supported and institutionalized by governments, especially when they go against vested interests in the system. This took several years to be accomplished, but the World Bank’s perseverance and partnership with the GOB was well worth the effort.

Bank Performance Rating: Satisfactory

D. RISK TO DEVELOPMENT OUTCOME 95. There are two primary risks to the development outcome, specifically related to financial sustainability and government/stakeholder ownership of, and commitment to, the continued rollout and institutionalization of e‐GP across the country. This requires continued political support for public procurement reforms that facilitate greater efficiency, transparency, and accountability.

96. Regarding financial sustainability, the GOB needs to remain committed to restructuring the CPTU to (a) enhance its authority to recruit high‐performing information technology professionals, (b) further refine its current business model that generates revenues from user fees to cover operational costs, and (c) continually upgrade the e‐GP system to meet the increasing demand and improve existing services.

97. As described in section IIIB, the GOB made some PPA amendments, of which couple were not consistent with international best practices or with World Bank procurement requirements. To ensure the continued development of a transparent and accountable e‐GP system, the GOB will need to remain committed to public procurement reforms that ensure the integrity of e‐GP, and public procurement more generally, even when there are political pressures against this.

V. LESSONS AND RECOMMENDATIONS

98. The following lessons learned and recommendations from PPRPII can inform future World Bank‐funded governance, and more specifically, procurement reform‐related projects focusing on policy reform, e‐GP, and capacity development. Given the success of this operation, which is the World Bank’s single largest operation in procurement reform under credit financing, these lessons and recommendations offer tangible and practical insights transforming global knowledge into ‘good local fit’ practices that may help in undertaking procurement reforms in other countries and regions.

99. Challenges and enabling conditions for a sustainable reform. Enabling conditions for a sustainable reform are not straightforward in a challenging environment like Bangladesh and often follows a complex trajectory, affected by numerous factors such as external influence, vested interest groups, political backlash, conflicting interest of stakeholders, corrupt practices, lack of appreciation to pursue a good governance agenda, and reluctance to change. Despite this, the factor that remained as crucial enabler was the appropriate recognition/understanding of the country’s operating context and its inherent political economy. Other key supporting enablers include—recognizing the Government as the driver of reforms; identifying a political leader to manage the political pathway; working with a few reform‐minded public officials both at the core ministry and the implementing agency level to design and implement the actual reform actions; and coordinating with influential stakeholders outside the Government (bidding community/civil society). In that regard, the development partner’s role was instrumental in terms of providing the required global knowledge and technical support with funding.

100. Commitment and ownership. Successful and sustainable procurement reforms require ongoing high‐level commitment and ownership of the key stakeholders, especially the implementing agencies and the bidding community, as well as high‐level political commitment from the government. Such combined ownership enhances the probability of sustainable reform. Although both operations enjoyed support of the two governments in power that came successively from two opposition parties, such backing was not high at the initial phase of reforms. Instead, the actual implementation of crucial reforms like e‐GP was a bottom‐up approach, mainly owned by the mid‐level public officials of a few key agencies (who wanted to see changes) and a relatively young bidding community, both with knowledge of computers. Initially, it was a slow growing ownership of the e‐GP system by those stakeholders. The project implementers, including the key officials of the CPTU and the four target agencies key officials, leveraged the support of enthusiastic mid‐level officers and the young community of bidders and banks by widening the ownership circle. When political leaders saw that e‐GP was becoming popular at the field level because it reduced rampant bid‐rigging/coercion/collusion, and thus directly contributing to containing unruly elements, then they started to support e‐GP. Within a short time, e‐GP became one of the most prominent political commitments of the Bangladesh Prime Minister, along with key members of the Cabinet (Finance, Planning) who supported and mandated its rollout in all of the nation’s 64 districts. This was evidenced by the successful development and exponential rollout of e‐GP.

101. Sustainable institutional mechanism. Having in‐built institutional mechanism within the Government’s existing framework makes the reform more sustainable in the longer term, although the pace of reform initially may be slow. An example of such sustainable mechanisms is how the CPTU has been fully funded by the Government, as opposed to creating a Project Implementation Unit with project funds. Over time, the CPTU’s visibility within the Government has become highly prominent, with credibility ratings of the implementers and the bidding community.

102. Self‐sustainable e‐GP business model. Early attention to developing a locally relevant, revenue generating e‐GP business model can help instill long‐term sustainability that is not dependent on the government or development partner resources. Generated revenue can cover e‐GP operation and maintenance costs, along with new investments in the system, and also can strengthen and accelerate the e‐GP roll‐out while increasing revenue over the long‐term. Despite initial concerns over such a model and its feasibility, it eventually worked very well in Bangladesh. PPRPII’s revenue generation model has resulted in the establishment of a sustainable e‐GP system and also contributed to its accelerated roll‐out.

103. Continuity of engagement. The slow process of procurement reform needs sustained engagement by the government as the leading agent, along with other stakeholders including the development partners (if applicable), to bring about desired changes in institutions and systems. Tangible outcomes on the ground resulted from a sustained effort over time through two successive interventions to granulate and institutionalize policy changes and capacity development within the government system.

104. Leadership continuity. Longer‐term and comprehensive reform initiatives require continuous and uninterrupted knowledge of, and commitment to, the sequential and inter‐related activities that achieve sustainable results. The GOB successfully managed changes in the CPTU with minimal disruption to implementation by appointing two successive qualified and experienced Project Directors with deep interest in reforms. In addition, there was only one PPRPII Task Team Leader throughout the project life cycle, from conception to completion. The continuity in leadership facilitated progressive implementation and ensured progression of the thought process, with the focus on the implementation of activities and achievement of sustainable results on the ground.

105. Change management and client‐Bank relationship. The Bank team and the key government team at the CPTU worked like one unit for the same goal, without a hierarchical regime. Building an informal relationship of collaboration created huge momentum within this core group to collectively pursue the arduous reform agenda. The CPTU Head and Bank Task Team Leader worked hand‐in‐hand at all times, while maintaining their own institutional boundaries. At the implementation level, members of technical working groups at the four target agencies implemented the critical tasks of actual reform actions at the head office and decentralized level.

106. Sequencing of reform activities. Proper sequencing of actions over a longer‐term horizon is critical, most likely through a series of successive interventions/operations that have specific objectives within the overall reform framework. Lessons learned from PPRP were incorporated in the design of PPRPII, and internal learning from the second operation was incorporated in the two subsequent PPRPII additional financing mechanisms. Based on PPRPII learning, recently another new operation has been approved with US$55 million credit financing for digital project implementation monitoring at national level and e‐GP enhancement across the entire country.

107. Supply and demand of reform. Implementation of politically sensitive and challenging public procurement reform requires attention to the supply and demand sides of the procurement life cycle. Supply‐side issues related to developing an efficient and effective legal and regulatory framework and enhancing the service delivery capacity of key implementers need to be complemented by nurturing informed demand for such reforms. PPRPII supported ongoing procurement reform, developed and rolled out e‐GP, and built the capacity of key implementers to effectively use this system while also focusing on educating potential users and beneficiaries about the benefits of PPR and, more specifically, e‐GP. The communications and behavior change components of the project specifically focused on the demand side.

108. Communication and behavior change of stakeholders. Sustainable and transformative policy reform needs an integrated suite of interventions that move beyond the actions of passing new laws and providing specialized, targeted training and technical assistance. Beyond technical contents of laws, capacity development and e‐GP, PPRPII undertook a massive stakeholders’ engagement program that supported a range of activities in engaging and sensitizing key stakeholders across the country on the importance of procurement reform and the benefits of e‐GP. The program involved different strata of

communities, including political leaders, implementers, bidders, bankers, civil societies, academia, journalists, and, most importantly, common citizens and project beneficiaries. The PPSC and GCF played an important role, combined with numerous workshops and deeper dialogues directly with the field level officials, bidding community, and journalists across the country that created visible momentum. Partnerships with BCCP and BIGD represented the recognition by the Bank and the CPTU that directly engaging stakeholders was essential to project success, in terms of communicating the content and form of reforms on a regular basis, and changing behavior based on the reforms. Taken together, these activities effectively moved beyond simplistic technical “solutions,” reflecting an integrated approach to reform that explicitly addressed both supply and demand issues.

109. Staffing at nodal agency (CPTU). The project suffered at times from the inadequacy of staff or non‐availability of skilled staff, in particular IT‐based technical staffing. Market constraints to identify properly skilled staff added to delays in implementation. Appropriate staffing, both in terms of numbers and skills, is important for timely implementation of the reform agenda.

110. Target‐setting for DLIs ‐ Government buy‐in and political economy considerations. During project development, the Government and the World Bank agreed on DLI targets that reflect the political economy risks related to challenging procurement reform while developing targets that challenge and incentivize government partners to proactively support difficult reform. During the development of PPRPII, the Government and the World Bank recognized the political economy and technical challenges associated with an effective and accelerated development and rollout of e‐GP, including inadequate IT infrastructure, limited capacity of key partners, a recalcitrant mindset to maintain the status quo, and reluctant political will. This influenced the establishment of DLI targets that were eventually mostly exceeded by three of the four target agencies.

111. Disbursement versus DLI targets and project restructuring. The project faced serious fund flow/disbursement issues for several months because the DLI values set against each DLI period did not match the actual expenses incurred for that corresponding period, although the implementing agency fully achieved DLI targets ahead of time in most cases. This required restructuring of the project to amend the specific provision of disbursement, allowing a more flexible approach of cumulative expenses over period(s) as against expenses of each DLI period. The lesson learned is that, in Investment Project Financing with DLIs, disbursement provisions should be made flexible enough in the Financing Agreement, allowing disbursement against DLI values upon cumulative expenses, instead of matching values with expenses in each DLI period.

112. Integrated, incentive‐based capacity development. Clear incentives for beneficiaries to participate in multiple professional development opportunities, improves the efficacy of training and technical assistance. PPRPII’s capacity development program comprised an integrated set of trainings with increasingly higher order levels, ranging from one‐day sessions to master’s level study abroad. Procurement professionals that excelled in specific programs could ‘graduate’ to the next level of training. Top performers went on to earn international procurement accreditation such as MCIPS/UK or master’s degrees at the University of Turin in Italy.

113. Country system and integration with associated fields. With this procurement reform, now all World Bank‐funded projects in Bangladesh have been largely using the country systems for all national procurement. Work is going on to adopt the country systems for international procurement as well. As

part of the overall improvement of the public‐sector institutions and public financial management system, an integrated approach is under way to link e‐GP with the budgetary system and national identification system of the Government for effective public service delivery. Also, the procurement review/audit function is being linked to e‐GP using a separate module.

114. Theory of change, PDO, and indicators. Focused PDO with measurable result indicators, complemented by measurable Intermediate Outcome Indicators, make the project design robust. PPRPII was designed with clearly measurable indicators, and the same approach has been continued in the follow‐on project that recently become effective. The theory of change provided the framework for effective change management throughout implementation, accommodating the evolving drivers for political reform to ensure that results were effectively captured by refining the PDO and indicators in the AF and AF II. This was implemented by adopting a combined approach of political economy, relationship management, technical solutions, stakeholder management, and behavioral change communications to the grassroots level.

115. Skill mix in the World Bank’s team. More diversified skills in the World Bank’s project team, with greater intensity of continuous field‐based supervision and adequate budget provision, results in better project design and implementation. The PPRPII World Bank team includes a broad array of multi‐sectoral skills (including fiduciary, transport, power, water resources, communication, social, and ICT). This lesson has now been duly incorporated in the follow‐on new project, with even more diversified skills on the team, including skills in cutting‐edge technology such as Big Data, Open Contracting, and IT‐based social media.

116. Capitalize on enabling policy environment. When developing the technical approach to procurement reform in partner countries, the World Bank needs to leverage the enabling environment for reform in addition to identifying the challenges and roadblocks. From the outset of PPRPII, the World Bank recognized that the national procurement laws in Bangladesh allowed foreign bidders to participate in procurements without restriction regardless of national or international competitive bidding. Thus, the project did not need to address the issue of eligibility and fairness for foreign bidders separately, instead focusing on the reforms required for its development, capacity development, and rollout and sustainability of e‐GP with performance measurement.

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Objective/Outcome: Four target agencies fully introduce e‐GP in national competitive bidding

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Four target agencies publish PROMIS quarterly report for monitoring of procurement performance covering 90% of bids invited/ contracts awarded annually (2013‐ 20%, 2014‐ 50%, 2015‐ 70%, 2016‐ 90%)

Percentage 0.00 5.00 95.00 100.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Comments (achievements against targets):

Unlinked Indicators

Target

Percentage of contracts Percentage 10.00 60.00 82.00 85.00

awarded within initial bid validity period by the 4 target agencies (2013‐ 68%, 2014‐ 73%, 2015‐ 78%, 2016‐80%, 2017‐82%)

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Four target agencies expand electronic government procurement (e‐GP) to all 64 districts using national competitive bidding procurem ent (2013‐ 10%, 2014‐ 35%, 2014‐ 60%, 2016‐ 80%, 2017‐ 83%))

Percentage 0.00 3.00 83.00 95.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Percentage of procuring entities of 20 additional agencies with one trained/ certified procurement staff (2013‐ 10%, 2014‐ 30%, 2015

Percentage 0.00 70.00 78.00 80.00

31‐Dec‐2012 29‐Dec‐2016 30‐Jun‐2017 30‐Jun‐2017

‐ 50%, 2016‐ 70%, 2017‐ 78%)

Target

Number of public sector agencies registered in the e‐GP system

Number 0.00 130.00 200.00 800.00

31‐Dec‐2012 29‐Dec‐2016 30‐Jun‐2017 30‐Jun‐2017

A.2 Intermediate Results Indicators

Component: Capacity development program institutionalized locally and develop skilled professionals

Target

Percentage of procuring entities of 20 additional agencies with one trained/ certified procurement staff (2013‐ 10%, 2014‐ 30%, 2015‐ 50%, 2016‐ 70%, 2017‐ 78%))

Percentage 0.00 70.00 78.00 80.00

31‐Dec‐2012 29‐Dec‐2016 30‐Jun‐2017 30‐Jun‐2017

Unlinked Indicators

Target

Number of weeks of procurement training delivered by local training institute (ESCB) with its own fund/ resources outside of PPRPIIAF (2013‐ 5, 2014‐ 25, 2015‐ 45, 2016‐ 55, 2017‐ 55))

Number 0.00 2.00 55.00 150.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Percentage of contract awards published by four target agencies in CPTU's website for awards above PPR specified threshold (2013‐ 75%, 2014‐ 80%, 2015‐ 85%, 2016‐ 90%, 2017‐ 91%))

Percentage 10.00 70.00 91.00 100.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Percentage of complaints handled satisfactorily by four target agencies (2013‐ 35%, 2014‐ 45%, 2015‐ 55%, 2016‐ 70%, 2017‐ 72%)

Percentage 5.00 25.00 72.00 74.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Four target agencies publish PROMIS quarterly report covering 90% of bids invited/ contracts awarded (2013‐ 20%, 2014‐ 50%, 2015‐ 70%, 2016‐ 90%, 2017‐ 95%)

Percentage 0.00 5.00 95.00 100.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Percentage of contract/ bid Percentage 0.00 3.00 83.00 95.00

invited through e‐GP by four target agencies using NCB in 64 districts (2013‐ 10%, 2014‐ 35%, 2015‐ 60%, 2016‐ 80%, 2017‐ 83%)

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Expanding Government‐contractors forum to all disitricts and holding semi‐annual meetings (2013‐ 10, 2014‐ 20, 2015‐ 40, 2016‐ 64, 2017‐ 64))

Number 0.00 4.00 64.00 64.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

Target

Number of e‐GP awareness workshops in all districts (2013‐ 10, 2014‐ 20, 2015‐ 45, 2016‐ 64, 2017‐ 64))

Number 0.00 64.00 64.00 64.00

31‐Dec‐2012 29‐Dec‐2016 30‐Jun‐2017 30‐Jun‐2017

Target

Number of Public‐Private Stakeholders Committee (PPSC) workshops/ meetings held (2013‐ 7, 2014‐ 10, 2015‐ 12, 2016‐ 14, 2017‐ 14))

Number 0.00 6.00 14.00 12.00

23‐May‐2007 31‐Dec‐2012 30‐Jun‐2017 30‐Jun‐2017

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: Strengthened Public Procurement System

Results/Outcome Indicators 1. Percentage of procuring entities of 20 additional agencies with one trained/certified procurement staff 2. Number of public sector organizations registered in the e‐GP system

Intermediate Results/Outcome Indicators

1. Expand GCF to all districts and holding semiannual meetings 2. Number of weeks of procurement training delivered by a local institute (ESCB and BIM) funds/resources outside of PPRPII AF 3. Percentage of contracts/bids through e‐GP by four target agencies using NCB in 64 districts 4. Number of e‐GP awareness workshops in all districts 5. Number of PPSC workshops/meetings held 6. Percentage of procurement entities of 20 additional agencies with one trained/certified procurement staff (also Outcome Indicators)

Key Outputs (linked to the achievement of the Objective/Outcome 1)

1. Public sector organizations registered in e‐GP 2. GCF established in all 64 districts and GCF meetings held 3. Procurement trainings delivered by ESCB and BIM 4. On‐line e‐GP system developed (procurement planning to contract completion 5. e‐GP awareness workshop held in all 64 districts 6. PPSC formed and workshops/meetings held

Objective/Outcome 2: Improved Performance of Four Target Agencies

Result/Outcome Indicators

1. Percentage of contracts awarded within initial bid validity period by 4 target agencies 2. Four target agencies publish PROMIS quarterly report for monitoring of procurement 3. Four target agencies expand e‐GP to all 64 districts using NCB procurement

Intermediate Results/Outcome Indicators

1. Percentage of contract awards published by 4 target agencies in CPTU’s website for awards above PPR specified threshold 2. Percentage of complaints handled satisfactorily by 4 target agencies 3. Four target agencies publish PROMIS quarterly report covering 90% of bids/invited contracts awarded (also Outcome Indicator)

Key Outputs (linked to the achievement of the Objective/Outcome 2)

1. On‐line system developed in e‐GP to calculate procurement contract performance 2. PROMIS system made operational 3. CPTU’s website published contract award information as per the PPR threshold 4. Complaint handling system introduced in e‐GP

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation/Supervision/ICR

Zafrul Islam Task Team Leader

Ishtiak Siddique Procurement Specialist

Mohammad Reaz Uddin Chowdhury Financial Management Specialist

Zahed Hossain Khan Team Member

Shakil Ahmed Ferdausi Environmental Safeguards Specialist

Zubair K.M. Sadeque Team Member

Masud Mozammel Team Member

Marghoob Bin Hussein Team Member

Tanvir Hossain Team Member

Sabah Moyeen Social Safeguards Specialist

Jorge Luis Alva‐Luperdi Legal

Satish Kumar Shivakumar Disbursement

Srivathsan Sridharan Disbursement

Mehrin Ahmed Mahbub Team Member

Dipanwita Chakraborty Team Member

Nadia Sharmin Environmental Safeguards Specialist

Nafisa Rusmila Team Member

Mahmuda Nusrat Hussain Team Member

Paul Schapper Team Member (Consultant)

Peter‐Armin Trepte Team Member (Consultant)

Joseph Fagan Team Member (Consultant)

Omar Khandaker Team Member (Consultant)

Md. Rafiqul Islam Team Member (Consultant)

A.N.M. Mustafizur Rahman Team Member (Consultant)

S M Hafiz Al Mamun Team Member (Consultant)

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY06 2.273 10,213.96

FY07 21.345 159,117.93

FY08 0 34.41

Total 23.62 169,366.30 Supervision/ICR

FY08 21.954 94,351.73

FY09 29.127 110,014.91

FY10 31.772 91,176.86

FY11 33.714 103,110.60

FY12 26.115 103,030.96

FY13 20.782 122,950.69

FY14 10.763 32,582.22

FY15 9.506 25,472.67

FY16 28.106 207,418.49

FY17 13.216 113,733.66

FY18 3.634 68,639.56

Total 228.69 1,072,482.35

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$M) Actual at Project Closing (US$M)

Percentage of Approval

Component 1: Furthering Policy Reform and Institutionalizing Capacity Development

21.40 21.40 100.00

Component 2: Strengthening Procurement Management at Sector Level and CPTU/IMED

15.90 10.35 65.09

Component 3: Introducing e‐Government Procurement (e‐GP)

30.20 33.12 109.67

Component 4: Communication, Behavioral Change, and Social Accountability

5.40 4.87 90.19

Total 72.90 69.74 95.66

ANNEX 4. EFFICIENCY ANALYSIS

1. The costs incurred to achieve project objectives were reasonable in terms of the benefits achieved (as described in sections I–V), and the ‘value for money’ given the high probability of sustaining and building on the project outcomes of strengthening the public procurement system and improving the performance of the four target agencies (see section IIC). Because the project results could not be fully quantified in monetary terms, there was no calculation of an economic rate of return. Nevertheless, beyond the achievements made in terms of efficiency, transparency, and competitiveness, the other major benefits of the project is the sustainability of major interventions, that is, building the institution for regulating public procurement that is fully funded from the Government’s own resources (CPTU); establishing the e‐GP system that is becoming self‐sustainable by generating its own revenues greater than expense; and institutionalizing procurement management capacity within the country using collaboration of local institutes with the internationally reputed organizations. Despite such achievements, there are still areas that need improvements, as summarized in the following paragraphs.

2. Overall system efficiency in public procurement had improved with this intervention of PPRPII. The country has one uniform system for all its public procurement organizations regardless of the sector, resulting in consistency of documentation throughout the entire procurement cycle. This streamlined process and uniformity of documents also helped the bidding community in reducing bid preparation time for different organizations. The end‐line evaluation shows an overall satisfaction level of about 69 percent in the use of public funds with variations as follows in the type of stakeholders: bidding community ‐ 76 percent, procurement officials ‐ 78 percent, civil society ‐ 48 percent, and media professionals ‐ 30 percent.

3. Respondents agreed that the improved delegation of approving authority in the later part of the project, and the sub‐delegation of that authority within organizations, speeded up the approval process in general, in particular, reference to the contract decisions made at decentralized levels or within the agency level. Also, this has been clearly demonstrated through reduction of procurement processing time in the results indicators. Nevertheless, still there are major issues of procurement delays, especially in the large‐value contracts requiring approval of the ministry and/or cabinet. Also, the propensity to push decisions to higher‐than‐necessary levels continues, thus increasing the time taken for approval.

4. Introduction of e‐GP has contributed to substantial improvements in improving the overall efficiency of the system, with specific reference to reducing inappropriate/coercive/collusive bidding practices, improving speed of the procurement process, significantly reducing the number of complaints, and reducing overall transaction costs. An analysis done for the four key agencies shows a savings range between 13 percent and 20 percent compared with the traditional tendering practices.

5. Overall, the complaints handling mechanism has built confidence levels, especially with the independent review panel mechanism. Review shows that in about 45 percent cases, the decision of this review panel was in favor of the bidders while about 55 percent was in favor of the procuring entities. This is a reasonable signal that the system is functioning well. Nevertheless, results also show that there are issues in dealing with complaints at some of the implementation agencies’ level, particularly in reference to the soft sectors, with some variations for the other sectors. The mechanism needs improvement for those agencies.

6. Self‐sustainability of the e‐GP system has increased substantially over time. The e‐GP system‐generated revenues have been exceeding the expenses being incurred by the system. Earnings are as follows: FY14: US$1.8 million; FY15: US$4 million; FY16: US$5 million; FY17: US$8.8 million (sale of bidding documents: US$7.67 million; registration of bidders: US$0.75 million; and renewal registration: US$0.38 million). Average annual cost

of operating and managing the system is about US$2.5 million. This clearly demonstrates that the system is self‐sustaining.

7. In conclusion, it may be said that though PPRPII made substantial contributions in bringing about a systemic change in the public procurement environment of the country, there are still systemic issues that affect the efficiency gains. It has been demonstrated that some of the problems can be effectively addressed by the procuring entities themselves, or by their controlling ministries, simply through greater respect for and sincerity toward current procurement laws/rules.

ANNEX 5. DISBURSEMENT LINKED INDICATORS

I: Improving procurement performance by using e‐GP in the Target Agencies

DLI DLI Target (Number of bids)/DLI Value (US$, millions) per Target Agency DLI Verification

Protocol

DLI Period Period 0

(July 1, 2012–June 30, 2013)

Period 1 (July 1, 2013–December 31,

Period 2 (January 1, 2014 –June 30, 2014

Period 3 (July 1, 2014–June 30, 2015)

Period 4 (July 1, 2015–June 30, 2016)

DLI 1: Minimum number of bids invited in NCB using e‐GP by each of the Target Agencies

Definition: All bids invited under NCB across the target agency regardless of location of its procuring entities (HQ, districts, and sub‐districts) will be eligible. Source/Verification: The target agency’s e‐GP/PROMIS cell reconciled data/reports based on actual invitations of bids by various procuring entities. The procurement monitoring coordinator of the target agency will report the DLIs achievement to the

RHD 100/ 0.20 400/0.20 1,000/0.20 2,400/0.20 3,200/0.20

LGED 100/0.30 400/0.30 1,000/0.30 2,400/0.40 3,200/0.40

BWDB 60/0.19 120/0.19 300/0.19 720/0.19 960/0.19

REB 8/$0.06 15/0.06 40/0.06 96/0.06 128/0.06

CPTU for validation by the Association. The DLIs will also be verified by an independent consultant contracted by the recipient.

II: Improving procurement performance monitoring using PROMIS by the Target Agencies

DLI DLI Target (Number of bids) and DLI Value (US$, millions) for Each Target Agency DLI Verification

Protocol

DLI Period Period 0

(July 1, 2012 – June 30, 2013)

Period 1 (July 1, 2013 – December 31,

Period 2 (January 1, 2014 – June 30, 2014

Period 3 (July 1, 2014 – June 30, 2015)

Period 4 (July 1, 2015 – June 30, 2016)

DLI 2: Minimum number of tenders entered into published PROMIS report, with procurement performance indicators covering various stages from bid invitations to award of contracts

Definition: Monitoring of procurement steps forward for all bids invited under NCB during the corresponding period of the respective DLIs across the target agency regardless of location of its procuring entities (HQ, districts, sub‐districts). Source/Verification: The target agency’s e‐GP/PROMIS cell reconciled data/reports based on actual invitations of bids by various procuring entities. The procurement

RHD 200/0.20 800/0.20 1,200/0.20 2,800/0.20 3,600/0.20

LGED 200/0.30 800/0.30 1,200/0.30 2,800/0.30 3,600/0.30

BWDB 120/0.19 240/0.19 360/0.19 840/0.19 1,080/0.19

REB 16/0.06 30/0.06 45/0.06 112/0.06 144/0.06

DLI DLI Target (Number of bids) and DLI Value (US$, millions) for Each Target Agency DLI Verification

Protocol

monitoring coordinator of the target agency will report DLIs to the CPTU for validation by the Association. The DLIs will also be verified by an independent consultant contracted by the recipient.

ANNEX 6. BORROWER, CO‐FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS

Government of the People's Republic of Bangladesh Ministry of Planning

Implementation Monitoring & Evaluation Division (IMED) Central Procurement Technical Unit (CPTU)

Sher‐e‐Bangla Nagar, Dhaka‐1207 Tel: 9144252‐3, Fax: 9180968, E‐mail: info@cptu.gov.bd; Website: www.cptu.gov.bd

21.00.0000.367.14.981.17.425 17 December 2017 Subject: Comments on Implementation Completion and Result Report (ICR): Public Procurement Reform Project II (P098146) [PPRP II: Cr. 58180, 52420 and 43500‐BD] The undersigned is directed to inform you that the draft ICR on PPRP II project has been reviewed by the CPTU. The CPTU has agreed almost on all assessment on implementation. However, the followings are the observations on the ICR: (1) In paragraph 24, line 4, Clause (i): the word eighteen will be replaced by seventeen. Line 5, Clause (ii): the words training for 5 days will be added after e‐GP. In line 8, first word Government will be replaced by Governance. (2) In Table 3, after paragraph 25, Figures put in Column 2 and 3 will be replaced as under:

Capacity Development Programme Type # of Beneficiaries Comments

14083 17 topics….

No Change The following words will be deleted “3 Week Training top performer &”

(3) In paragraph 34, line 8: 2012 will be replaced by 2011; (4) In paragraph 40, from line 4 onward the paragraph will be reconstituted as: “The key activities included: conducting 99 events on procurement reforms including 64 Future Search Conference at District level, National Level Launching event of Social awareness program, National Level Launching of e‐GP having Prime Minister as Chief guest covering over 5,700 participants; convening 64 e‐GP awareness workshops at district level with over 2,400 participants, establishing 64 Government and Contractor’s Forums (GCF) covering over 3391 PEs and tenderers, and organizing four e‐GP workshops for 44 registered Bank’s in the e‐GP system, one at the national level in Dhaka and three in three were held in the Divisional Headquarters e.g. Sylhet, Rajshahi and Khulna. The GCF provided an informal platform for GOB officials and bidders to share information and experience including issues and resolutions about procurement and e‐GP system. BCCP also developed a mobile app, produced an e‐GP theme song, videos, radio and

television commercials, TV talk shows, success story videos, info graphics; disseminated mobile SMS, TV scrolling messages, email and on line marketing and created two digital billboards in Dhaka that displays on‐line live procurement data with a direct feed from the e‐GP platform and conducted opinion and satisfaction surveys. CPTU produced newsletters, managed social media, arranged TV talk‐shows and numerous news, articles in media on public procurement issues.” (5) In paragraph 41, the second last sentence will be reconstituted as: “BIGD held four divisional workshops, produced two documentaries on citizen engagement, formed and trained citizen committees and got monitored a number of public works projects and distribution of school text books.” (6) In paragraph 42, line 5: the word attend will be replaced by attended by CPTU and Bank officials. You are kindly requested to instruct the concerned wing to forward these comments to the World Bank for incorporating into the ICR for finalizing the same.

Shish Haider Chowdhury

Director (Coordination and Training) Tel: 9144232

Secretary Economic Relations Division Ministry of Finance Block No. 8, Room No. 3 Sher‐e‐Bangla Nagar Dhaka Copy to:

1. Ms. Mahmuda Begum, Additional Secretary, ERD; 2. Dr. Zafrul Islam, Lead Procurement Specialist and Task Team Leader, DIMAPPP; 3. PS to Secretary, IMED, Ministry of Planning

ANNEX 7. SUPPORTING DOCUMENTS

2002: The World Bank: Bangladesh Country Procurement Assessment Report

2005: BD National Strategy for Accelerated Poverty Reduction

2006: The World Bank BD: Country Assistance Strategy

2006: BD Assessment of Readiness for Introduction of Electronic Bidding

2007: Public Procurement Reform Project II‐ PPRPII (P098146)‐ Project Appraisal Document

2007–2012: Procurement Management Information System Reports

2007–2017: The World Bank Project Implementation Status Reports

2007–2017: Capacity Development Consultants Progress Reports

2007–2017: M&E Consultants Semi‐Annual Reports

2007–2017: Behavioral Change Communication Consultant’s Reports

2011: BD Sixth Five‐Year Plan‐ Accelerating Growth and Reducing Poverty

2011: The World Bank BD: Country Assistance Strategy

2013: PPRPII Additional Financing (P132743)‐ Project Appraisal Document

2013–2017‐ Quarterly e‐Procurement Management Information System (e‐PMIS) Reports

2013: PPRPII Additional Financing (P132743)‐ Project Appraisal Document

2016: BD Seventh Five‐Year Plan‐ Accelerating Growth and Empowering Citizens

2016: The World Bank BD: Country Partnership Framework

2016: PPRPII Second Additional Financing (P158783)‐ Project Appraisal Document

2016: CPTU restructuring and e‐GP Study Report

2017: Consultant’s Citizen Engagement Report

2017: Consultant’s End‐Line Evaluation Report

Document of The World...

Documents

ECA Knowledge Brief - World Bankdocuments.worldbank.org › curated › en › 372151468092957477 › pd… · органы местного самоуправления в сотрудничестве

وثيقة البنك الدولي - World Bankdocuments.worldbank.org/curated/en/948291468266162325/... · Web viewوثيقة البنك الدولي للاستخدام الرسمي

Public Disclosure Authorized - World Bankdocuments.worldbank.org/curated/en/954541468170349892/pdf/539… · ح٦٬ُٞ ً٬ـًؾسٝج٧رػحـل٠ٝج٧٫ـيجفٰٝرـ٠حِٝجرث٬٦ٝج

Public Disclosure Authorized - World Bankdocuments.worldbank.org/curated/en/306471468251761064/...ცხრილი 2.4. წარმოების ფუნქციების

LITHUANIAN HOUSING STRATEGY - World Bankdocuments.worldbank.org/curated/en/355461508490990967/...ОЦЕНКА НА ЖИЛИЩНИЯ СЕКТОР В БЪЛГАРИЯ (ОКОНЧАТЕЛЕН

ХУРААНГҮЙ - World Bankdocuments.worldbank.org/curated/pt/512491468286291144/... · Web viewИйм асуудлаас болж Ашиг сонирхлын зөрчлийг

Russia Ag Policy Note RUS - World Bankdocuments.worldbank.org/curated/en/...Несмотрянаулучшениепоказателейсельскохозяйственного производства

APPENDIX: ANALYSIS OF SUSTAINABILITY - World Bankdocuments.worldbank.org/curated/en/402151468173962496/071652160... · APPENDIX: ANALYSIS OF SUSTAINABILITY ... Evaluation of The World

“План Мероприятий по Переселению” - World Bankdocuments.worldbank.org/curated/en/707921468273643973/... · 2016-07-11 · Коридор «Западная

World Bankdocuments.worldbank.org/curated/en/677281468032970584/...EAP FLEG Jan/03 TF/AG Meeting

OLLJ C srl$l -T - World Bankdocuments.worldbank.org/curated/en/... · OLLJ C srl$l -T - World Bank ... --j

agribusiNess iNdicators: ethiopia - World Bankdocuments.worldbank.org/curated/en/631391468008109813/pdf/682370ESW0P1260ators0...agribusiNess iNdicators: ethiopia Public Disclosure

SFG1843 Public Disclosure Authorized - The World Bankdocuments.worldbank.org/curated/en/... · ေနထိုင်မှုမရှိသည့်အပြင် ရှိပြီးေသာသဲ၊

ICR IL - World Bankdocuments.worldbank.org/curated/en/729721468044083800/... · Web viewDocument of The World Bank Report No: ICR00001845

مجموعة البنك الدولي - World Bankdocuments.worldbank.org/curated/pt/162331468247460930/... · Web viewالإطار 3: النساء والوظائفتبلغ نسبة

general Principles - The World Bankdocuments.worldbank.org/curated/en/983621468241197481/... · Web viewSFG1572 V3. World Bank Financed. Ningbo Sustainable Urbanization . Demonstration

УКРАЇНА - World Bankdocuments.worldbank.org/curated/en/271141468338061859/... · 2016-07-19 · Україна має розвинуту інфраструктуру, успадковану

Document of - World Bankdocuments.worldbank.org/curated/pt/... · Document of The World Bank FOR OFFICIAL USE ONLY

Public Disclosure Authorized - World Bankdocuments.worldbank.org/curated/en/433151481108871251/...RETROSPECTIVE PROGRAMMATIC MAPPING MODELLING ANALYSIS SYNOPSIS REPORT 2016 World Bank

Благодарность - World Bankdocuments.worldbank.org/curated/pt/625881468167640458/... · Web view11. Реформы в секторе образования Таджикистана