Operation Performance Evaluation Review

Komi Municipal Water Services Development Project

Russian Federation(A Technical Cooperation operation)

May 2009

ab0cd

OPER No: PE08-419T

Operation Code: 22163

Board Report: BDS03-143

Evaluation Department (EvD)

Operation Performance Evaluation Review (OPER)

PREFACE

This evaluation report The subject of this Operations Performance Evaluation Review (OPER) report is the technical cooperation (TC) operation Komi Municipal Water Services Development (the TC) in the amount of €1.67 million and funded through the European Bank for Reconstruction and Development’s (EBRD) Technical Cooperation Funds Programme (TCFP) from contributions by Canada, Finland, the European Union and the Northern Dimension Environmental Partnership Support Fund. The related TC assignments were carried out during the period from 2001 to 2008. The operation leader (OL) of this TC was Alexander Rogachevsky. The operation team and other relevant Bank staff commented on an early draft of this report. The Basic Data Sheet on page iii of this report and the project completion reports (PCRs) in Appendix 4 are complementary to this OPER and are designed to be read together. The evaluation was carried out by Victoria Millis, Evaluation Analyst (the OPER team). Information on the operation was obtained from relevant teams and departments of the Bank and its files as well as from external sector and industry sources. Fieldwork was carried out in September 2008. Appendix 1 presents a list of contacts. The Evaluation Department (EvD) would like to take this opportunity to thank those who contributed to the production of this report. Post-evaluation selection and process Selection of an operation for post-evaluation by EvD uses the following criteria: • relevance to the Bank’s likely future operations • lessons-learned potential • size of the Bank’s investment commitment/exposure • balance among countries of operations • balance among sectors and types of operations • relative priority of investment operation OPERs within EvD’s overall work

programme priorities and resources. The Bank’s post-evaluation process is described in Chapter 8 of the Operations Manual. The responsible operation leader first writes a TC PCR. The PCR serves a self-evaluation function and establishes the basic facts and lessons from the operation’s preparation, implementation, and outcome. EvD’s independent evaluation follows, using the PCR as one of several inputs.

1

TABLE OF CONTENTS

Preface 1 Abbreviations and defined terms 3 Basic data sheet 5 1. The project 6

1.1. The investment operation 6 1.2. Change of scope 7 1.3. The TC operations 8

2. Project rationale 12 3. Achievement of objectives 12

3.1. Overall achievement of objectives 12 3.2. Individual achievement of objectives 13

4. Overall assessment 19 5. Transition impact and the Bank additionality 19

5.1. Company impact 20 5.2. Impact on the industry and on the economy as a whole 21 5.3. Environmental impact 22 5.4. Country strategy and sector policies 22 5.5. Additionality 23

6. Bank handling 23

6.1. Consultant procurement and contracts administration 24 6.2. Project preparation and definition 24 6.3. Relations with the client and consultants 25

7. Key OPER issues and lessons learned 25

7.1. Using existing networks to promote the Bank’s work 25 7.2. Institutional development 26 7.3. Targeting institutional development TCs at the correct level for the client 26 7.4. Adequacy of TC budget and consultant selection 27 7.5. Staffing of the PIU 28

List of appendices Appendix 1 Operations performance ratings Appendix 2 Transition impact analysis

2

ABBREVIATIONS

BD Banking Department CAP Consolidated action plan CDP Corporate development support programme CEP Creditworthiness enhancement programme CIDA Canadian International Development Agency EAP Environmental action plan EMS Environmental management system ESD Environment and Sustainability Department EIRR Economic internal rate of return EU European Union EvD Evaluation Department FIRR Financial internal rate of return GOST Russian state procurement standards IAS International Accounting Standards IFI International finance institution ISO International Organization for Standardization IT Information technology MIS Management information systems NDEP Northern Dimension Environmental Partnership OCE Office of the Chief Economist (EBRD) OGC Office of the General Counsel (EBRD) OL Operation leader OPER Operation Performance Evaluation Review OpsCom Operations Committee OT Operation team PIU Project implementation unit RUB Russian rouble SIDA Swedish International Development Cooperation Agency SNiP Russian construction norms and rules SPVK St. Petersburg Vodokanal SV Syktyvkar Vodokanal TC Technical cooperation TC Com Technical Cooperation Committee TOR Terms of reference USD United States Dollar VT Vorkuta Teplovodokanal XMR Expanded monitoring report

3

DEFINED TERMS

the Bank European Bank for Reconstruction and Development the companies Syktyvkar Vodokanal and Vorkuta Teplovodokanal the cities Syktyvkar and Vorkuta the Republic Komi Republic the OPER team Staff of the Evaluation Department and the independent sector

consultant who jointly carried out the post-evaluation the operation team Staff in the Banking Department and other respective

departments within the Bank responsible for the operation appraisal, negotiation and monitoring, including the XMR

the project Komi Municipal Water Services Development Project

4

BASIC DATA SHEET Operation Code 25018, 30304, 30306, 34746, 38480 Location: Russian Federation Operation: Komi Municipal Water Services Development Project Sector: Water & Wastewater Type: Technical Cooperation Facilitators: Canada, EU, Finland, NDEP Bank Unit: Municipal & Environmental Infrastructure

A. Funding TC TCFP Commitment Commitment

number Commitment title Amount (€)

TC1 CA3F-2002-06-01 CAN-2002-01-01

Corporate development support programme

€501,187

TC2 FIN-2004-10-10PS Project implementation unit €188,499 TC3 EIPF01-2004-12-12 Creditworthiness enhancement

programme €248,320

TC4 NDEP-2004-12-04 NDEP-2007-08-04

Engineering design, procurement, contracts supervision (and extension)

€735,000

B. Procurement

Mode Sources by country TC1 Competitive selection Canada TC2 Competitive selection Finland TC3 Competitive selection Russia TC4 Competitive selection Russia C. Visits Type of visit No. of visits Person-days EvD/OPER 1 4

5

1. The project 1.1. The investment operation On 25 November 2003 the European bank for Reconstruction and Development’s (EBRD) Board approved the Komi Municipal Water Services Development Project, consisting of loans of €10 million and €5 million, respectively, to the municipal water companies of Syktyvkar and Vorkuta (the cities) in the Komi Republic (the Republic), located in the north of European Russia. The loans would be made directly to the water companies, Syktyvkar Vodokanal (SV) and Vorkuta Teplovodokanal (VT) (the companies), with a guarantee from the Komi Republic and municipal support agreements from the cities. The transaction would be co-financed by a grant of €5.9 million from the Northern Dimension Environmental Partnership (NDEP) support fund. The project would also be supported by technical cooperation (TC) funds of €900,000 from the same source and a further €1.7 million from other sources. A principal aim of the investment was environmental improvement through reducing the level of polluting wastewater discharge into the Barents Sea basin. Transition impact would be achieved through: • restructuring and commercialisation of the water companies • adopting a tariff policy ensuring cost recovery • creating economic incentives for rational consumption of water by implementing

the tariff policy and a metering programme • rationalising the relationship of the cities and companies in relation to subsidised

consumers. Box 1: The Northern Dimension Environmental Council The Northern Dimension was launched in December 1999 by the European Council as a demonstration of European cooperation. It aimed at providing a common framework for the promotion of dialogue and concrete cooperation, strengthening stability, well-being and intensified economic cooperation, promotion of economic integration and competitiveness and sustainable development in northern Europe. It has four partners: the European Union (EU), Iceland, Norway and the Russian Federation. Its priorities include protection of the marine environment in the Baltic and Barents Seas. On 11 December 2001, in order to provide a comprehensive multilateral approach to environmental projects in the Northern Dimension Area, the EBRD’s Board approved the Northern Dimension Environmental Partnership Support Fund. The Fund, managed by the EBRD, had the aim of strengthening and coordinating financing of important environmental projects with cross-border effects in the Northern Dimension area of Europe. An inaugural pledging conference for the support fund took place on 9 July 2002 in Brussels. The first assembly of contributors met on 15 November 2002 and decided the allocation of grant funds to a number of projects, including the Komi Municipal

6

Water Services Development Project. Funds were provided to the project both as an investment grant for specific procurement items and for one of the related TCs, the engineering, design, procurement and contracts supervision assignment (TC4). Following the exit of Vorkuta from the project, the grant for Syktyvkar totalled €3.72 million, including €600,000 for TC. The TC commitment was the first from this fund to be processed and implemented. The impetus for the project came from SV. The general director was in contact with his counterpart in St Petersburg, where the EBRD had been involved in a water rehabilitation project since 1997 (see EvD OPER report PE04-287). Impressed by the progress made in St Petersburg, he persuaded his municipality to approach the EBRD to implement a similar project with SV. The signing of the loan agreement occurred around 11 months later than planned partly owing to delays in TC aspects but, more especially, to a delay in obtaining the necessary guarantees from the Republic the cities pertained to. These guarantees were required before the Federal Ministry of Finance would give its “no-objection” approval to the project. Ultimately, the guarantee for Vorkuta was not forthcoming (see Section 1.3 below) and the project went ahead in Syktyvkar only. As is not uncommon with projects of this type, the procurement process fell somewhat behind schedule and some components turned out more expensive than expected. Cost increases were exacerbated by the time lapse and intervening inflation. In May 2007, SV obtained approval from the NDEP for €135,000 of TC funding originally intended for Vorkuta to be transferred to the Syktyvkar project. In November of the same year, approval was granted for €2.2 million of grant funds to be transferred in the same way, on the condition that the EBRD approved a similar request for the €5 million loan originally planned for Vorkuta to be transferred to the Syktyvkar project. A concept review for a €5 million loan increase went to the Operations Committee (OpsCom) in May 2007 (OpId 37880). However, the request has not progressed further pending internal approvals from the Republic. In late 2008 the client experienced the effects of a major increase in the Moscow prime rate. After trying unsuccessfully to fix the rate, the client decided against pursuing an increase in the loan amount and instead proposed to cancel the undisbursed portion of the loan, around 19 per cent of the original amount. In the meantime, some of the intended savings and benefits have not been realised as the investment programme is still under way. 1.2. Change of scope In August 2004 the team reported to the Bank’s Technical Cooperation Committee (TC Com) that the implementation schedule of the Vorkuta part of the project remained unclear. They proposed freezing the TC assignments relating to Vorkuta and proceeding with Syktyvkar only at that stage. A memo to the Board in January

7

2005, requesting re-approval of the still-unsigned Vorkuta project, related the problem to difficulties in restructuring the debts of VT, particularly its tax arrears. VT faced particular problems because it was highly dependent on a single customer, the coal mining company, which was in arrears. Vorkuta itself is shrinking as coal production falls. Its population has halved in recent years. Located in the inhospitable far north, it relies on subsidies from the Republic. The Republic concluded that the large-scale investment, on top of the cost of settling the tax arrears, was not cost effective for a shrinking city and chose not to provide the guarantee. The Vorkuta operation has now been cancelled by the EBRD. 1.3. The TC operations The current report considers the TC support provided through the EBRD, as listed in Table 1 below. Table 1: Scope of the TC evaluation

TC commitment No

TC name/description Amount (€)

Funder Current status

TC1

CA3F-2002-06-01 / CAN-2002-01-01

Corporate development support programme

0.60 CIDA (Canada)

Complete

TC2

FIN-2004-10-10PS Project implementation unit 0.20 Finland Complete

TC3

EIPF01-2004-12-12 Creditworthiness enhancement programme

0.25 EU Complete

TC4

NDEP-2004-12-04 / NDEP-2007-08-04

Project management including engineering, procurement, design and contract supervision

0.74 NDEP Extended 2007 Ongoing

The Board document listed two other elements of associated TC: the technical feasibility study funded by SIDA (Swedish International Development Cooperation Agency) on a bilateral basis and the network modelling and optimisation of operations, which was to be funded by the water companies themselves. These two items did not pass through the Bank’s TC funds programme and are outside the scope of this study. The performance of the investment operation also falls outside the scope of the study.

8

Chart 1: structure of the project and TC support

KOMIREPUBLIC

EBRDSYKTYVKARVODOCANAL

SYKTYVKARCITY

VORKUTACITY

Financial Guarantee

Counter-guarantee

Loan Agreement

Municipal Project Support Agreement

Loan Agreement

Financing of Equipment

Counter-guarantee

Financing of Equipment

VORKUTAVODOCANAL

Municipal Project Support Agreement

NDEP GRANT

PIUTC2: PIU support

TC4: Project Manager

TC1: CDP

TC3: CEP

Vorkuta portion was cancelled.

1.3.1. TC1: Corporate development support programme The corporate development support programme was presented to the TC Com on 14 November 2001, a month after the investment operation had passed concept review. Budgeted at €600,000, it was intended to consist of two phases: Phase I, to be carried out in parallel with the feasibility study, was to help the Syktyvkar and Vorkuta water utilities to prepare a corporate development plan (CDP). The CDP was intended to outline the steps necessary to transform the utilities’ financial, operational and managerial structure into efficiently run companies in accordance with good corporate governance principles. Phase II was to assist the companies in implementing the CDP once the loan and grant agreements with the Bank and co-financiers had been signed. Funding was obtained from Canada and the procurement notice was published in February 2002. Twelve firms responded, of which three were from ineligible countries. The selection committee, which consisted of six representatives of the Republic, the cities and the companies, short listed four companies. Proposals were received from three. The same committee, with a member of EBRD staff as an observer, made the final selection in January 2002. 1.3.2. TC2: Project implementation unit Originally the team presented two separate assignments to the TC Com, on 24 September and 1 October 2003, for the environmental action plan (EAP) and the project implementation unit (PIU) respectively. Each proposed assignment would cost €300,000.

9

• The environmental action plan TC was intended to support the cities in implementing the EAP prepared by Swedish consultants as part of the technical feasibility study. The TC provided for the consultant to arrange a twinning partner for each city and its water utility and assist in drafting legislation necessary to implement the EAP. The relevant laws should be passed before the completion of the consulting assignment.

• The project implementation unit assignment required consultants to assist the

companies in establishing PIUs to manage the project in each city; advise and train the PIUs on procurement; and establish and implement mechanisms for project-related financial operations, control and monitoring. The budget incorporated funding for many of the staff members of the PIUs.

Following discussion at the TC Com, the two assignments were merged into one with a total budget of €300,000 (that is, half as much as originally envisaged). Savings were achieved by specifying that consultants would no longer be responsible for twinning arrangements, implementation of legal infrastructure or staffing the PIU. Instead they would advise and train local staff on implementation and institutional arrangements. The main part of the terms of reference (TOR) for the EAP assignment was attached as an annex to the revised, combined TOR that was approved by the TC Com on 23 October 2003. Funding was obtained from Finland, and the procurement notice published in February 2004. Two consultancy companies submitted expressions of interest and were invited to submit proposals. As agreed at the TC Com, the selection was made by the EBRD on behalf of the client because of distances and logistics involved. It was during the negotiation period that the freeze on the Vorkuta project was approved. The consultant was asked to prepare a revised financial proposal for Syktyvkar only, with a budget of €200,000. The contract was signed in November 2004. 1.3.3. TC3: Creditworthiness enhancement programme The creditworthiness enhancement programme (CEP) was approved by the TC Com, substantially without amendment, a week before the PIU consultancy in September 2003. It aimed to improve the financial performance and hence creditworthiness of the Republic of Komi and the two cities. The EBRD received expressions of interest from 16 parties and a shortlist of four was selected in May 2004. Three full tenders were received and the final selection, again delegated to the EBRD, was made in August 2004. Following negotiations the contract was issued in January 2005 for work commencing 17 January 2005 for 10 months. An amendment to the contract dated 26 April 2005 reduced the budget from €299,050 to €223,050. By this amendment, the Vorkuta part of the contact was removed, and some of the funds were added to the Syktykar project and the remainder cancelled.

10

1.3.4. TC4: Project management including engineering, procurement, design and contract supervision

The project management assignment was approved by the TC Com on 23 October 2003, at the same time as the revised TOR for the PIU assignment. The TC Com approval was for a €900,000 assignment to cover both cities. The committee (more specifically, the CSU) queried the fee rates, which were approximately 30 to 40 per cent lower than those currently paid for similar expertise. It was agreed that a revised budget breakdown would be prepared. The shortlisting and selection for this assignment, as for the PIU and CEP, were delegated to the EBRD. Six consultants were shortlisted in April 2004. By May 2004 it had become apparent that the Vorkuta project might need to be separated from the Syktyvkar project in order to allow Syktyvkar to go ahead without further delay. The invitation to tender was issued in June 2004 with a TOR in two sections, budgeting €600,000 for Syktyvkar and €300,000 for Vorkuta, as per TC Com approval. In August, the EBRD wrote to the shortlisted companies explaining that the Vorkuta subproject would be postponed, requesting an amended proposal covering Syktyvkar only and extending the deadline for proposals to allow this. Four of the six shortlisted consultants pulled out, three of them citing an insufficient budget. Two proposals were received and opened. Following evaluation, a preferred bidder was selected on 13 September and negotiations commenced. In December 2004, the inception report from the PIU consultant recommended that neither proposal conformed to the TOR. It advised that the budget needed to be more than doubled (from €600,000 to €1.3 million) to allow the project manager to provide “design of works in accordance with the Russian State Standards (GOST) and Construction Norms and Rules (SNiP)” as specified in the TOR. The two proposals received had overcome this by minimising the design of works they would perform: they would prepare the concept only and the detailed design would be delegated to the contractors. The PIU consultant warned that this would lead to delays and greater expense overall. The second-placed consultant, a Russian company and, therefore, cheaper than the Finnish alternative, had incorporated plans to take the design further than the first-placed consultant and allowed a lot more consultant time for project implementation. It was largely as a result of the advice from the PIU consultant that the negotiations with the first-placed bidder were terminated and negotiations began with the second placed company. These were concluded quickly and the contract signed in March 2005. In August 2007, after the client had obtained approval from the NDEP steering committee to use funds intended for Vorkuta on the Syktyvkar project, an amendment to the contract extended the budget by a further €135,000.

11

2. Project rationale The Republic of Komi is a priority region for the NDEP. The Republic is situated in north-western Russia with the large rivers of Northern Dvina and Pechora flowing through it into the Barents Sea basin. Although the Republic is one of the wealthiest regions of the Russian Federation (thanks to oil and gas, bauxites, pulp and paper, and electricity generation), its municipal infrastructure was obsolete, inefficient and the cause of health and environmental concerns. The main objective of the Republic, the cities and the companies was to achieve sustainability in accordance with Russian and EU regulations in their water and wastewater infrastructure and services. The long-term investment requirements to upgrade municipal water and wastewater infrastructure in the city of Syktyvkar to EU standards were estimated to exceed €90 million. The project would finance only priority investment components at a cost of up to €20 million. The project would constitute a first step of a long-term capital investment programme of the companies improving their efficiency and reducing their operation and maintenance costs, water losses and discharges of untreated sewage into the Barents Sea basin. The project was also intended to contribute to the institutional strengthening of the companies, the cities and the Republic in respect of procurement, corporatisation and enhancement of creditworthiness. The TC components were specifically targeted at achieving these objectives, as well as ensuring the satisfactory implementation of the project through the project management assignment (TC4). 3. Achievement of objectives 3.1.Overall achievement of objectives The overall achievement of objectives is rated Satisfactory. Table 2 summarises the achievement of individual TC objectives and the overall rating of each TC for this indicator. The individual TC assignments are discussed in more detail in Section 3.2 below. Table 2: Achievement of objectives TC Objective Achievement of

objective TC rating

Define and implement the financial and operational improvement programme

Partly achieved

Support the adoption and implementation of a sound tariff policy

Not achieved

TC1: Corporate development support programme

Outline a long-term corporate strategy for the Vodocanals

Partly achieved

Marginal

TC2: Project implementation unit

Ensure the proper and effective set up of the PIU, including PIU staffing and essential training of selected experts

Fully achieved Excellent

12

Advise the companies and the Bank on any improvements to the Structure and the TOR for the PIU, where deemed necessary

Fully achieved

Establish proper business processes within PIU in order to implement the project effectively and in a timely manner

Fully achieved

Set up a mechanism for the development and implementation of an EAP for the project

Fully achieved

TC3: Creditworthiness enhancement programme

Enhance the cities and Republic’s creditworthiness by improving all aspects of their financial management and planning

Fully achieved Good

TC4: Project management

Facilitate the timely and effective implementation of the project

Partly achieved Marginal

Overall achievement of objectives Satisfactory

3.2. Individual achievement of objectives 3.2.1. TC1: Corporate Development Support Programme Box 2: Objective of TC1 • define and implement the financial and operational improvement programme • support the adoption and implementation of a sound tariff policy • outline a long-term corporate strategy for the Vodocanals. The assignment was split into two phases, of which Phase I involved the preparation of the CDP and Phase II was the implementation of the plan. Phase II would only start once the steering committee was satisfied with the CDP and the loan agreement had been signed. The detailed TOR for Phase II would be elaborated once the CDP had been approved. The objectives for this TC are considered to be Partly Achieved. The contract specified that work would begin on 4 September 2002 and take 18 months, with the CDP to be completed by the end of week 14, that is, 11 December 2002. The consultant visited Syktyvkar and Vorkuta in the autumn of 2002 but delivered a draft CDP only in October 2003. As early as January 2003, the client wrote to the operation leader (OL) to complain that the consultant was failing to achieve the tasks in the contract. In the same month, the OL met representatives of the consultant and agreed a new timetable for the provision of acceptable deliverables. On submission of the first two invoices in February and May 2003 (covering the period September 2002 to February 2003), the client approved only a part of the invoice. Specifically, it accepted only 70 per cent of the services, reduced the fees of the project director from €875 to €500 a day and excluded her bill for costs “due to the Recipient exams (sic) to request of her dismissal as incompetent”. The remaining 30 per cent would only be approved “after deliverables specified for Phase I of the Consultancy Contract Appendix A are completed to the Recipient satisfaction”. The

13

balance of the second invoice was not paid until September 2005 following negotiations among the Bank, the office of the Canadian director, consultant and client (see under “Bank handling”). Completion of the CDP was a condition precedent for disbursement of the loan. As SV was dissatisfied with the performance of the consultant, the PIU in Syktyvkar undertook to complete the CDP with itself. It appears that the draft CDP was used as a basis, and the various sections of the document were amended and updated by the relevant departments of SV, with the PIU pulling the whole document together. It had assistance from two sources: the PIU consultant and colleagues in the St. Petersburg Vodokanal (SPVK), who were further ahead in implementing a similar project with the EBRD. The PIU also prepared the service agreement with the city administration, which incorporated many of the plans set out in the CDP as formal undertakings. The TOR stated that key tests of the outcome of the assignment would be: • extent to which the CDP is “internalised” into the on-going operational practices

of the cities and Vodocanals and is sustainable after the consultants finish their assignment

• extent to which the financial and operational performance are improved • adoption of tariff policy that is based on full cost recovery • fulfilment of the operational and financial covenants and reporting requirements

under the EBRD loan agreement • definition of a clear corporate strategy for the future. In terms of these tests, the outcome would be considered as largely successful. The definition of a clear corporate strategy and the “internalisation” of the CDP were doubtless aided by the fact that SV prepared the CDP itself based on a draft provided by the company. The second two bullet points were also satisfactorily achieved, as discussed further in the transition impact section below. SV has complied with the loan covenants and has been reasonably up to date on reporting requirements, with some minor delays. By the time the parties had settled the disputes over Phase I and moved on to Phase II, Vorkuta had dropped out of the project. It was agreed that Phase II would consist of training, in Canada, for seven key staff from the city and Vodokanal of Syktyvkar. The training included classroom sessions, contact with water experts and site visits. The consultant was very satisfied with the quality of this training. However, while the amended Phase II assignment was fully achieved, the original concept for Phase II was not addressed. Although detailed TOR for Phase II were not presented in the original contract, in the view of the evaluation team, a training session in Canada does not match the general description of “implementation of the CDP”. The original objectives of Phase II are considered to be Not Achieved. 3.2.2. TC2: Project implementation unit

14

Box 3: Objective of TC2 • ensure the proper and effective set up of the PIU, including PIU staffing and

essential training of selected experts • advise the companies and the Bank on any improvements to the structure and the

TOR for the PIU, where deemed necessary • establish proper business processes within PIU in order to implement the project

effectively and in a timely manner • set up a mechanism for the development and implementation of an EAP for the

project. The objectives under this assignment were Fully Achieved. The contract was signed on 1 November 2004, and the consultant started work on 1 December. By 30 December, the consultant had developed the PIU manual and the PIU was formally established by the general director of SV approving the document specifying the duties of the PIU members and the charter of the department. The consultant’s first progress report, dated 31 January 2005: • listed outstanding conditions precedent to be achieved before loan disbursement • analysed the weaknesses of the CDP and the possibility of consequent delays • commented on possible problems with the engineer’s contract • highlighted the need of the PIU for more IT infrastructure. The PIU consultant provided formal training and shared its practical experience with the PIU in the course of its everyday activities. It covered topics including corporate and business planning, environmental issues and financial planning. The consultant’s final report noted that the PIU still lacked experience in procurement and supervision of the project as the actual procurement process, which had been intended to coincide in part with the work of the PIU consultant, instead had only just started at the very end of this assignment. During the period of the assignment, the PIU decided to complete the CDP and service agreement with the city administration itself, as described above. The PIU consultant assisted in coordinating this and putting together the final report and provided training to staff on business planning. The consultant provided detailed comments on the draft CDP. In this way it contributed to remedying some of the failings of the CDP assignment (TC1). On request of SV, the PIU consultant extended its involvement in order to assist the PIU in producing a Russian/English financial model. For this purpose, the term (but not the cost) of the contract was extended, and the final visit took place in November 2005. In the environmental field, SV management committed the company to improve its environmental management system (EMS) sufficiently to achieve International Organization for Standardization (ISO) 14001. In this context, the PIU consultant also assisted the PIU in updating the EAP from the feasibility study and developed an initial environmental review that would form the basis of an EMS. The EMS was implemented and received ISO 14001 accreditation in 2007.

15

Besides reporting on the work programme undertaken, the consultant’s final report made a number of constructive criticisms of the project. These are addressed in more detail in the relevant sections of this report. It identified which components were already unlikely to be completed within the original time-frame and proposed improvements to procedures. For example, it recommended better communication between the EBRD, the project manager and the PIU. In the opinion of the evaluation team, this consultant’s comments were highly perspicacious. The final report identified most of the problems that would delay the project. 3.2.3. TC3: Creditworthiness enhancement programme Box 4: Objective of TC3 Enhance the cities and the Republic's creditworthiness by improving all aspects of their financial management and planning: In order that the cities and the Republic be able to ensure that they can meet necessary current and future expenditures (including those related to social protection measures) and honour guarantee/debt-service obligations. The objective of this assignment was Fully Achieved. The contract was signed in December 2004 and implementation commenced in January 2005. It was completed to schedule by year-end 2005. The relationship between the consultant and client seems to have been good. The results of initial research by the consultant showed that the condition of public finances in the Republic and in the city of Syktyvkar was good and characterised by an active involvement in the reform process. Therefore, the focus of the assignment was shifted from identification of problems and weaknesses to the development of effective proposals within the tasks of the consolidated action plan (CAP) under Phase I of the Republic’s comprehensive regional finance reform programme. The consultants focused on providing support for the implementation of the CAP and providing training through procurement seminars and a study tour. As outputs of the assignment, the consultant assisted in the production of: • budget scenarios • draft regulatory documents relating to maintenance of the register of expenditure

obligations • several measures proposed to improve medium-term financial planning in the

Republic • recommendations for the development and implementation of investment

programmes • the introduction of performance budgeting principles, analysis and

recommendations relating to tariffs and collections • draft procurement code and guidance for the Republic. All these outputs were discussed and agreed with the Republic, city and Vodokanal.

16

In terms of resulting improvements to the financial performance of the Republic, the following key points are noted. • The credit rating of the Republic has risen from B2 (Moody’s) and CCC (Fitch) in

2001 to Ba2 (Moody’s) and BB- (Fitch) in 2008. • The financial performance of the Republic remains adequate following a sharp

drop in revenues between 2006 and 2007 due to regulatory changes. • The Republic’s current surplus stood at 8.4 per cent of current revenues at the end

of 2007. Despite heavy capital expenditures, it had managed to balance its budget up to the end of 2007 (the date of the latest audited figures available).

• The Republic’s cash balance continued to increase and exceeded the total debt stock at the end of 2007.

• In June 2006 the Republic was able to issue a 1 billion roubles denominated 10-year bond with effective yield to maturity of 8.1 per cent.

3.2.4. TC4: Project management Box 5: Objective of TC4 Facilitate the timely and effective implementation of the project by: • rendering assistance to the company in the implementation of the project,

including design, all aspects of procurement, contract administration and disbursement

• providing timely recommendations and reporting to the company in the contract administration process

• carrying out supervision of the works and administration of supply of goods contracts including all aspects that, in the consultant’s professional judgment, may jeopardise the project’s successful implementation and/or constitute violation of the contract conditions

• if requested, helping the company to provide the Bank with data, advice and information in relation to contract administration and supervision of works processes, which shall be impartial and take due regard of the best industry practices.

The objective of this assignment was Partly Achieved. The client, supported by the PIU consultant, has commented on the slow start to the assignment which “practically resulted in losing the first construction season for the critical-path components”.1 Once in post, the consultant took longer than planned to produce the procurement plan rather. The first two components to be procured were “small supply and training components” which suffered “inexplicable delays in relation to the undemanding task”. The PIU admits that it was slow in producing some of the necessary inputs, especially at the early stages when it was not yet familiar with what was required. The PIU consultant also made recommendations on improved communications between the client and the project manager.

1 Final Report of the PIU Consultant, section 2.2.1.

17

By late 2008, around 70 per cent of the original procurement plan had been achieved with a total delay of around 31 months over the original schedule. Of this, around 10 months was attributable to later-than-expected signature of the loan agreement, with a further four months before the contract with the project manager was signed. The remainder of the delay was picked up during the implementation of the project. In the experience of the evaluation team, procurement plans for large projects of this type tend to be over-ambitious in terms of timing and an implementation delay of around 17 months is not unusual. Any problems with project manager do not appear to have been regarded as being as serious as those related to the CDP consultant because there is no sign of invoices being refused. There is no evidence that the expertise or quality of the project manager’s input was criticised, only the timing and communication with the client. Following the PIU consultant’s criticisms, improvements were made to the communications and working practices. A revised scheduled was agreed with the project manager in 2007 and slippage was reduced after that point. Of more concern is the budget overrun. As of late 2008, the project was expected to be completed in the spring of 2009. However, this was contingent on the client obtaining a requested extension of €5 million to the EBRD loan and € 2.2 million to the NDEP grant, a 50 per cent increase on the original budget. If the additional finance were not available, some elements originally included in the project could not be funded. These were expected to include the refurbishment of a number of water and wastewater pumping stations and the renovation of certain water treatment facilities. It was noted in the December 2008 monitoring report that such cancellations “will not affect the investments made to date or the objectives of the project”. In practice, events were overtaken by the effects of the worldwide financial crisis, and the client asked to cancel the unutilised portion of the loan rather than extend it. At time of writing it is not clear if the city or the company intends to finance these elements itself or postpone them indefinitely. Project monitoring reports do not show any cost overruns other than the increase to the TC budget for the project manager. However, this is because “capital costs, equipment” is shown as a single item and the project has not yet disbursed fully. A more detailed breakdown would undoubtedly show significant overspending on individual components. These are attributed by various parties to two causes: (1) an inadequate consultancy budget which did not allow for the consultant

preparing detailed project specifications (2) implementation delays then compounded the problem as inflation raised costs

further. The purpose of this section is not to review the achievement of objectives of the investment operation but of the TC which had as its objective “the timely and effective implementation of the Project”. This objective was only partially achieved.

18

Although the consultant performed technically to a high standard and largely achieved the secondary objectives with some delay, it is clear the project was likely to remain incomplete without significant new funds. 4. Overall assessment The Bank’s TC ratings overall are rated Successful. This is particularly justified by the Good transition impact rating as described below. The Bank’s additionality is Verified in All Respects as at the time of loan approval there were very limited prospects for a Russian regional municipality to obtain long-term financing. Fulfilment of TC objectives is rated Satisfactory as described above. Project and client financial performance and environmental impact are not rated for TC operations. Bank handling is rated Good. The project is in compliance with the relevant sector policy and country strategy documents. 5. Transition impact and additionality Transition impact is rated Good overall. This reflects in particular the evaluation team’s impression that the client has genuinely taken on board the skills learned from the institution-building TCs. The TC components were imposed by the EBRD as part of the package. Nevertheless, the client apparently took a decision to make the most of the training and support offered and is now actively promoting similar practices in other sectors of the municipality and even in other cities. In addition, the effects on frameworks for markets are also important, through the introduction of the service contract including a new tariff mechanism. As is not uncommon, the TC assignments were intended to assist in the achievement of many of the transition impact objectives of the project as a whole (that is, the investment project). Table 3 below shows the transition impact objectives and monitoring benchmarks of the investment operation as presented in the Board approval document. The third column indicates which objectives were addressed through the TC and, where applicable, the level of achievement. Table 3: Transition impact objectives of the investment project Transition impact objectives of project

Monitoring benchmarks Achievement of objectives

Financial and operational restructuring of the companies including payables

Continuous control over increases in payables and enhanced operational performance of both companies and Teplovodocanal in Vorkuta in particular

Achieved for Syktyvkar (TC1, TC2) Vorkuta project was cancelled

Adjustment of tariff levels Achieved (TC1, TC2) Introduction of the tariff policy Achieved (TC1, TC2)

Adjustment of tariff levels and overall support to the tariff reform Installation of meters Not TC-related Increase in revenue collection rates

Increase of overall collection ratios and of cash collection ratios by the companies

Achieved (TC 3)

Operational efficiency through the investment and corporate development

Decrease in electricity consumption per unit water and wastewater and modest but continuous reduction in staff levels

Electricity consumption not TC-related Improved staff productivity

19

programmes achieved (TC1, TC2) Commercialisation of the companies through international consultant planning process and on the job training

Implementation of the corporate development support programme (CDSP), including corporate development plan, IAS and management information system (MIS) training, a service agreement and selected outsourcing

Achieved (TC1, TC2)

5.1 Company impact 5.1.1 Skills transfer The TC assignments have involved significant training and coaching of management and staff at SV, the city management and the Republic. Apart from classroom training and on-the-job coaching in Komi, there were study trips to Finland and Canada for representatives of SV, Syktyvkar municipality and the Komi Republic administration. The staff in receipt of training through the TC assignments were already employees of the relevant body and for the most part have remained there, eventually moving on to other projects. For example, the PIU was drawn from existing staff of SV and has returned to SV once they had completed their work in the PIU. While TC1 (the CDP assignment) did not wholly achieve its objectives, the remedial activities of the PIU to produce a final version of the CDP and service agreement with support from the PIU consultant (TC2) and colleagues from SPVK, led to a much greater potential for skills transfer. The involvement of all the departments of SV in contributing to the updated document also spread an awareness of the process much wider than the PIU itself. 5.1.2 New standards for business conduct Greater commercialisation of SV has been achieved through the development and implementation of the CDP. The original CDP was for the years 2005 and 2006. For the later years, new versions have been produced and implemented by SV without the assistance of consultants, based on the skills and experience gained during the original exercise. Financial sustainability has been achieved through the new tariff structure which has achieved full cost recovery. Water tariffs have increased between 12.1 and 32.9 per cent per year and sewage tariffs between 17 per cent and 34.1 per cent per year since 2002. SV has been in profit since 2003. The CEP also had an impact on the financial performance of the Republic and city of Syktyvkar, which is summarised in Section 3.2.3 (achievement of objectives) above. As described further in Section 5.4 below, new standards have been achieved in the area of environmental management. This process was put into motion with the assistance of the PIU consultant and carried through to fruition by the company itself.

20

5.2 Impact on the industry and on the economy as a whole 5.2.1 Frameworks for markets The service agreement, which was signed in May 2006 with three years’ duration, has placed the company at arms length from the management of the city and formalised all aspects of the relationship between the city of Syktyvkar and SV. It sets out: • principal tasks, functions, rights and responsibilities of each party • operational modes of the network and facilities • standards of service, maintenance requirements, tariff policy, reporting

requirements, contracts with consumers • procedures for dealing with emergency situations. The service agreement requires Vodokanal to develop an operation and management improvement programme that implements many of the objectives set out in the CDP such as: • a modern financial management system • reorganisation of SV’s organisational structure and staffing • training and development of staff • efficient procedures for invoicing and collection of revenues • MIS • reductions in leaks • energy saving measures. It provides a model for similar service agreements with other municipal companies. Although the implementation of the tariff policy has enabled SV to achieve cost recovery, the implementation of tariffs based on water usage is delayed as it has not yet been possible to install water meters in all properties. This aspect of the investment operation has fallen behind schedule. The company faces particular problems enforcing installation of meters in residential blocks that are partly or completely in private ownership. However, the relationship between the city and company and its subsidised consumers has been rationalised. Previously, some consumers paid a lower rate for water and wastewater, and SV was supposed to be compensated from the city budget. This process tended to be delayed causing SV to lose out. By the new procedure, all consumers pay the full tariff but privileged groups receive a payment from the city. Furthermore, in the case that the city sets the tariff too low for cost recovery, the city is obliged to provide a subsidy to the company to make up the gap. 5.2.2 Skills transfers Some of the staff receiving training through the project have moved on to other municipal companies. The current deputy mayor of Syktyvkar was formerly general director of SV and still oversees the project. The management of the city and

21

Republic have ensured that the new standards and skills learned from the water project have been applied not only to other activities of SV but also to the activities of other municipal companies, such as the district heating company. In 2008 the manager of the PIU was seconded to Kazan for a few weeks, at the request of the municipal water company, to assist in establishing a PIU to manage a new EBRD-financed water project in that city. 5.2.3 Demonstration effects It has been noted that the project was inspired by the Bank’s work in St. Petersburg and that SV sought assistance from staff involved in that water project when the CDP consultant did not deliver as hoped for. This of course reinforces the demonstration effects of the St. Petersburg project. It appears that the Komi project may have had a similar effect on others. In the summer of 2008 the head of the PIU at SV spent some time in Kazan where she assisted the municipal water company in establishing its own PIU to handle a planned EBRD-financed water project there. 5.3 Environmental impact As this is a TC operation, it is not rated for environmental impact. However, it should be noted that the project resulted in the establishment of an environmental department at SV for the first time. The PIU consultants updated the EAP and assisted the PIU in establishing procedures to implement the plan. This was an important part of the project since upgrading the water and wastewater infrastructure of the city to EU standards was one of the principal objectives of the investment operation. SV has implemented the international standard ISO 14001 for environmental management, and this was certified in 2007. 5.4 Country strategy and sector policies The 2000 Strategy for the Russian Federation (BDS/RF/00-1(F)) was applicable at the time of the project’s conception. It set the following Bank transition objectives for the water, sewerage and waste treatment sub-sectors: • commercialisation and corporatisation • economic pricing and regulatory framework conducive to private sector

involvement • improve operating efficiencies • reduce public health hazards. The types of investment envisaged were investments in system reconfiguration metering, replacement of energy inefficient pumping and treatment plant and the establishment of billing and management information systems as well as environmental protection equipment.

22

Section 3 of the Municipal and Environmental Infrastructure Operations Policy of 1998 (BDS98-075) stated that the Bank pursued the following main objectives in the municipal and environmental infrastructure (MEI) sector: • decentralisation of municipal and environmental infrastructure provision • commercialisation and corporatisation of service provision • promotion and optimisation of private sector involvement • development of regulatory structures • environmental protection and energy efficiency. Both the investment operation and the TCs were fully in conformity with these objectives. The investment operation had “environmental protection” as its major focus. The TCs provided support for the implementation of the EAP (TC2). TCs 1 and 3 focused on: • “commercialisation and corporatisation” • “economic pricing” (through a cost-based tariff system) • “improving operating efficiencies” • “decentralisation of the municipal and environmental infrastructure provision”

(through the service agreement). 5.5 Additionality The additionality of the operation is Verified in All Respects as no commercial alternative was available to supply such long-term financing of municipal infrastructure in secondary regions of Russia. The average maturity of sub-sovereign bonds reviewed at the time of Board approval was only four years, inadequate for an investment project of this type. The additionality of the investment operation was further reinforced in the Board document through reference to the Bank’s added value in terms of institutional development, skill transfer and tariff reforms. These aspects, for the most part, were to be achieved through the TC. 6. Bank handling Bank handling is rated Good overall. It is felt that some of the delays to the project are attributable to an error in setting the budget for TC4 too low. Nevertheless, in other aspects the Bank performed well, particularly in: • managing the contracts • maintaining contacts with the client and consultants • monitoring the operation • intervening where necessary to resolve problems. The evaluation team notes that this operation, like many TC operations previously evaluated, has had several OLs (at least five, including two during the year when the project was under evaluation). In this case, there do not appear to have been major

23

continuity problems, as can sometimes happen. This may be because one member of the team, now a senior banker, has been constant throughout and is available to support the handover to a new OL. 6.1 Consultant procurement and contracts administration The administrative aspects of the Bank’s handling of the project were on the whole well managed. The project management contract (TC4) was a challenge from the start as it was the first NDEP-funded TC the Bank had handled. A lot of documents and procedures had to be developed from scratch. Matters were further complicated by the decision, part-way through, to drop the Vorkuta project and concentrate only on Syktyvkar. With guidance from the NDEP secretariat, the CSU is to be congratulated on their efforts in taking the assignment through to consultant selection and contract signing without unreasonable delay. Such delays as occurred had more to do with the definition of the project as a whole and uncertainties over Vorkuta as to the challenges of dealing with a new funding source. 6.2 Project preparation and definition The process of TC Com scrutiny of proposals had a clear result in the case of the PIU consultant, where two assignments were reduced to one with half the budget. The move away from performing many of the tasks of the PIU to providing training and guidance to local staff enhanced transition impact as well as saving money. In the case of TC4, an inadequate budget led to a reduction in the choice of consultants who would bid for the contract and also caused time and cost overruns in the investment project. At the time, the project was already falling about a year behind schedule as a result of delays to preparation of the CDP and in obtaining necessary guarantees and sign-offs by authorities in the Republic and the federal administration. There was pressure to conclude the consultant selection and contracting quickly to avoid further delay. Bank staff did react and give consideration to the comments of the PIU consultant on the budget. However, the decision was taken to go ahead with negotiations with the shortlisted consultants rather than go right back to TC Com stage with a new proposal. This was despite the warning signs from other shortlisted firms withdrawing from the competition over the same issue. The EBRD had also responded to a query from one bidder who was unclear about the demarcation between the PIU and the project manager, querying who would be responsible for the detailed design. Although a direct line can be traced from the inadequate budget to overruns in the project, it is also true that the overruns are not so much different from those seen in many large procurement projects of the type. Taking the project back to the TC Com would certainly have led to delays, which might still have led to cost overruns from inflation so that the outcome was not improved.

24

6.3 Relations with the client and consultants The Bank’s relations with the client appear to have been generally good. There were occasions when the Bank’s intervention was required to iron out differences between the client and consultants. In the case of the CDP assignment (TC1), the Bank had to intervene when SV complained that the consultant’s performance was not up to scratch and refused to authorise the full payment of invoices. With the assistance of the Canadian office, a negotiated settlement was reached that SV would authorise payment of €450,000 in total (of the €600,000 contract ceiling) on the condition that the consultants made up the deficiencies in a Phase II contract. The Bank also intervened in the project management assignment (TC4) to help the parties reach agreement on a revised schedule and improve communication according to the advice of the PIU consultant. In such cases, the Bank intervened when a problem became apparent, kept clear records of such interventions (through file notes where necessary) and was successful in facilitating a satisfactory settlement. 7. Key issues and lessons learned 7.1. Use existing networks to promote the Bank’s work A key theme running through this project has been the effects of informal contacts among water sector professionals in Russia. The project had its genesis in such contact between Komi and St. Petersburg, with the demonstration effect passed between colleagues at a senior technical level in the water companies of the two cities. This contact was again tapped when the client was not happy with the performance of the CDP consultant and relied on the guidance not only of the PIU consultant but also of colleagues from St. Petersburg to assist in the completion of the CDP and the service agreement. Finally, similar contacts doubtless came into play when the head of the Syktyvkar PIU was seconded to Kazan to assist in establishment of a PIU there. They may even have had a similar role in instigating the Kazan project as they did the Komi project. Such informal contacts and networks are not unexpected in a specialist field requiring high-level technical know-how and experience in some of its most senior staff. It is even likely that many professionals in this field in Russia have known each other since university days. The EBRD should seek to make the maximum use of such networks to enhance the demonstration effects of its projects and to identify new business. Lesson: Tap into existing professional networks to market the Bank and enhance demonstration effects. Business networks exist in many areas and may be particularly strong in sectors requiring a high level of technical or engineering expertise, such as water and wastewater management or the power or telecoms

25

sectors. Tapping into such networks makes the Bank’s job of marketing its business much easier. It is through such networks that demonstration effects are spread through to the decision-makers in a particular sector. The Bank should ensure that it involves its satisfied customers in its marketing efforts. 7.2. Institutional development Close scrutiny by the TC Com led to the reformulation of TC2 in a way that both saved donors’ money and enhanced transition impact. The original proposal envisaged the consultant providing staff and performing the tasks of the PIU. The TC Com halved the budget by reducing the consultant’s role to an advisory one. This meant that local staff acquired hands-on experience of running the project rather than classroom training. In this way, much greater skills transfer and institutional development were achieved than would otherwise have been the case. Lesson: Limiting consultants to an advisory role enhances institutional development. The consultant in this case was not contracted to run the PIU but instead to serve in an advisory role. The PIU was staffed by existing employees of the water company, who managed the project with the support and advice of the consultant. As well as saving money on expensive consultancy fees, this approach enhanced the institutional development aspects of the project as the local staff obtained hands-on experience in project management. 7.3. Targeting institutional development TCs at the correct level for the client One of the objections which SV expressed with regard to the CDP assignment (TC1) was that the consultant initially assumed very limited knowledge and competence on the part of SV about many aspects of the business. Time was wasted while SV convinced the consultant that it only needed to provide support in certain specific aspects of development. By contrast, in the case of the CEP assignment (TC3), the consultant began its work by performing some initial research into the financial situation of the client and identifying areas that required its expertise. When this revealed that the client was reform-minded and generally competent in most of the areas covered by the original TOR, the focus of the assignment was shifted from remedial work to a more forward-looking programme. The consultant was, therefore, able to provide focused training and support the implementation of the consolidated action plan that had already been developed by the Republic. Lesson: Consultants should not make assumptions about the level of knowledge already existing in the client company. Consultants engaged in institutional development programmes may waste valuable time and resources if they do not first assess the business practices already followed by the client at the start of the assignment. Such an assessment may indicate that the client is competent in modern business practices and has already instigated many management reforms.

26

In this case, the focus of the assignment can be changed from remedial work to a more forward-looking programme better suited to the existing level of expertise of the client. Pushing ahead with basic training in such a situation not only wastes valuable TC resources but is likely to alienate the client and its staff. 7.4. Adequacy of TC budget and consultant selection It is clear in this case that the budget for TC4 was set low, which has had knock-on effects on the schedule and, particularly, the cost of implementation of the investment operation. There were warning signs early on in the withdrawal of several shortlisted consultants. But also a question from one bidder about the proportion of design work to be performed under the assignment and, finally, the blunt advice of the PIU consultant in its inception report were indications of inadequate projections. The project team had to consider whether it was worth delaying the project and returning to the TC Com for additional funds. By this time the loan agreement had been signed and there was pressure to get the project manager contracted as soon as possible. The consultant for this assignment was selected by the EBRD and was not the client’s preferred candidate. The city and the utility preferred the second-placed Russian candidate and ultimately concluded the contract with this company after talks had broken down with the first choice. The choice of consultant had been delegated to the EBRD because of the distances and logistics involved. Although this approach seems to have worked well in the case of the PIU consultant, it was less successful in this case. Closer coordination with the client would probably have avoided the delays in the negotiation process, as the client’s first choice of candidate would have been selected. It might also have resulted in a budget that was better founded in the specific requirements of local procurement rules. It has been noted in a number of TC evaluations in the past that it is important to coordinate closely with the client and ensure their involvement with the budget process and consultant selection. The client is closer to the project than the EBRD’s OL can be. With adequate advice from consultants or EBRD experts, who bring experience of Western procurement standards, the client should be best placed to prepare an accurate budget and select a suitable consultant. Lesson: The client should always play a leading role in budget preparation and consultant selection. The client knows its own project and local conditions better than the EBRD. Involving the client closely in the budget preparation and consultant selection is the best way to ensure that the assignment is appropriate to the client’s needs. The EBRD should support the client with guidance, either directly through its own experts or through consultants with expertise in Western procurement practices. In the project under consideration, consultant selection was delegated to the EBRD for logistical reasons. As the Bank moves further east, logistics will become increasingly

27

challenging. This should not be allowed to create a barrier to the client’s close involvement at all stages of project preparation. 7.5. Staffing of the PIU In this operation, unlike in some of the Bank’s other operations, the staff of the PIU were drawn from existing staff of the client and returned there once their services were no longer required. This ensured that the skills acquired through training and experience of the project were spread through the company. The management of the city and the Republic have also ensured that practices developed through the CEP and the PIU support assignments have been applied to other projects of SV and at other municipal companies. In some other Bank operations, the PIU has been staffed by local consultants. These are more highly paid than the company’s regular employees and unlikely to remain in the area with their skills once the PIU has been disbanded. This might develop skills within a small number of professional consultants, but it does nothing to ensure the adoption of new practices by the client. Lesson: Staff the PIU with employees of the company, not with outside specialists. Contracting specialists to staff the PIU for the duration of the project does not assure skills transfer to the company as a whole, nor to other municipal companies in the area. Such specialists are likely to seek out further specific projects elsewhere. Creating a specialist and mobile group of highly skilled consultants in this way inhibits the uptake of new practices in the regular management of the industry. Similarly, existing staff seconded at a much higher salary than normal are unlikely to return to their regular positions and pass on their new skills to colleagues once the project is complete. A better result is obtained if clients allocate internal staff to the project for a limited period and then return them to positions within the company or at other municipal companies in the city.

28

OPERATION PERFORMANCE RATINGS

KOMI MUNICIPAL WATER SERVICES DEVELOPMENT

Performance Indicator

Rating OVERALL TRANSITION IMPACT (Analysis in Appendix 3) (Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Negative) Good

ENVIRONMENTAL PERFORMANCE OF THE PROJECT AND SPONSOR (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) Not rated for TC operations

NA

EXTENT OF ENVIRONMENTAL CHANGE (Ratings: Outstanding, Substantial, Some, None/Negative) Not rated for TC operations

NA

ADDITIONALITY (Ratings: Verified in all respects, Verified at large, Verified only in part, Not verified) No commercial alternative was available at the time for long-term financing of municipal infrastructure in secondary regions of Russia.

Verified in All Respects

PROJECT FINANCIAL PERFORMANCE (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) Not rated for TC operations

NA

COMPANY FINANCIAL PERFORMANCE (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) Not rated for TC operations

NA

FULFILMENT OF PROJECT OBJECTIVES (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) This is a composite rating: TCs 2 and 3 had Excellent and Good fulfilment of objectives, respectively, while, TCs 1 and 4 were less successful in meeting their objectives. The transition-related outcomes which the TCs were intended to achieve were largely met through the determination of the client to make up deficiencies on the part of two of the Consultants. This is reflected in the Good rating for transition impact.

Satisfactory

BANK HANDLING (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) Although some of the delays and overruns of the project may be attributable to setting the budget for TC4 too low, in other aspects the Bank performed well.

Good

BANK’s INVESTMENT PERFORMANCE (Ratings: Excellent, Good, Satisfactory, Marginal, Unsatisfactory, Highly Unsatisfactory) Not rated for TC operations

NA

29

OVERALL PERFORMANCE (Ratings: Highly Successful, Successful, Partly Successful, Unsuccessful) While the investment operation is delayed and overspent, the related TC has for the most part achieved its aims and had a good transition impact. The companies, cities and Republic seem to have fully taken on board the corporate development aspects and is actively promoting these in other areas.

Successful

30

31

TRANSITION IMPACT ANALYSIS (Komi Municipal Water Services Development)

TI checklis

t categori

es

Steps of Rating Transition Impact Ex Post

Short-term

verified impact

Longer- term

transition impact

potential

Risk to potential

TI

Step I: Change by the project at corporate level

Ratingii Ratingiii Ratingiv

3

Private ownership

NA NA NA

5

Skill transfers

Excellent Excellent Low

6

Demonstration effects

Good Good Low

7

New standards for business conduct

Good Good Low

Step II: Transition impact at the level

of the industry and the economy as a whole

Rating Rating Rating

1

Competition

NA NA NA

2

Market expansion

NA NA NA

3

Private ownership

NA NA NA

4

Frameworks for markets

Good Good Low

5

Skills transfers

Excellent Excellent Low

6

Demonstration effects

Good Good Medium

7

New standards for business conduct

Good Good Low

Summary of verified, potential and risk ratings

Good Good Low

Overall transition impact rating: v

Good

32