ASIAN DEVELOPMENT BANK · 2014-09-29 · asian development bank pcr:ino 26314 project completion report on the rural water supply and sanitation sector project (loan 1352-ino) in

ASIAN DEVELOPMENT BANK PCR:INO 26314

PROJECT COMPLETION REPORT

ON THE

RURAL WATER SUPPLY AND SANITATION SECTOR PROJECT (Loan 1352-INO)

IN

INDONESIA

October 2002

CURRENCY EQUIVALENTS

Currency Unit – Rupiah (Rp)

At Appraisal At Project Completion (November 1994) (August 2000)

Rp1.00 = $0.000455 $0.00012056 $1.00 = Rp2,193 Rp8,295

ABBREVIATIONS ADB – Asian Development Bank BAPPENAS – Badan Perencanaan Pembangunan Nasional

(National Development Planning Agency) CPAO – Central Project Administration Office DED – detailed engineering design DGHS – Directorate General of Human Settlements DPP – desa pusat pertumbuhan (growth center village) DPU – Departemen Pekerjaan Umum (Public Works Department) DT – desa tertinggal (lesser developed village) DGRD-MPW – Directorate General of Rural Development-Ministry of Public

Works DGRCD – Directorate General of Rural Community Development DGCDCEH – Directorate General of Communicable Disease Control and

Environmental Health IKK – ibu kota kecamatan (district administrative center) KGS – kelompok pengguna sarana (facility user group) KPKN – Kantor Pembayaran dan Kas Negara (local government

treasury) KPS – kabupaten pengelola sarana (Project Secretariat) lcd – liters / capita / day (liters of water consumed per person per

day) MCK – mandi cuci kakus (public sanitation facilities) MOF – Ministry of Finance O&M – operation and maintenance PBME – project benefit monitoring and evaluation PCR – project completion report PDAM – Perusahaan Daerah Air Minum (regional water supply

enterprise) PPAR – project performance audit report REPELITA – Rencana Pembangunan Lima Tahun (five year development

plan) RRP – report and recommendation of the President SPAR

UPS – –

sub project appraisal report unit pengelola sarana (users management groups)

VAP – village action plan WSS – water supply and sanitation WTP – water treatment plant

GLOSSARY

Bupati Administrative head of a kabupaten Kabupaten – The third level of government in Indonesia, a subdivision of a

province Kecamatan – The fourth level of government in Indonesia, a subdivision of a

kabupaten Desa – The administrative unit representing a village

NOTES

(i) Before 2000, the fiscal year (FY) of the Government ended on 31 March. Since 2000, the fiscal year has ended on 31 December. FY before a calendar year denotes the year in which the fiscal year ends. For example, FY2000 begins on 1 January 2000 and ends on 31 December 2000.

(ii) In this report, "$" refers to US dollars.

CONTENTS

Page BASIC DATA

I

MAP

vii

I. PROJECT DESCRIPTION

1

II. EVALUATION OF DESIGN AND IMPLEMENTATION 1 A. Relevance of Design and Formulation

B. Project Outputs C. Project Costs D. Disbursements E. Project Schedule F. Implementation Arrangements G. Loan Agreement Conditions and Covenants H. Consultant Recruitment and Procurement I. Performance of Consultants, Contractors, and Suppliers J. Performance of the Borrower and the Executing Agency K. Performance of ADB

125777889

1010

III. EVALUATION OF PERFORMANCE 11 A. Relevance

B. Efficacy in Achievement of Purpose C. Efficiency in Achievement of Outputs and Purpose D. Preliminary Assessment of Sustainability E. Environmental, Sociocultural, and Other Impacts

1111121213

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 13 A. Overall Assessment

B. Lessons Learned C. Recommendations

131315

APPENDIXES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Project Framework Project Outputs Project Budget and Costs Loan Disbursements Implementation Schedule Project Organization Status of Compliance with Loan Covenants Consulting Services Financial and Economic Evaluation Project Performance Rating Project Benefits Lessons Learned

182033343536374344536065

BASIC DATA A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number

Indonesia 1352-INO Rural Water Supply and Sanitation Sector Republic of Indonesia Directorate General of Human Settlements in Ministry of Public Worksa $85.0 million

B. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual – Number of Extensions 6. Closing Date – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan – Interest Rate – Maturity (number of years) – Grace Period (number of years) 8. Terms of Re-lending (if any) – Interest Rate – Maturity (number of years) – Grace Period (number of years) – Second-Step Borrower

25 August 1994 15 September 1994 2 February 1995 6 March 1995 4 June 1995 10 April 1995 0 30 September 2000 30 September 2000 0 Variable 25 years 5 years NA

a Since the 23 August 2002 Cabinet reshuffle, the Ministry of Settlements and Regional Infrastructure.

iii

9. Disbursements a. Dates Initial Disbursement

29 August 1995

Final Disbursement

12 December 2001

Time Interval

76 months

Effective Date

10 April 1995

Original Closing Date

20 September 2000

Time Interval

65 months

b. Amount ($ 000) Category or Sub-loan

Original

Allocation

Last Revised

Allocation

Amt

Cncled

Net Amount

Available

Amount

Disbursed

Undisbursed

Balance

Civil Works Part A 27,200 27,200 0 27,200 19,671.09 7,528.91 Eq & Mats LCB 2,736 2,736 0 2,736 3,427.72 -690.72 Eq & Mats LCB/IS 24,624 24,624 0 24,624 17,152.52 7,471.48 Cons. Services Part A

5,540 5,540 0 5,540 2,721.25 2,818,75

Civil Works Part B 3,950 3,950 0 3,950 4,702.76 -752.76 Eq & Mats Part B 2,650 2,650 0 2,650 2,188.52 461.48 Cons. Services Part C

8,500 8,500 0 8,500 6,889.87 1,610.13

Interest During Construction

9,800 9,800 0 9,800 9,800 0

Impress Account 762.77 -762.77

Total 85,000 85,000 0 85,000 67,315.51 17,684.49

10. Local Costs (Financed) ADB-financed - Amount ($) 25.8 million - Percent of Local Costs 40.7 - Percent of Total Cost 24.6 C. Project Data

1. Project Cost ($ million) Cost Appraisal Estimate Actual Foreign Exchange Cost 38 41.6 Local Currency Cost 104 63.4 Total 142 105.0

iv

2. Financing Plan ($ million) Cost Appraisal Estimate Actual Implementation Costs Borrower-Financed 57.0 37.6 ADB-Financed 75.2 57.6 Other External Financing 0.0 0.0 Total 132.2 95.2 IDC Costs Borrower-Financed 0.0 0.0 ADB-Financed 9.8 9.8 Other External Financing 0.0 0.0 Total 9.8 9.8

ADB = Asian Development Bank, IDC = interest during construction.

v

3. Cost Breakdown by Project Component ($ million) Component Appraisal Estimate Actual A. Water Supply Foreign Local Total Foreign Local Total - Land 4.8 4.8 0.0 0.0 0.0 - Civil Works 7.1 47.3 54.4 9.5 29.7 39.2 - Equipment and Materials 20.2 14.0 34.2 19.8 5.2 25.0 - Feasibility Study and DED 0.0 2.9 2.9 0.8 1.0 1.8 - Construction Supervision 0.0 1.4 1.4 0.0 0.9 0.9 - Incremental O&M 0.0 6.9 6.9 0.0 0.8 0.8Subtotal I 27.3 77.3 104.6 30.1 37.6 67.7 B. Sanitation - Civil Works 0.0 7.9 7.9 0.0 11.6 11.6 - Equipment 0.0 3.4 3.4 0.0 0.8 0.8 - Incremental Cost 0.0 0.7 3.4 0.0 0.7 0.7Subtotal II 0.0 12.0 12.0 0.0 13.1 13.1 C. Institutional Support - Implementation Assistance 0.9 3.1 4.0 0.4 3.0 3.4

- Hygiene and Sanitation Education and Water Quality Surveillance Program

0.0 0.1 0.1 0.0 0.2 0.2

- Community Management Program 0.0 4.1 4.1 1.3 1.4 2.7 - WSS Institutional Development Program 0.0 0.3 0.3 0.0 0.4 0.4 - Project Administration 0.0 7.1 7.1 0.0 7.7 7.7Subtotal III 0.9 14.7 15.6 1.7 12.7 14.4 D. Interest During Construction 9.8 0.0 9.8 9.8 0.0 9.8 Total 38.0 104.0 142.0 41.6 63.4 5.0DED = detailed engineering design, O&M = operation and maintenance, WSS = water supply and sanitation 4. Project Schedule Item Appraisal Estimate Actual Date of Contract with Consultants Jan 1995 20 Sept 1995Completion of Engineering Designs Dec 1999 Civil Works Contract Date of Award Sep 1995 30 August 1995 Completion of Work 31 March 2000 Sept 2000Equipment and Supplies Dates First Procurement Sept 1995 25 Oct 1995 Last Procurement 31 March 2000 24 Feb 2000 Completion of Equipment Installation 30 March 2000 July 2000Start of Operations Completion of Tests and Commissioning 30 Sept 2000 Beginning of Start-Up 30 Sept 2000

vi

5. Project Performance Report Ratings

Ratings Implementation Period

Development Objectives

Implementation Progress

From 10 April 1995 to 30 September 2000 Satisfactory Satisfactory D. Data on Asian Development Bank Missions

Name of Mission

Date

No. of Persons

No. of Person-Days

Specialization of Membersa

Identification Fact finding Pre-appraisal Appraisal November 1994 Inception 15 Dec 1995 2 12 a Project Review 1 25 February 1997 2 14 h Project Review 2 11 June 1997 1 8 h Mid Term Review 4 October 1998 2 12 h Project Review 3 19 November 1999 1 14 h Project Review 4 12 September 2000 2 22 h Project Completion Review 12 March 2002 3 65 h, jb a a-engineer/project officer, h-project implementation officer, j-staff consultant b Laurence Pochard, Project Implementation Officer; Siti Hasanah, Project Officer; Robert Simpson, Staff Consultant

vii

I. PROJECT DESCRIPTION

1. The objectives of the Rural Water Supply and Sanitation Project were to (i) provide safe, adequate and reliable water supply and sanitation (WSS) services to selected low-income rural communities, lesser developed villages (DTs), rural growth centers, growth center villages (DPPs), and kecamatan towns through community-based arrangements; and (ii) support hygiene and sanitation education, water quality surveillance, and community management activities for targeted communities. Project sustainability was to be achieved through the development of community awareness and active community participation in all stages of the project development cycle, including planning, design, implementation, and operation and maintenance (O&M). The Project also incorporated institutional development and strengthening programs for all involved agencies at central, provincial, and local levels. 2. The Project consisted of three parts: A, Water Supply Facilities; B, Sanitation Facilities; and C, Institutional Support and Project Administration. Part A consisted of (i) the construction of new and the rehabilitation of existing water supply systems; and (ii) the provision of consulting services to carry out feasibility studies, detailed design, and construction supervision. Part B comprised construction of public and private latrines linked to hygiene and sanitation programs. Part C consisted of (i) consulting services for implementation assistance, (ii) a hygiene and sanitation education and water quality surveillance program, (iii) training and media campaigns for a community management program, (iv) a WSS institutional development program, and (v) incremental project administration costs. 3. The project area included all 12 provinces on the islands of Sumatra and Kalimantan, and covered 80 kabupatens. For project management reasons, the provinces were grouped into three regions: (i) region I, including D.I. Aceh, North Sumatra, Riau, and West Sumatera; (ii) region II, including Bengkulu, Jambi, Lampung, South Sumatera; and (iii) region III consisting of Central Kalimantan, East Kalimantan, South Kalimantan, and West Kalimantan.

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation

4. The design1 for this project emphasized (i) a predominantly community-based participatory approach, (ii) adaptable selection of appropriate water supply technology, (iii) economic justification of subprojects, (iv) institutional development, and (v) involvement of communities in O&M of facilities. At the time of project preparation, the operational strategy of the Asian Development Bank (ADB) for Indonesia focused on human resource development and poverty reduction. This policy was supported by investments in infrastructure that provided poor communities with basic social services. 5. In assessing the lessons learned from previous projects, the Report and Recommendation of the President (RRP2) noted that in the previous two water supply projects 1 Prepared under TA No.1818-INO (Asian Development Bank. 1992. Technical Assistance to the Republic Indonesia

for Preparing the Rural Water Supply Sector Project. Manila), approved 23 December 1992 for $600,000; and reviewed in April 1994.

2 ADB. 1994. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Republic of Indonesia for the Rural Water Supply and Sanitation Sector Project. Manila.

2

for small rural towns3, the main problems were (i) insufficient community consultation and participation in the planning, design, and implementation of subprojects; (ii) start up delays due to slow recruitment of consultants; (iii) initial construction problems caused by inexperienced local contractors; (iv) poor community acceptance of schemes, which resulted in slow uptake of house connections and inadequate cost recovery; (v) lack of trained staff in water enterprises, resulting in poor organizational performance; and (vi) the use of inappropriate technology. These lessons were considered to have been incorporated fully in the design of the Project. 6. REPELITA VI is the government’s sixth five year plan, and commenced in April 1994. The Government’s WSS target under REPELITA VI was to provide 20,600 villages with safe water through a program incorporating a community-participatory approach to the planning, implementation, operation, and rehabilitation of rural WSS facilities. Target locations were to be villages in water scarce areas, where diarrhea is communities endemic, coastal villages, and transmigration sites. The Project, as designed, was compatible with the Government’s approach. 7. During preparation of the Project, ADB and the Government were involved in a policy dialogue to guide the formulation of WSS sector policies and strategies, including institutional development, improved financial management, extensive social preparation, and community involvement in planning and O&M of rural WSS systems. The essential elements of this policy dialogue were incorporated in the project design, and it was assumed that Government implementing agencies would be capable of and willing to implement the Project according to these broad policy approaches. 8. The approach adopted for the Project incorporated key policy approaches of the Government and ADB and was called the “bottom-up” approach. Essentially this approach was intended to involve the beneficiary communities in all aspects of project implementation from planning through to O&M. As intended, the approach was consistent with expert opinion on how community-based water and sanitation projects could be successfully implemented. A logical framework was not prepared for the Project, and so a framework based on the text of the RRP is included in Appendix 1. 9. The project approach also included a set of technical, financial, and economic criteria to be applied to the selection of communities and the investigation and design of works. For example, design populations for district community center (IKK) schemes were to be for 7,000- 10,000 people, for DPPs, they should have been greater than 2,500 people, and for DTs, the populations were to be between 1,000-2,000. B. Project Outputs

10. The project produced physical works and institutional improvements. The physical works included water supply and sanitation facilities. Appendix 2 shows the numbers of facilities reported as constructed. The data was derived from kabupaten project office monthly reports, and was sourced from the Government’s project completion report4 (PCR). From field work observed during the Project Completion Review Mission (PCRM), which were only a 3 These projects were Loan No.731-INO: IKK Water Supply Sector Project approved for $40.2 million on 17 January

1985, and Loan No. 1069-INO: Second IKK Water Supply Sector Project approved for $39.0 million on 18 December 1990.

4 Government of the Republic of Indonesia, Ministry of Settlement and Regional Development, Directorate General of Regional Development. 2001. Rural Water Supply and Sanitation Sector Project, Project Completion Report. Jakarta.

3

sample, it is clear that the reported numbers of constructed facilities does not accurately represent numbers of facilities actually constructed, particularly in the case of sanitation facilities where the results appear to be significantly overstated (para. 16). Table 1 shows the number of facilities planned and reported as being constructed. Because the PCRM was only able to observe a sample of the “reported” works, it is not possible to quantify those actually constructed as compared to those reported in the Government’s PCR.

Table 1: WATER SUPPLY AND SANITATION FACILITIES PLANNED AND REPORTED

Facility Output Reported Output Planned Difference Water Treatment Plant

141

198

- 57

House Connection 36,918 73,026 - 36,108 Public Hydrant 9,691 10,788 - 1,097 Infiltration Gallery 273 41 + 232 Simple Water Treatment Plant 269 124 + 145 Deep Well 40 40 0 Gravity Pipe System 760 254 + 506 Deep Well With Hand Pump 5,211 4,899 + 312 Hand Dug Well 29,359 30,584 - 1,225 Rain Water Collector 6,125 478 + 5,647 Family Latrine 68,284 75,310 - 7,026 Public Latrine 3,604 852 + 2,752 School Latrine 1,490 2,585 - 1,095 11. The over statement of outputs occurred for a number of reasons, including (i) poor commissioning procedures so that nonoperational works were recorded as completed; (ii) use of the community participation system5 of construction for sanitation facilities whereby facilities were recorded as completed if the materials had been ordered from the supplier; (iii) possible misuse of funds; and (iv) poor contract administration, where the number of facilities reported reflected the number required by the contract rather than facilities actually constructed.

1. Water Supply

12. Water supply systems consisted of piped and nonpiped systems. A range of technologies was used depending on the quality and location of the water source and on the appropriateness of the technology for the target community. Piped water supply systems included sources not requiring treatment such as deep wells and springs, and sources requiring treatment such as rivers, and shallow wells. Treatment was provided for larger systems through standard plants in sizes ranging from 2.5 to 40 liters per second (l/s), and for smaller systems through simple treatment plants based on rapid filters. Distribution systems included house

5 The system was an implementation method intended to involve communities in project implementation. In essence

the Project was to provide materials and training, and the community constructed the works. Problems occurred where project managers did not monitor construction and communities were not motivated to construct the planned works. This approach was consistent with the Government’s policy for DTs to encourage communities to contribute to the capital cost of facilities.

4

connections and public collection points. Nonpiped systems included deep and shallow wells with or without hand pumps, and rainwater collection systems. 13. The location and design of facilities was determined during preparation of village action plans (VAPs) and subproject appraisal reports (SPARs). VAPs were prepared for DTs in lieu of detailed engineering designs (DEDs), and consisted of technical and socioeconomic data, and a layout and description of the most suitable technical option. During appraisal it was proposed that VAPs would be prepared by the local community with assistance of kabupaten officials. However because of delayed implementation of the training programs, most VAPs were prepared by local government officials. DEDs were prepared by kabupaten officials with assistance of consultants for DPP and IKK WSS subprojects. Groups of DT, DPP, and IKK proposals were combined as subprojects and described in SPARs. Local governments, assisted by regional consultants, prepared bills of quantity, cost estimates of civil works and materials, and tender documents. 14. Construction was supervised by the Directorate General of Human Settlement (DGHS) through the Dinas Pekerjaan Umum, Public Works Department, and kabupaten, and was assisted by regional and project secretariat (KPS) consultants. Regional consultants, engaged at the central level, were responsible for supervision of construction quality between 1995 and 1998. From 1999 this responsibility was transferred to KPS consultants, but did not extend to day-to-day supervision. In an attempt to lower costs and increase community participation, the Project used the joint operation system, which involved use of nonskilled village labor for construction, and which reduced the quality of constructed works. In some cases where costs estimates were inadequate, contractors built only as many facilities as funds allowed. This has also contributed to over reporting, as reported numbers probably reflect numbers of facilities contracted rather than completed.

2. Sanitation

15. Sanitation facilities consisted of family, community, and public latrines typically in schools or close to mosques and markets. Most public latrines were built using the public sanitation facilities model, which is intended to provide, in addition to the latrines, a facility for bathing and washing clothes. Field investigations showed that in many areas, public sanitation facilities were operating and were being maintained. However, in some cases it was clear that both private and community facilities had not been constructed. As a result the reported number of facilities built was overstated. 16. A specific example would be Kabupaten Tabalong, where up to 1440 private latrines were reported to have been installed. However, the PCRM was unable to see any. In Desa Pulau Pinang, Kabupaten Tapin, 3 private latrines only had been built under the Project whereas around 50 should have been built. In spite of the low number of latrines built under the Project, 50% of the households in the Desa now own a private latrine, as the 3 constructed latrines, combined with a promotion campaign from the Desa health officer, had a catalytic effect. The PCRM estimated that only 1-5% of private latrines had been built in desas along rivers. In other desas, an estimate of up to 20% might reflect the level of construction achieved.

3. Institutional and Community Support

17. Planned institutional outputs can be characterized in terms of (i) improved management capacity of the implementing agencies achieved through training and better implementation

5

approaches; (ii) increased capability of communities to plan, implement, operate, and maintain rural water and sanitation schemes; and (iii) improved knowledge of water-, sanitation-, and health-related matters by the communities. These outputs were intended to improve the implementation and the probability of the Project achieving its intended outcomes. The level of use of the physical outputs produced indicates a very low level of achievement of the institutional outputs, even though around 36,000 people benefited from 55 different training programs in project implementation practices, sanitation education and water quality surveillance, community management programs, water supply construction and maintenance, and the preparation of village action plans, etc. Around 2% of the total project cost was spent on training and institutional strengthening programs. 18. The training programs at the village level were generally not synchronized with work implementation, not sufficiently financed, and too short in duration. In some cases, the local labor force was not sufficiently educated to attend the training. Part of the labor that was skilled or became skilled after the training often refused to work under the Project because of the low remuneration offered under the joint operation system. Training material was too often theoretical and complicated for laborers, and insufficient practical training was organized. At the kabupaten level, too many officials had an inappropriate background for attending the training, and too many changed position after the training. Some training programs were carried out by government officials with limited experience and qualification in teaching and training, instead of using the qualified contracted consultants. Table 2 (p. 6) provides the consolidated training included under the Project. C. Project Costs

19. The project cost estimate at appraisal was $142 million, including a foreign exchange component of $38.0 million and local cost component of $104.0 million. An estimated $104.6 million was provided for water supply, $12.0 million for sanitation, and $15.6 million for institutional support activities, with $9.8 million allocated to interest during construction. Due to the rupiah depreciation and the resulting loan savings, the total project costs only amounted to $122.5 million, with expenditures of $82.5 million for water supply, $13.4 million for sanitation, and $16.8 million for institutional support. Actual project financing is shown in Table 3, and more details are included in Appendix 3.

Table 3: Project financing ($ million)

Appraisal Estimate Actual Change Item Foreign

Exchange Local

Currency Total Cost

Foreign Exchange

Local Currency

Total Cost

Foreign Exchange

Local Currency

Total Cost

Borrower- 0.0 57.0 57.0 0.0 37.6 37.6 0.0 (19.4) (19.4) Financed

%change 0 -34.0 -34.0 ADB 38.0 47.0 85.0 41.6 25.8 67.4 3.6 (12.8) (17.6) Financed %Change 9.5 -27.2 -20.0 Total 38.0 104.0 142.0 41.6 63.4 105.0 3.6 (40.6) (37.0) %change 9.5 -39.0 -26.1

6

Table 2: Consolidated Training Program

Description Duration (day)

No. of Participants

DGURD – Ministry of Settlements and Regional Infrastructure (former DGHS – Ministry of Public Works)

Management of Foreign Aid Project Training of Local Labor for Civil Works Dissemination of Guidelines for O&M Training of O&M Operators Project Benefit Monitoring and Evaluation & Project Completion Report RWSS Project Media Campaign Project RWSS

30 12 3 5 3

3

97119,690

2488

265

120

Subtotal 56 21,158 DGRD – Ministry of Home Affairs

Orientation for Bupatis Training of Trainers Evaluation & Follow up Program of RWSS RWSS Project Management for Post Construction Phase Program Evaluation for Sanitation & WS Programs Project Management Training for Kecamatan Teams Community Training in Project Management (preconstruction) Training in the Preparation of Village Action Plan Training in the Application of CBMA for District Coordination TeamCommunity Development Training

5 4 4 5 5

20 40 12 40

150

17660

10060

2002,6156,9761,954

269291

Subtotal 285 12,701 DGCDC and EH – Ministry of Health Department

Training in Health & Sanitation Education & WQS Training of Trainers, Provincial level Training of Trainer, Kabupaten Level Participatory Plan for Provincial Facilitator Training for Instructors of WQS & Environmental Improvement Training in WQS for Skilled People HSE training for Communities

270 5

48 5 5 5

12

3038

67343

300480

1,057

Subtotal 350 2,621

TOTAL 691

36,480

CBMA=cost benefit monitoring analysis, DGCDCEH=Directorate General of Communicable Disease Control and Environmental Health, DGHS=Directorate General of Human Settlements, GDRD=Directorate General of Rural Development, DGURD=Directorate General of Urban and Rural Development, O&M=operation and maintenance, RWSS=rural water supply and sanitation, WQS=water quality standard,WS=water supply, 20. By April 1998, the forecasted surplus loan funds were estimated to be $25.4 millions as the reported outputs of the Project reached 88% of target, while the contracted expenditure was only 52% of target. Therefore, in October 1998, ADB and the Government agreed to use the surplus loan funds to extend the project scope and a revised budget was prepared.

7

D. Disbursements

21. Funds were disbursed efficiently and effectively based on the following procedures: (i) contractors and suppliers submitted invoices to project managers for approval; (ii) project managers submitted invoices to the local government treasury (KPKN) for payment; (iii) the KPKN requested payment by the local representative bank and sent payment evidence to the Ministry of Finance (MOF); (iv) after requesting and receiving funds from Bank of Indonesia, the local representative bank paid the contractor or supplier; (v) Bank of Indonesia reported payments to MOF; (vi) MOF submitted statements of expenditure and requested replenishment of the special account and bridging fund; (vii) ADB verified statements of expenditure against records of contract procurement and replenished the accounts as appropriate. 22. In 1998 Indonesia experienced a sharp depreciation in the rupiah against the United States (US) dollar, which precipitated a monetary crisis and resulted in difficult financing conditions for the Government and a reduced availability of counterpart funds. The project financing plan was adjusted in 1998. Project disbursement details are shown in Appendix 4. 23. The Project had an undisbursed balance of $17.6 million because of the rupiah depreciation and because ADB was unable to accept the financing of many very small contracts for which the Government was unable to provide data and justifications. E. Project Schedule

24. The project schedule as proposed at appraisal is shown in Appendix 5. A number of scheduling issues arose during implementation and these impacted on the quality of implementation. The issues included lack of coordination between agencies, late release of counterpart funds, and delays in recruiting consultants. Although these did not impact on the overall schedule of implementation, or the loan closing date, i.e. the Project was completed on schedule, they tended to cause problems with internal scheduling of activities. This impacted on the overall performance of the Project as many training courses were not carried out in the sequence appropriate or needed to strengthen project implementation. Training in the VAPs came after the VAPs were completed, training on preconstruction activities started once the works were ongoing. For the community programs and users associations, most of the training and establishment of the kabupaten project secretariats and user management groups (UPS), were carried out toward the end of the project, a considerable time after the completion of the majority of construction work. The effect was that many communities were not included in decisions about the facilities to be constructed.6 F. Implementation Arrangements

25. The Directorate General of Rural Development (DGRD, formerly DGHS) of Ministry of Settlements & Regional Development (MSRD) was the Executing Agency of the Project. A central project administration office (CPAO) was established to coordinate activities between central government agencies, the provincial agencies and district levels. DGRD provided guidance to local government concerning the implementation of the WSS schemes. Ministry of Home Affairs (MOHA) through its Directorate General for Rural Community Development (DGRCD) and Directorate General for Regional Development (DGRD-MOHA) was the 6 In IKK Pekansaptu, Kecamatan Luhak, Kabupaten 50 Kota, West Sumatra a spring capping was destroyed by

community members and has never operated. The community was concerned that the spring capping would take all available water and leave none for people who did not have access to IKK facilities. In fact the spring capping was only designed to take 10 l/s from the 40 l/s available.

8

implementing agency for the community management and WSS institutional development programs. DGRCD was responsible for the community preparation, the sustainability of the constructed facilities, and establishment of water and sanitation user groups and user management groups (UPS). Most of the user groups were established toward the end of the Project with assistance from the non-government organization (NGO), Bina Swadaya. The Ministry of Health (MOH) through its Directorate General for Communicable Disease Control and Environmental Health (DGCDCEH) was the Implementing Agency for hygiene and sanitation education and water quality surveillance program. DGCDCEH conducted several training courses on the community participatory approach and distributed kits for water quality surveillance. The project organization is shown in Appendix 6. G. Loan Agreement Conditions and Covenants

26. Most were complied with covenants (Appendix 7). While DGRD submitted quarterly reports, the reports did not adequately reflect the implementation of the health education and community programs because no reporting was provided by DGCDCEH, DGRCD, or DGRD-MOHA. As a result there is no formal project report, for example, DGCDCEH implemented health programs or DGRCD implemented the community management program. Submission of audited reports was slow. Generally audit opinions were qualified. 27. Loan agreement/covenants or preset conditions were adequate to control such a dispersed and extensive project, and monitoring of compliance was insufficient to indicate noncompliance. This was evident for example in terms of compliance with schedule 5 paragraph 17 of the loan agreement, which covers criteria for subprojects. Many water supply schemes were constructed in towns with populations too small to provide adequate demand and it is unclear how the Government justified the selection of these towns. In this case, the loan agreement is unclear as to the process and procedure to be followed. Compliance may, or may not, have occurred, depending on interpretation. H. Consultant Recruitment and Procurement

28. Consulting services were provided for (i) technical assistance for subproject appraisal reports (SPARs) and detailed engineering designs (DEDs); (ii) project management support at the central and kabupaten levels, including production of the Government PCR and the project benefit monitoring and evaluation report; (iii) community development activities; and (iv) training (Appendix 8). The process for engaging consultants was generally satisfactory, with some problems being encountered because of late engagement and late production of designs. Insufficient resources were allocated for construction supervision in all three regions. 29. At the beginning of the Project, provincial governments were responsible for procurement, but in 1999 this was transferred to the kabupaten level. Contract packages for supply generally were only for water treatment plants, electromechanical devices, and pipes for IKK schemes. Goods and equipment for DPP and DT works were included in civil works contracts. 30. International shopping was used for equipment and supplies costing $0.5 million to $50,000, and procurement of value less than $50,000 was done locally. The Project awarded 225 contracts through international shopping, 1,113 contracts through local competitive bidding, and 624 contracts using the “swakelola” method. All procurement of value greater than $50,000 required a no objection letter from ADB. In all, 528 such letters were sought from ADB and 494

9

were given. In some cases the government was required to self-finance some of the contracts due to irregularities in the procurement process. I. Performance of Contractors, Suppliers, and Consultants

31. Because of the number of contracts for suppliers and contractors, and the amount of procurement, there was a wide range in the level of performance. In some cases, contracted works were not completed. In other cases the quality of work was poor. Generally, the quality of water treatment plants constructed was adequate because they were installed by national contractors as standard package plants. The use of standard designs reduced the design work required and simplified construction but created problems in matching operating capabilities of plants to water source characteristics. For other facilities, work by local contractors was often below design specifications or was incomplete. 32. The low quality level was due to several factors, including the scarce supervision of the works, inadequate design quality, insufficient local government funds for each module, and use of the joint operation system, as contractors were required to use unqualified labor for the construction of works. The insufficiency of funds, exacerbated by the rupiah devaluation, produced a lowering of the quality of the works together with a reduction in the capacity of the schemes in order to keep the total costs of the IKK within the fixed budget. This reduction was both in terms of the technical specification required (for instance the use of 6 millimeter (mm) instead of 10 mm steel plates for an IKK water treatment plant (WTP), and in the length of the distribution systems and the number of hydrants. The low quality of the works necessitated the need for more O&M thereby affecting cost recovery and resulting in a major negative impact on the balance sheet of regional water supply enterprises (PDAMs). 33. All the consulting firms were Jakarta-based and were not associated with local partners. This led to serious coordination and supervision problems. Most of the consultants were sufficiently skilled in their tasks. However frequent staff changes, lack of local bases and partners, and the short time spent on sites by consultants made coordination with local governments difficult. Also, these factors did not allow the consultants to become fully familiar with local social and technical needs. 34. In most cases, the consultants recruited for the preparation of the SPARs and the DED did an adequate job in view of the large number of subprojects. However, in several instances, weak synchronization between consultants starting their assignments and works scheduled by the local government led to poor performance in the case of some activities. 35. Over the four years of implementation, less than one supervisory consultant per kabupaten per year was supervising the construction of the works; this is an unacceptable ratio to implement a sector project comprising 67 kabupatens (1 kabupaten may have covered up to 20 desas) scattered through Aceh, Kalimantan, and Sumatra.

10

Table 4: Number of Consultants (excluding training)

Tasks

Person-Months

Assist provincial officials to prepare of SPARs and DEDs and monitor civil works constructions.

3,022

Assist DGRD to program, budget, report and coordinate project activities. Assist training preparation and implementation and on benefit and monitoring programs. Assist in project management.

269

Secretarial technical assistance to CPAO, in administration and finance

110

Technical assistance to kabupaten project secretariat in project management and monitoring progress of various activities in the concerned kabupaten.

2,577

Assist to central project administration office to evaluate the project implementation of the years 1995-1997.

72

Advise kabupaten project secretaries

1,138

Prepare audited financial report

195

CPAO=central project administration office, DED=detailed engineering design, DGRD=Directorate General of Rural Development, SPAR=subproject appraisal report J. Performance of the Borrower and the Executing Agency

36. A number of significant events external to the project had a major impact on government operations, including the financial crisis of 1998 and the decentralization of government services starting in 1999. Despite these, the Project’s broad schedule was maintained. However, the efficiency of outcomes was well below the intended level (paras. 42-44) and this was due in a large part to poor project management practices of the executing and implementing agencies. This is an issue broader than the Project but is important in the context of the Project’s sectoral nature, with an inherent need to control many small subprojects. 37. Agencies involved in the Project have been reorganized, in some case more than once. For this reason it is not directly possible to compare the Project’s organization with current institutional arrangements. However, the issues that caused poor performance in the Project are still relevant, and future projects involving many subprojects over a large area in remote regions will likely have similar management and control issues. Generally the performance of the borrower and executing agency was only partly satisfactory. K. Performance of ADB

38. In terms of banking operations such as loan administration and disbursements, the performance of ADB was adequate. However, in terms of project administration, one of the challenges for future scattered sector projects will be to ensure that structures allowing for sufficient monitoring of outputs and on-going assessment of project outcomes are in place. The nature of the Project as a sector loan required a resourcing level for monitoring that ADB could

11

not achieve on its own. Monitoring could have been outsourced to a technical and financial team to audit between 10-15% of the facilities. Such requirement should be identified at the appraisal stage and when not integrated within the design, will need to be identified by review missions and acted upon.

III. EVALUATION OF PERFORMANCE

A. Relevance

39. The project goal and purpose remains relevant in terms of both the Government’s stated priorities and ADBs development strategy. Increased access to improved water supply and sanitation is required for improved health and social welfare. The rural areas targeted under the Project were disadvantaged in this regard when compared to some other areas of Indonesia. The design addressed these issues but failed to adequately provide an implementation process to achieve the project’s purpose. To some degree this can be attributed to a lack of clear purpose in the institutional and community activities and a failure to convert the rhetoric of “bottom-up planning” to a demand-driven community-based program. It should have been clear that the proposed approach was failing at the time of the midterm review but no changes were made to the design. B. Efficacy in Achievement of Purpose

40. The Government’s PCR suggests that the Project achieved more output than originally intended. From limited field assessment (10 kabupaten visited out of 67) after the completion of the project, this does not appear accurate due to a systematic failure of the project reporting systems. The project benefit monitoring and evaluation (PBME) report7 which evaluated the outcome of the project in terms of level of service provided by facilities, considered that the effective output of the project was around 20% of the reported output, where effective output is output that leads to an adequate level of achievement of intended project outcomes8. In terms of benefits received by communities, the effective output of the Project has been reduced by (i) inappropriate choice of technology, standard designs, or locations that resulted in little or no use; (ii) poor construction reducing facility life to close to zero; and (iii) poor community participation and support resulting in unaffordable O&M or unwillingness to maintain facilities. 41. In the case of swallow wells the design standard used by government agencies is 5 meters (m) deep. However, most wells seen in South Kalimantan, Kabupaten Kapuas, are flooded during the rainy season and dry during the dry season because the water table is around 20-25 m deep. In this case the outputs have been constructed but their effectiveness is nil. 42. Less tangible outputs of the project included institutional development and community strengthening. The effectiveness of these outputs is measured in terms of improved achievement of the project purpose, or conversely a reduction in implementation risk. Clearly, these outputs have not generally produced the intended outcomes. However, field assessment indicated that the results varied across the Project, and in some cases subprojects have been implemented well and beneficiaries have gained. For example, in a desa in Kabupaten Tapin, 7 The project benefit and monitoring evaluation (PBME) report is based on review of 38 DTs, 12 DPPs, and 13 IKKs

surveyed. 8 The sampling methodology used in the PBME report probably overestimates the effective output of the Project as

villages where facilities were known not to have been built were not investigated or included in the report.

12

50% of the households now own a private latrine following the training received by the Ministry of Health. In some cases, communities have themselves extended or repaired facilities that were not operational at the end of the Project, and this in itself suggests that communities are willing and capable to contribute to project activities. C. Efficiency in Achievement of Outputs and Purpose

43. Assessment of the efficiency of the Project requires an assessment of the number of facilities producing intended levels of benefits. The PBME report suggests that this may vary from around 50% for larger piped systems to less than 20% for sanitation facilities (only surveyed sites showing constructed output are included in these percentages). An evaluation of the economic internal rate of return (EIRR) for the Project based on reported outputs indicates that, if implemented as intended, the Project would have achieved an EIRR of around 18%. However when estimated levels of outcome achievement are considered it is clear than the real EIRR for the Project is less than 5%. The financial and economic evaluation of the project is shown in Appendix 9. 44. A sample of IKK piped water supply systems indicates a range of FIRR from – 43% to 18%, with only 30% of the sample having an FIRR greater than zero, and only one exceeding the weighted average cost of capital. Size of the customer base, low tariff levels, and high connection fees are among the factors resulting in financial inefficiency. On the basis of the sample, it is reasonable to conclude that nearly all piped water supply systems constructed by the Project will require external assistance to remain viable unless demand is increased and tariffs are raised. As most of the IKK were handed to PDAMs, the net impact on the operational viability of PDAMs is negative and exacerbates the difficulties experienced by PDAMs after the financial crises and devaluation of the rupiah. In a small number of cases,9 PDAMs have been able to integrate IKK systems into larger distribution networks. 45. In terms of efficiency of implementation process, it appears that neither ADB nor the Government were able to recognize and resolve the Project’s problems. Issues such as lack of coordination between the implementing agencies occurred from the beginning of the Project and continued to cause problems throughout the Project. D. Preliminary Assessment of Sustainability

46. Poor location and choice of technology probably occurred due to a failure of the Project to involve communities in decision making during planning. These issues indicate that many of the subprojects were not based on properly implemented demand surveys and certainly many of the subprojects do not meet basic criteria such as the size of the beneficiary population. The inability of many PDAMs to cover O&M costs stems from a lack of demand, exacerbated by poor pricing of connection and supply services. PDAMs are covering O&M costs through operating and capital repayment subsidies. Many WTPs built by the Project are poorly designed in terms of ease of maintenance and are expensive to operate because of unsuitable of water source quality. Similarly, pumping costs for many systems are high and could have been avoided by better choice of location or intake design.

9 In Kabuapten Kerinci, the PDAM has been able to integrate a number of IKK systems within one pipe network.

The PDAM is now planning to reduce operational costs by replacing pumped sources with a spring source capable of supplying the combined network by gravity.

13

47. The issue of tariffs is constant across all systems built under the Project. Since 1999, the role of local government has been strengthened and as PDAMs come under local government, more appropriate tariffs are likely to be set. However, the role of the Dewan Perwakilan Rakyat Daerah (regional parliament) has also been strengthened and it is proving difficult to overcome a general political reluctance to raise tariffs. While the issue is widely recognized so far, not many PDAMs have been allowed to set appropriate tariffs. E. Environmental, Sociocultural, and Other Impacts

48. Generally, rural water supply and sanitation projects produce little to no impact on the environment. Similarly, the Project has had little impact on regulatory and political issues and has failed to have a significant institutional impact. Compared to the major institutional changes occurring because of increased local autonomy, the impact of the Project has been negligible and has been counteracted by rapid and significant staff changes. Water supply and sanitation projects can have an impact on local communities by providing shared goals and a focus on social organization. Similarly, water supply and sanitation projects can have a positive impact on gender roles in Indonesia because a large proportion of the work of rural women is directed to water collection, and women are generally more concerned about health and hygiene issues. The Project will impact on these things only in proportion to its achievement of planned outcomes.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment

49. The Project is rated as partly successful. Full details of the rating evaluation are shown in Appendix 10 (and benefits are described in Appendix 11). The Project was not implemented in accordance with the clear intentions of the project design. There was no will to implement the Project as intended and for this reason the proposed community-based approach was, to a large degree, subverted. The impact of attitudes and behaviors within institutions acted against implementation of the intended approach and forced the Project to be implemented in the same manner as previous projects. Thus the lessons learned from previous projects remained unheeded and mistakes were repeated. This is partly due to a lack of leadership from senior government officers and ADB. B. Lessons Learned

50. Many of the lessons described in the RRP (Appendix 11) could be repeated in this PCR. In particular, the importance of local knowledge in the design process, the importance of the price of water on demand, and the importance of tariffs to achieve sustainable operations were all mentioned. More generally, most international agencies have changed their approach to rural water supply and sanitation based on a need to better respond to community demand. The project implementation failed to reflect these changes. Details of lessons learned are given in Appendix 12. 51. The fundamental reason for the Project’s lack of success is the need to move away from the supply oriented approach of traditional government engineering agencies toward a more consumer-oriented approach to providing community services. While a significant degree of technical skill and management rigor is still required, the new approach needs to be totally

14

premised on a service-oriented organization culture, and this has not been apparent in the government agencies involved in the Project. 52. It is surprising that such a complex project was implemented with such loose project management structures and processes. Coordination was identified as a problem many times during preparation of this PCR, specifically, between implementing agencies, between levels of government, and between consultants and contractors. Many of the fundamental requirements for a project management system were lacking and certainly there was no overall quality management system. Aspects of project implementation were impacted by the lack of project management systems. The lack of a logical framework meant that the processes linking inputs, activities, outputs, and outcomes were not explicitly recognized and certainly not monitored. Monitoring was generally restricted to input counting as evidenced by failure of reporting systems to indicate problems early in the Project. A clear lesson of this Project is the need to implement a project management system that is based on valid implementation processes and that provides relevant information about achievement of purpose. This is particularly true when the Project is intended to produce many small outputs in widely dispersed remote locations. 53. Associated with the need to adopt adequate project management systems, is the need to recognize risks involved in project implementation and to design and implement robust risk monitoring and response systems. In the case of rural water supply and sanitation systems, a key characteristic of project design is that many subprojects will be undertaken as part of a “design and construct” process. While criteria were stated for subprojects and the Project’s covenants required a process that considered demand, there was no overall risk management approach to ensure that these critical activities were implemented. 54. The issue of demand management is not restricted to communities. PDAMs are more attuned to the demands of their consumers than are the Government’s implementing agencies. The relationship between public works authorities and PDAMs is one of service provider and client. This relationship was generally poorly handled within the Project as PDAMs were forced to take on systems that were mostly unprofitable and for which they had little or no involvement in planning and design. Future projects should see PDAMs and other utilities as a key client group. This lesson has been recognized in the new set of water projects being developed in ADB. 55. Institutional strengthening in the Project was limited to training. As a result, the institutional impact flowed from the agencies to the Project rather than the other way. While institutional strengthening is about building capacity it is also about building capability and the will to use it. Capability includes the attitude and behavioral traits that enable achievement of purpose. At the community level this is characterized by demand for water. At the institutional level, demand for the project should be assessed in terms of a real desire to see the project purpose achieved. This level of institutional demand should be as real a prerequisite for institutional strengthening as demand for water is for water facilities. In the same way, projects can’t create institutional capability if there are no adequate prospects to supply basic skill levels. Alternative institutional approaches are needed when the basic institutional prerequisites cannot be satisfied. 56. Consultants were used extensively throughout the Project, but the coordination of consultant inputs was poor. As a result consultant inputs were probably both ineffective and inefficient. An example is the use of consultants for essentially administrative tasks and the failure to provide sufficient resources for consultants to adequately inspect works. In a similar way, audits did not pick up the major underperformance of the Project. Audits should focus on

15

risk areas such as outcomes and inappropriate use or misuse of project resources, and should be provided with sufficient incentive to report problems. 57. The use of standard designs was intended to improve quality and reduce cost. But many of the standard designs were inappropriate for local conditions. There was a failure to recognize when standard designs were inappropriate, and to act to change specifications. 58. The agencies involved in the Project acted in a way that indicated they firmly believed their views were more important than those of the recipient community. This impacted on the Project in a number of ways. In some cases officials who seemed not to want to spend time or resources on community involvement usurped the community’s role. In other cases community participation was reduced to labor. All of this makes a bottom-up approach difficult to implement. Unless project designs include ways to overcome such attitudes, community-based approaches will be difficult to implement, and will continue to impact negatively on community, gender, and poverty objectives. C. Recommendations

1. Project Related

59. Further Action. A structured program of activities to enhance the value of the Project should be considered. Such a program would work through a series of steps to determine if further action can add value: (i) further monitoring and should focus on determining the actual output of the Project; (ii) writing off outputs that have no potential for rehabilitation; (iii) developing a rehabilitation plan ; and (iv) implementing a rehabilitation plan if appropriate. Before each step, a value analysis should be undertaken to determine whether further cost would be rewarded with an appropriate level of added value. 60. Future Monitoring. The Government and ADB should focus future monitoring on identifying subprojects that could produce sustained benefits. This will require a detailed survey of project outputs and identification of all ineffective outputs. 61. Covenants. Several covenants have not been complied with. ADB, as a minimum, should require Government agencies to identify the true output of the Project in a way that will allow successful subprojects to be monitored and if necessary rehabilitated. 62. Additional Assistance. Some TA will likely be required by to help the Government formulate a program of rehabilitation and improvement of project outputs. 63. Timing of Project Performance Audited Report (PPAR) Preparation. A PPAR should only be undertaken if the government can provide sufficient good quality data on actual facility completion rates. If this is possible, the PPAR could commence around the middle of 2003. 64. Project Audit. The PCRM’s field observations showed that constructed facilities were overstated in the government’s PCR. ADB received allegations about malpractices. In some instances, fraudulent procurement processes were identified. Because the overstatement and allegations may involve potential misuse of funds, the PCRM recommends that a full technical and financial audit of the Project be carried out.

16

2. General

65. For preparing and appraising of rural water supply and sanitation sector projects, the following recommendations are made:

(i) Project design should adequately describe the implementation procedures required to achieve intended project outcomes. In addition to economic appraisal, project designs should be shown to have valid internal logic; be implementable; and have a risk management process that is driven by a monitoring system reporting inputs, outputs, and outcomes.

(ii) Project design for rural water supply and sanitation projects will result in a set of minimum criteria for implementation. These criteria should be included in loan covenants and contract specifications, and should be enforced as minimum acceptable performance levels.

(iii) The assessment of executing and implementing agencies’ capabalities should consider broader issues than the number and skills of staff. The decision to proceed with a project should be based on a demand analysis for institutional strengthening programs that address staff availability and skills, and considers organization culture and willingness to change.

(iv) A multilevel approach to demand for project services should be adopted. Where small town water supply systems are to be constructed and handed over to a water enterprise, the primary client for the system should be the water enterprise. However, the project should ensure that the water enterprise has the capability and desire to provide responsive and satisfactory services to the beneficiary community.

(v) Where standard designs are to be used, the project implementation process should include procedures to ensure the applicability of the standard design to the subproject location, and to ensure that, where the standard design is not applicable, appropriate modifications are made. Where consultants are contracted to provide design services, inappropriate use of standard designs should be sufficient reason for reducing contract payments.

(vi) The primary role of the community should be during the planning phase for subprojects. This role should be exercised through real consultation leading to a detailed knowledge of the community’s preference for facilities and their willingness accept ownership of completed facilities either in terms of contributions to the operating entity or the formation of community based operating groups. The level of community willingness to be involved should be the first determinant in the decision to proceed to implementation of a subproject.

(vii) The level of detail of project scheduling should be sufficient to identify key precedence relationships between activities.

66. For the implementation of rural water supply and sanitation sector projects in Indonesia the following recommendations are made:

(i) For large and complex projects, ADB and the Government should develop and use appropriate project management systems based on valid implementation methodologies and robust implementation procedures.

(ii) Project management systems for complex projects should be managed and controlled by a single project management organization with sufficient authority to resolve implementation issues such as poor coordination and a nonresponsive

17

institutional environment, including removal of high level policy and regulatory constraints.

(iii) Consideration should be given to privatization of project implementation. In this model the role of Government would be largely policy development with project implementation being contracted out. This approach would shift detailed project covenants into project management contracts and provide better linkages between outcomes and payments.

(iv) Institutional strengthening activities of projects need to be sustained through improved acceptance of change within organizations, adequate documentation of improved systems, and training and handover procedures for new staff to minimize the impacts of staff mobility.

(v) Employment of consultants should be based on identified needs. The resources provided for their tasks should be adequate for the proposed purpose of their assignment. Auditors should be given sufficient incentive to provide reports that expose major project risk. Project management consultants should not submit reports that are inaccurate due to lack of resources to inspect works. Resources should be allocated on a risk-return basis to reduce loss from fraud and incompetence. For example, the auditing costs of the Project were around $50,000, and this was probably inadequate given the Project’s scope and coverage. A highly intensive auditing program would have saved more than it cost.

(vi) Contracting procedures should be based on achievement of project outputs and sufficient money withheld to ensure that works are completed to specification.

(vii) ADB or outsourced monitoring review missions need to be sufficiently frequent and cover a sufficient sample of project implementation to ensure that reported outputs are verified. Such missions should allow sufficient time and resources to check each reported output at that site. In view of the ADB human resources constraints in project administration, monitoring of such projects should be outsourced to a technical and financial team, to audit 10-15% of the facilities.

18 Appendix 1

PROJECT FRAMEWORK

Design Summary

Performance Indicators/Targets

Monitoring Mechanisms

Assumptions And Risks

Goal To raise the overall standard of living and economic productivity for rural low-income communities

• no major changes in

national economic or sector policy

• no major impacts on the project areas from political, natural or economic events

Purpose 1. To provide safe, adequate, and reliable water supply and sanitation (WSS) services to selected low-income communities through community-based arrangements 2. To support hygiene and sanitation education, water quality surveillance, and community management activities

• 3.0 million (m) people

with improved water by the end of 2000

• 2.3m people with improved sanitation facilities by end of 2000

Improved hygiene and sanitation education and community management capability in 3000 rural low income communities {2,750 lesser developed villages lesser developed desas (DTs), 250 growth center villages (DPPs)} by end of 2000

Benefit monitoring and evaluation system The Government’s project completion report

• delay in the implementation

caused by lack of counterpart funds

• delay in the implementation caused by lack of institutional capacity

• limited institutional capacity of the communities to operate and maintain the water supply systems

Outputs 1. Water supply facilities 2. Sanitation facilities 3. Institutional support and project administration

150 district administrative center (IKK) systems 73,000 house connections (HCs) 10,800 public hydrants 30,000 wells 500 rainwater collect. 4,900 handpumps 850 public latrines 2,600 school latrines 75,000 family latrines

Monthly reporting from Kabupaten offices Monthly reports from central project administration office (CPAO).

• all outputs are converted to

use by communities and project outcomes are achieved

• water sources are adequate

and use is feasible • institutional strengthening

provides sufficient capability to achieve outputs

Activities 1. Construction of piped and nonpiped water supply systems and or the rehabilitation of existing water supply

Inputs $54.4m for civil contracts $34.2m for equipment and materials

Monthly reports from central project administration office (CPAO).

Appendix 1 19

systems 2. Consulting services to carry out feasibility studies, detailed design, and construction supervision 3. Construction of public and private latrines 4. Consulting services for implementation assistance 5. A hygiene and sanitation education and water quality surveillance program 6. A training and media campaigns for a community management program 7. A water supply and sanitation (WSS) institutional development program 8. Incremental Project administration costs

$2.9m for feasibility studies 1450 person-months of consulting services $7.9m for civil works $3.4m for materials $4.0m for consulting assistance 434 person-months of consulting time $100,000 for public education programs and training $4.1m for community education and training $300,000 $7.6m

20 Appendix 2

Appendix 2 comprises 10 tables detailing water supply and sanitation facilities in all provinces under the Project (Table A2.1), and in the individual provinces: Bengkulu, D.I. Aceh, Jambi, Kalimantan Barat, Kalimantan Selatan, Kalimantan Tengah, Kalimantan Timur, Lampung, Riau, Sumatra Barat, Sumatra Selatan, and Sumatra Utara (Tables A2.2-A2.13).

WATER SUPPLY AND SANITATION FACILITIES PLANNES AND REPORTED

TABLE 2-1: All Provinces

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL1995-1996 - Reported 4 200 95 63 5 0 27 1,192 888 732 6,516 373 203 - Planned 12 3,630 249 1 10 0 16 1,710 0 9 5,170 47 190 - Difference (8) (3,430) (154) 62 (5) 0 11 (518) 888 723 1,346 326 13 1996-1997 - Reported 0 4,000 512 71 4 0 236 6,606 1,995 1,981 19,271 719 379 - Planned 39 13,607 988 10 45 0 33 7,305 2,441 148 26,005 182 947 - Difference (39) (9,607) (476) 61 (41) 0 203 (699) (446) 1,833 (6,734) 537 (568)1997-1998 - Reported 13 3,420 749 125 181 0 260 7,987 1,496 2,998 24,156 1,717 504 - Planned 22 22,965 1,152 16 43 0 47 6,640 1,249 44 21,620 171 849 - Difference (9) (19,545) (403) 109 138 0 213 1,347 247 2,954 2,536 1,546 (345)1998-1999 - Reported 67 28,960 1,185 14 79 40 224 8,816 425 290 18,081 777 396 - Planned 68 32,486 1,249 14 26 40 145 10,171 642 153 22,255 434 591 - Difference (1) (3,526) (64) 0 53 0 79 (1,355) (217) 137 (4,174) 343 (195)1999-2000 - Reported 57 338 7,150 0 0 0 13 4,758 407 124 260 18 8 - Planned 57 338 7,150 0 0 0 13 4,758 567 124 260 18 8 - Difference 0 0 0 0 0 0 0 0 (160) 0 0 0 0 Total - Reported 141 36,918 9,691 273 269 40 760 29,359 5,211 6,125 68,284 3,604 1,490 - Planned 198 73,026 10,788 41 124 40 254 30,584 4,899 478 75,310 852 2,585 - Difference (57) (36,108) (1,097) 232 145 0 506 (1,225) 312 5,647 (7,026) 2,752 (1,095)

DW= deep well, DWHP= deep well with hand pump, FL= family latrine, GPS= gravity pipe system, HC = house connection, IG= Infiltration Gallery, PH= public hydrant, PL= public latrine, RWC= rain water collector, SL= school latrine, SWTP= simple water treatment plant, Well= hand dug well, WTP= water treatment plant

Appendix 2 21

TABLE A2.2: BENGKULU

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL

1995-1996 - Reported 0 0 5 0 0 0 1 93 76 0 30 0 10 - Planned 1 0 15 0 0 0 3 150 0 0 490 4 17 - Difference (1) 0 (10) 0 0 0 (2) (57) 76 0 (460) (4) (7)1996-1997 - Reported 0 1,240 44 0 0 0 5 0 0 0 0 0 0 - Planned 5 870 94 0 0 0 0 585 11 0 1,810 17 72 - Difference (5) 370 (50) 0 0 0 5 (585) (11) 0 (1,810) (17) (72)1997-1998 - Reported 3 540 63 0 0 0 16 552 92 5 0 65 0 - Planned 3 1,163 116 0 0 0 6 135 8 0 740 16 41 - Difference 0 (623) (53) 0 0 0 10 417 84 5 (740) 49 (41)1998-1999 - Reported 4 777 15 0 0 0 34 1,244 30 12 879 46 0 - Planned 3 503 45 0 0 0 3 360 31 0 1,110 9 42 - Difference 1 274 (30) 0 0 0 31 884 (1) 12 (231) 37 (42)1999-2000 - Reported 2 10 200 0 0 0 0 255 20 0 0 0 0 - Planned 2 10 200 0 0 0 0 255 20 0 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 9 2,567 327 0 0 0 56 2,144 218 17 909 111 10 - Planned 14 2,546 470 0 0 0 12 1,485 70 0 4,150 46 172 - Difference (5) 21 (143) 0 0 0 44 659 148 17 (3,241) 65 (162)


22 Appendix 2

TABLE A2.3: D.I. Aceh

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL1995-1996 - Reported 0 0 0 10 0 0 3 196 146 64 1,074 43 26 - Planned 1 0 22 0 2 0 2 450 0 0 1,035 8 38 - Difference (1) 0 (22) 10 (2) 0 1 (254) 146 64 39 35 (12)1996-1997 - Reported 0 0 0 8 3 0 17 1,449 135 50 3,015 89 17 - Planned 0 450 55 2 4 0 5 1,575 45 0 3,675 15 127 - Difference 0 (450) (55) 6 (1) 0 12 (126) 90 50 (660) 74 (110)1997-1998 - Reported 0 0 0 8 28 0 58 1,923 80 12 3,426 191 4 - Planned 0 2,278 20 0 0 0 3 1,545 0 0 3,175 3 106 - Difference 0 (2,278) (20) 8 28 0 55 378 80 12 251 188 (102)1998-1999 - Reported 7 14,988 222 0 0 0 33 1,614 125 2 2,750 312 1 - Planned 7 14,988 222 0 4 0 12 1,369 26 0 2,751 66 106 - Difference 0 0 0 0 (4) 0 21 245 99 2 (1) 246 (105)1999-2000 - Reported 12 60 1,200 0 0 0 0 465 10 0 80 4 4 - Planned 12 60 1,200 0 0 0 0 465 10 0 80 4 4 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0 Total - Reported 19 15,048 1,422 26 31 0 111 5,647 496 128 10,345 639 52 - Planned 20 17,776 1,519 2 10 0 22 5,404 81 0 10,716 96 381 - Difference (1) (2,728) (97) 24 21 0 89 243 415 128 (371) 543 (329)


Appendix 2 23

TABLE A2.4: Jambi

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL1995-1996 - Reported 3 200 48 0 0 0 0 224 12 70 485 0 16 - Planned 1 220 16 0 0 0 1 0 0 0 140 2 4 - Difference 2 (20) 32 0 0 0 (1) 224 12 70 345 (2) 121996-1997 - Reported 0 1,960 87 0 0 0 54 834 15 64 755 77 27 - Planned 2 1,560 125 6 6 0 3 0 1,140 0 1,970 16 86 - Difference (2) 400 (38) (6) (6) 0 51 834 (1,125) 64 (1,215) 61 (59)1997-1998 - Reported 9 1,980 29 0 0 0 11 1,016 45 97 1,654 79 67 - Planned 7 3,215 125 10 2 0 4 0 0 0 1,080 24 48 - Difference 2 (1,235) (96) (10) (2) 0 7 1,016 45 97 574 55 191998-1999 - Reported 3 650 14 0 0 0 15 794 65 60 824 44 15 - Planned 3 650 14 0 0 0 15 794 65 60 824 44 15 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 01999-2000 - Reported 3 58 600 0 0 0 10 315 30 14 0 0 0 - Planned 3 58 600 0 0 0 10 315 30 14 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 18 4,848 778 0 0 0 90 3,183 167 305 3,718 200 125 - Planned 16 5,703 880 16 8 0 33 1,109 1,235 74 4,014 86 153 - Difference 2 (855) (102) (16) (8) 0 57 2,074 (1,068) 231 (296) 114 (28)


24 Appendix 2

TABLE A2.5: Kalimantan Barat


1995-1996 - Reported 0 0 7 0 3 0 2 0 30 146 582 23 22 - Planned 0 100 11 0 1 0 1 0 0 0 130 2 4 - Difference 0 (100) (4) 0 2 0 1 0 30 146 452 21 181996-1997 - Reported 0 0 147 0 1 0 21 122 87 837 1,303 183 39 - Planned 3 1,589 62 0 3 0 4 435 0 140 1,960 17 63 - Difference (3) (1,589) 85 0 (2) 0 17 (313) 87 697 (657) 166 (24)1997-1998 - Reported 0 0 292 0 0 0 43 176 129 1,019 1,543 213 0 - Planned 0 0 31 0 6 0 1 435 0 20 1,350 8 45 - Difference 0 0 261 0 (6) 0 42 (259) 129 999 193 205 (45)1998-1999 - Reported 7 1,400 70 0 0 0 0 1,180 0 0 3,480 0 126 - Planned 7 1,400 70 0 0 0 0 3,210 0 0 6,540 0 126 - Difference 0 0 0 0 0 0 0 (2,030) 0 0 (3,060) 0 01999-2000 - Reported 1 5 200 0 0 0 0 468 0 44 0 0 0 - Planned 1 5 200 0 0 0 0 468 0 44 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 8 1,405 716 0 4 0 66 1,946 246 2,046 6,908 419 187 - Planned 11 3,094 374 0 10 0 6 4,548 0 204 9,980 27 238 - Difference (3) (1,689) 342 0 (6) 0 60 (2,602) 246 1,842 (3,072) 392 (51)


Appendix 2 25

TABLE A2.6: Kalimantan Selatan


1995-1996 - Reported 0 0 2 1 0 0 1 231 266 97 786 61 15 - Planned 5 2,780 25 0 0 0 0 345 0 9 1,000 7 31 - Difference (5) (2,780) (23) 1 0 0 1 (114) 266 88 (214) 54 (16)1996-1997 - Reported 0 0 12 0 0 0 13 249 509 232 3,300 0 124 - Planned 7 1,805 111 0 2 0 5 1,965 0 0 4,910 14 159 - Difference (7) (1,805) (99) 0 (2) 0 8 (1,716) 509 232 (1,610) (14) (35)1997-1998 - Reported 0 0 61 18 26 0 40 229 481 366 4,210 150 126 - Planned 2 1,749 205 4 5 0 9 1,900 1 0 5,015 26 179 - Difference (2) (1,749) (144) 14 21 0 31 (1,671) 480 366 (805) 124 (53)1998-1999 - Reported 9 900 90 0 8 0 0 555 0 30 1,740 9 55 - Planned 9 1,800 90 0 8 0 0 555 0 30 1,740 9 55 - Difference 0 (900) 0 0 0 0 0 0 0 0 0 0 01999-2000 - Reported 2 10 300 0 0 0 0 540 0 0 0 0 0 - Planned 2 10 300 0 0 0 0 540 0 0 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 11 910 465 19 34 0 54 1,804 1,256 725 10,036 220 320 - Planned 25 8,144 731 4 15 0 14 5,305 1 39 12,665 56 424 - Difference (14) (7,234) (266) 15 19 0 40 (3,501) 1,255 686 (2,629) 164 (104)


26 Appendix 2

TABLE A2.7: Kalimantan Tengah


1995-1996 - Reported 0 0 0 52 0 0 0 0 4 156 676 23 17 - Planned 1 330 20 0 2 0 0 0 0 0 210 3 6 - Difference (1) (330) (20) 52 (2) 0 0 0 4 156 466 20 111996-1997 - Reported 0 0 14 50 0 0 14 285 115 182 1,625 34 20 - Planned 6 1,076 94 0 11 0 0 0 0 0 1,190 17 34 - Difference (6) (1,076) (80) 50 (11) 0 14 285 115 182 435 17 (14)1997-1998 - Reported 0 0 0 0 92 0 0 27 269 0 2,174 16 72 - Planned 4 647 67 0 12 0 0 0 0 0 1,000 16 32 - Difference (4) (647) (67) 0 80 0 0 27 269 0 1,174 0 401998-1999 - Reported 2 348 25 0 52 0 0 96 0 115 1,080 0 55 - Planned 2 348 25 0 3 0 0 0 0 0 320 5 10 - Difference 0 0 0 0 49 0 0 96 0 115 760 (5) 451999-2000 - Reported 0 0 0 0 0 0 0 150 10 0 0 0 0 - Planned 0 0 0 0 0 0 0 150 10 0 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 2 348 39 102 144 0 14 558 398 453 5,555 73 164 - Planned 13 2,401 206 0 28 0 0 150 10 0 2,720 41 82 - Difference (11) (2,053) (167) 102 116 0 14 408 388 453 2,835 32 82


Appendix 2 27

TABLE A2.8: Kalimantan Timur


1995-1996 - Reported 0 0 0 0 0 0 0 143 14 67 695 22 0 - Planned 1 0 30 0 4 0 0 0 0 0 350 5 10 - Difference (1) 0 (30) 0 (4) 0 0 143 14 67 345 17 (10)1996-1997 - Reported 0 0 49 0 0 0 0 320 13 96 1,582 19 0 - Planned 9 0 211 0 12 0 0 0 0 0 1,470 21 42 - Difference (9) 0 (162) 0 (12) 0 0 320 13 96 112 (2) (42)1997-1998 - Reported 0 0 132 0 0 0 0 0 0 218 2,166 90 0 - Planned 2 0 122 0 12 0 0 0 0 0 860 14 28 - Difference (2) 0 10 0 (12) 0 0 0 0 218 1,306 76 (28)1998-1999 - Reported 11 1,200 97 0 4 0 0 540 0 0 1,860 4 42 - Planned 11 2,500 97 0 4 0 0 1,800 0 0 3,480 4 42 - Difference 0 (1,300) 0 0 0 0 0 (1,260) 0 0 (1,620) 0 01999-2000 - Reported 3 15 600 0 0 0 0 285 42 0 0 0 0 - Planned 3 15 600 0 0 0 0 285 42 0 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 14 1,215 878 0 4 0 0 1,288 69 381 6,303 135 42 - Planned 26 2,515 1,060 0 32 0 0 2,085 42 0 6,160 44 122 - Difference (12) (1,300) (182) 0 (28) 0 0 (797) 27 381 143 91 (80)


28 Appendix 2

TABLE A2.9: Lampung


1995-1996 - Reported 0 0 0 0 0 0 2 143 145 0 247 72 22 - Planned 0 0 0 0 0 0 0 0 0 0 0 0 0 - Difference 0 0 0 0 0 0 2 143 145 0 247 72 22 1996-1997 - Reported 0 0 60 0 0 0 21 693 964 0 1,470 87 41 - Planned 1 653 76 2 1 0 4 0 1,200 0 2,630 14 100 - Difference (1) (653) (16) (2) (1) 0 17 693 (236) 0 (1,160) 73 (59)1997-1998 - Reported 1 900 42 0 0 0 0 1,371 141 0 2,406 118 34 - Planned 0 620 112 2 5 0 2 0 1,170 0 2,590 13 107 - Difference 1 280 (70) (2) (5) 0 (2) 1,371 (1,029) 0 (184) 105 (73)1998-1999 - Reported 2 471 108 0 0 0 13 1,058 0 0 1,317 51 0 - Planned 4 2,071 142 0 0 0 10 0 315 0 1,080 12 43 - Difference (2) (1,600) (34) 0 0 0 3 1,058 (315) 0 237 39 (43)1999-2000 - Reported 1 20 200 0 0 0 3 90 80 0 0 0 0 - Planned 1 20 200 0 0 0 3 90 240 0 0 0 0 - Difference 0 0 0 0 0 0 0 0 (160) 0 0 0 0 Total - Reported 4 1,391 410 0 0 0 39 3,355 1,330 0 5,440 328 97 - Planned 6 3,364 530 4 6 0 19 90 2,925 0 6,300 39 250 - Difference (2) (1,973) (120) (4) (6) 0 20 3,265 (1,595) 0 (860) 289 (153)


Appendix 2 29

TABLE A2.10: Riau


1995-1996 - Reported 0 0 0 0 0 0 0 20 53 130 438 19 17 - Planned 0 0 15 0 0 0 3 315 0 0 660 6 30 - Difference 0 0 (15) 0 0 0 (3) (295) 53 130 (222) 13 (13)1996-1997 - Reported 0 0 0 0 0 0 0 500 0 310 905 77 42 - Planned 2 2,986 44 0 2 0 3 1,350 0 8 3,195 17 115 - Difference (2) (2,986) (44) 0 (2) 0 (3) (850) 0 302 (2,290) 60 (73)1997-1998 - Reported 0 0 22 72 0 0 0 806 4 1,208 840 85 18 - Planned 1 2,574 66 0 0 0 3 1,305 0 24 3,035 16 110 - Difference (1) (2,574) (44) 72 0 0 (3) (499) 4 1,184 (2,195) 69 (92)1998-1999 - Reported 5 2,239 54 0 0 0 1 420 0 8 1,040 6 37 - Planned 5 2,239 54 0 0 0 1 420 0 8 1,040 6 37 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 01999-2000 - Reported 3 15 300 0 0 0 0 540 10 21 0 0 0 - Planned 3 15 300 0 0 0 0 540 10 21 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 8 2,254 376 72 0 0 1 2,286 67 1,677 3,223 187 114 - Planned 11 7,814 479 0 2 0 10 3,930 10 61 7,930 45 292 - Difference (3) (5,560) (103) 72 (2) 0 (9) (1,644) 57 1,616 (4,707) 142 (178)


30 Appendix 2

TABLE A2.11: Sumatera Barat


1995-1996 - Reported 0 0 0 0 2 0 18 6 48 2 656 15 23 - Planned 0 0 26 0 0 0 3 450 0 0 705 3 36 - Difference 0 0 (26) 0 2 0 15 (444) 48 2 (49) 12 (13)1996-1997 - Reported 0 0 0 0 0 0 50 406 35 22 1,180 70 17 - Planned 2 962 70 0 4 0 4 1,290 0 0 2,055 15 111 - Difference (2) (962) (70) 0 (4) 0 46 (884) 35 22 (875) 55 (94)1997-1998 - Reported 0 0 0 0 0 0 69 414 0 0 2,416 105 20 - Planned 2 3,310 113 0 1 0 9 1,260 30 0 2,125 15 113 - Difference (2) (3,310) (113) 0 (1) 0 60 (846) (30) 0 291 90 (93)1998-1999 - Reported 4 3,167 263 0 13 0 67 447 0 8 1,031 70 11 - Planned 4 3,167 263 0 5 0 43 795 0 0 1,290 44 61 - Difference 0 0 0 0 8 0 24 (348) 0 8 (259) 26 (50)1999-2000 - Reported 13 65 1,300 0 0 0 0 345 0 0 50 3 2 - Planned 13 65 1,300 0 0 0 0 345 0 0 50 3 2 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0 Total - Reported 17 3,232 1,563 0 15 0 204 1,618 83 32 5,333 263 73 - Planned 21 7,504 1,772 0 10 0 59 4,140 30 0 6,225 80 323 - Difference (4) (4,272) (209) 0 5 0 145 (2,522) 53 32 (892) 183 (250)


Appendix 2 31

TABLE A2.12: Sumatera Selatan

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL1995-1996 - Reported 1 0 5 0 0 0 0 132 31 0 427 95 21 - Planned 2 200 32 0 0 0 0 0 0 0 140 2 4 - Difference (1) (200) (27) 0 0 0 0 132 31 0 287 93 171996-1997 - Reported 0 800 20 0 0 0 0 1,724 68 81 2,126 29 0 - Planned 2 609 14 0 0 0 0 0 0 0 140 2 4 - Difference (2) 191 6 0 0 0 0 1,724 68 81 1,986 27 (4)1997-1998 - Reported 0 0 0 0 2 0 2 1,410 202 1 1,571 93 135 - Planned 1 258 52 0 0 0 0 0 0 0 70 1 2 - Difference (1) (258) (52) 0 2 0 2 1,410 202 1 1,501 92 1331998-1999 - Reported 4 800 30 0 2 0 20 844 144 0 1,071 158 16 - Planned 4 800 30 0 2 0 20 844 144 0 1,071 158 16 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 01999-2000 - Reported 6 35 1,150 0 0 0 0 600 100 45 0 0 0 - Planned 6 35 1,150 0 0 0 0 600 100 45 0 0 0 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0Total - Reported 11 1,635 1,205 0 4 0 22 4,710 545 127 5,195 375 172 - Planned 15 1,902 1,278 0 2 0 20 1,444 244 45 1,421 163 26 - Difference (4) (267) (73) 0 2 0 2 3,266 301 82 3,774 212 146


32 Appendix 2

TABLE A2.13: Sumatera Utara

WTP HC PH IG SWTP DW GPS WELL DWHP RWC FL PL SL1995-1996 - Reported 0 0 28 0 0 0 0 4 63 0 420 0 14 - Planned 0 0 37 1 1 0 3 0 0 0 310 5 10 - Difference 0 0 (9) (1) (1) 0 (3) 4 63 0 110 (5) 4 1996-1997 - Reported 0 0 79 13 0 0 41 24 54 107 2,010 54 52 - Planned 0 1,047 32 0 0 0 5 105 45 0 1,000 17 34 - Difference 0 (1,047) 47 13 0 0 36 (81) 9 107 1,010 37 18 1997-1998 - Reported 0 0 108 27 33 0 21 63 53 72 1,750 512 28 - Planned 0 7,151 123 0 0 0 10 60 40 0 580 19 38 - Difference 0 (7,151) (15) 27 33 0 11 3 13 72 1,170 493 (10)1998-1999 - Reported 9 2,020 197 14 0 40 41 24 61 55 1,009 77 38 - Planned 9 2,020 197 14 0 40 41 24 61 55 1,009 77 38 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0 1999-2000 - Reported 11 45 1,100 0 0 0 0 705 105 0 130 11 2 - Planned 11 45 1,100 0 0 0 0 705 105 0 130 11 2 - Difference 0 0 0 0 0 0 0 0 0 0 0 0 0 Total - Reported 20 2,065 1,512 54 33 40 103 820 336 234 5,319 654 134 - Planned 20 10,263 1,489 15 1 40 59 894 251 55 3,029 129 122 - Difference 0 (8,198) 23 39 32 0 44 (74) 85 179 2,290 525 12


Appendix 2 33

PROJECT BUDGET AND COSTS ($'000)

Budget at Appraisal Actual Cost Foreign Local Total Foreign Local Total Item Exchange Currency Cost Exchange Currency Cost A. Water Supply 1. Land 0.0 4.8 4.8 0.0 0.0 0.0 2. Civil Works 7.1 47.3 54.4 9.5 29.7 39.2 3. Equipment and Materials 20.2 14.0 34.2 19.8 5.2 25.0 4. Feasibility Studies and Design 0.0 2.9 2.9 0.8 1.0 1.8 5. Work Supervision 0.0 1.4 1.4 0.0 0.9 0.9 6. Incremental Cost 0.0 6.9 6.9 0.0 0.8 0.6 Subtotal (A) 27.3 77.3 104.6 30.1 37.6 67.5 B. Sanitation 1. Civil Works 0.0 7.9 7.9 0.0 11.6 11.6 2. Equipment and Materials 0.0 3.4 3.4 0.0 0.8 0.8 3. Incremental Cost 0.0 0.7 0.7 0.0 0.7 0.7 Subtotal (B) 0.0 12.0 12.0 0.0 13.1 13.1 C. Institutional Support 1. Implementation Assistance 0.9 3.1 4.0 0.4 3.0 3.2 2. Hygiene, Sanitation, Water Quality 0.0 0.1 0.1 0.0 0.2 0.2 3. Community Management Program 0.0 4.1 4.1 1.3 1.4 2.9 4. Institutional Strengthening Program 0.0 0.3 0.3 0.0 0.4 0.6 5. Project Administration 0.0 7.1 7.1 0.0 7.7 7.7 Subtotal (C) 0.9 14.7 15.6 1.7 12.7 14.6 D. Interest During Construction 9.8 0.0 9.8 9.8 0.0 9.8 Total 38.0 104.0 142.0 41.6 63.4 105.0

Source: Asian Development Bank estimates.

34 Appendix 2

LOAN DISBURSEMENTS Original Undisbursed Undisbursed Budget Contracted Disbursed Contract Allocated

Description 1995 Balance BalancePart A: Water Supply

Civil Works 27,200 26,893 19,671 7,222 7,529 Equipment & Material 27,360 24,139 20,580 3,559 6,780 Consulting Services 5,540 2,796 2,271 525 2,819

Part B : Sanitation

Civil Works 3,950 4,980 4,703 227 -753 Equipment & Material 2,650 2,573 2,189 384 462

Part C: Institutional Support 8,500 7,638 6,890 748 1,610 Interest During Construction 9,800 0 9,800 -9,800 0 Imprest Account 763 -763

TOTAL 85,000 69,020 67,316 1,704 17,684Note : Exchange rates were as follows 1995/1996 1 $=Rp2,100 1996/1997 1 $=Rp2,200 1997/1998 1 $=Rp3,700 1998/1999 1 $=Rp7,520 1999/2000 1 $=Rp8,500

Appendix 2 35

IMPLEMENTATION SCHEDULE 1994 1995 1996 1997 1998 1999 2000 Preselection of Communities Proposed Training in Village Action Plan Preparation Implemented Final list of Communities Proposed Consulting Services Contract 1 Proposed Consulting Services Contract 2 Proposed Implemented Consulting Services Contract 3 to 5 Proposed Implemented Community Management Proposed Implemented Health Education Measures Proposed Implemented Surveys in the Field Proposed Design and Tendering Proposed &

implemented

Construction of the Systems Proposed &

implemented

Training in System Construction Implemented Operation of the Systems Proposed &

implemented

36 Appendix 6

PROJECT ORGANIZATION

Agency for Regional

Planning and Development

VILLAGE

CENTRAL

PROVINCE

DISTRICT

DGHS DGCDC & EH

National Planning

Board

DGRD DGRCD DG of Budget

MOH MOF MOHA MPW

Project Implementation Team

Kabupaten Project Office

Bupati

MPW Provincial

Office

Project Implementation Team

Public Works Office

Health Office RCD Office

Central Project Administration

Office

Project Project Project

Community Support Group /

Village Development

Council

Governor

Village Head

Head of BAPPEDA Kabupaten

Dept of Health

Provincial Office

RCD Provincial

Office

RCD DGHS Provincial

Office

Legend Commend Cooperation Coordination

36 Appendix 6

BAPPEDA=Badan Perencanaan Pembangunan Daerah (Agency for Regional Planning), DG=Directorate General, DGCDCEH=Directorate General of Communicable Disease Control and Environmental Health, DGHS=Directorate General of Human Settlements, DGRCD=Directorate General of Rural Community Development, DGRD=Directorate General of Regional Development, MOF=Ministry of Finance, MOH=Ministry of Health, MOHA=Ministry of Home Affairs, MPW=Ministry of Public Works, RCD=rural community development

Appendix 7 39

STATUS OF COMPLIANCE WITH LOAN COVENANTS

Covenant

Reference in Loan

Agreement

Status of Compliance Operational 1. The Borrower will make available, promptly as

needed, the funds, facilities, services, land, and other resources required, in addition to the Project and for the operation and maintenance of the project facilities.

Sec. 4.02

Complied with. All funds, facilities, services, land and other necessary resources (in addition to the proceeds of the loan) have been made available as required.

2. The Borrower will ensure that all land, rights in land, and other right and privileges required for the Project are acquired or otherwise made available on a timely basis.

Sch. 5, para. 13

Complied with.

3. The Borrower will insure make arrangements satisfactory to the Asian Development Bank (ADB) for insurance of the project facilities to such extent and against such risks and in such amounts as shall be consistent with sound practice.

Sec. 4.05 (a) The Government accepted responsibility for facility damage and claims risks directly, rather than through specific insurance policies.

4. The Borrower will ensure the goods to be imported for the Project and to be financed out of the proceeds of the loan against hazards incident to the acquisition, transportation and delivery thereof to the place of use or installation.

Sec. 4.05 (b) These risks were primarily covered by the contractor’s insurance policies, as required in the standard Ministry of Public Works contract documents.

5. The Borrower will ensure that Directorate General of Human Settlements (DGHS) furnishes to ADB consolidated quarterly reports on the carrying out of the Project and on the operation and management of the project facilities.

Sec. 4.07 (b) Complied with. Quarterly progress reports were submitted by the central project administration office (CPAO). Progress reports not supplied by Directorate General of Communicable Disease Control and Environmental Health (DGCDCEH), Directorate General of Rural Community Development (DGRCD), Directorate General Rural Development (DGRD) to Directorate General of Human Settlements (DGHS) for incorporation in quarterly progress report.

Appendix 7 37

38 Appendix 7


Covenant

Reference in Loan

Agreement

Status of Compliance 6. After physical completion of the Project, but not

later than 3 (three) months thereafter, the Borrower will ensure that DGHS coordinates with DGCDEH, DGRCD and DGRD on their respective components of the Project before DGHS prepares and furnishes to ADB a consolidate project completion report.

Sec. 4.07 (c) Submitted.

7. The Borrower will (i) ensure that DGCDCEH, DGHS, and DGRD maintain separate accounts for their respective components; (ii) have such accounts audited annually by independent auditors acceptable to ADB; (iii) furnish to ADB, through DGHS, not later than 9 months after the end of each fiscal year fiscal year (FY), certified copies of such audited account and financial statements and the English language.

Sec. 4.06 (b) Complied with. The report received on 13 March 2001.

8. The Borrower will cause its auditors, acceptable to ADB, to periodically audit the accounts and records relating to the special account and statement of expenditure (SOE), and furnish the audit reports to ADB not later than 9 months after the end of each related FY.

Sec. 5, para. 14 (b)

The copies of audited accounts and financial statement and/or the audit reports were submitted as required.

Implementation 9. DGHS will be the Project Executing Agency and

coordinate project activities with Agency for National Planning (BAPPENAS), DGCDCEH, DGRCD, DGRD, and any other agencies concerned.

Sec. 5, para. 1

Complied with. .

10. BAPPENAS will coordinate with the government agencies on policy matters and shall provide guidance on annual budgetary allocations relating to the project component.

Sec. 5, para. 2

Complied with.

11. DGHS will appoint, within 1 month after the effective date of the loan, a central project administrator to manage and coordinate the Project.

Sch. 5, para. 5

Complied with. The CPAO was established on March 10, 1995.

12. DGCDCEH will be the Implementing Agency for the Hygiene and Sanitation Education program under part C (ii) of the Project.

Sch. 5, para. 5

Commenced in 1997. DGCDCEH implemented the programs, but progress reports were not submitted by DGCDCEH to the CPAO.

Appendix 7 39


Covenant

Reference in Loan

Agreement

Status of Compliance 13. DGRCD will be the Implementing Agency for the

community management program under Part C (iii) of the Project.

Sch. 5, para. 6

Complied with. DGRCD implemented the community management program, but progress reports of its central level activities were not submitted to the CPAO.

14. DGRD will be the Implementing Agency for Part C (iv) of the Project relating to the training of provincial and Kabupaten office for the development and monitoring of the water supply and sanitation (WSS) management program.

Sch. 5, para. 8

Completed for FY1995/1996. Additional locations were included in FY1998/1999. Training was completed.

15. The governor of the province concerned will be responsible for developing the WSS management program and monitoring implementation of project activities at the provincial level.

Sch. 5, para. 9

Complied with. All governors have accepted responsibility for development of the WSS management program and program monitoring at the provincial level.

16. The bupati of the kabupaten concerned will be responsible for managing implementation of project activities at the kabupaten level.

Sch. 5, para. 10

Complied with. All bupatis accepted responsibility for managing implementation of project activities at the kabupaten level.

17. The Borrower will ensure that the project activities at the kabupaten level are carried out by public works department (DPU) II for construction and/or rehabilitation of WSS schemes, dinas kesehatan II for the hygiene and sanitation education program and water quality surveillance program, and village community development (PMD) II for the community management program, and the program development division BAPPEDA II of the kabupaten concerned for the development of WSS management program and monitoring of project activities.

Sch. 5, para. 11

Complied with. Responsibility was delegated to respective agencies as required by Loan Agreement.

18. The Borrower will ensure that the village community empowerment units (LKMDs) concerned participate in all aspects of Project activities and coordinate with district administrative center (IKK) and water users groups

Sch. 5, para. 12

Complied with.

40 Appendix 7


Covenant

Reference in Loan

Agreement

Status of Compliance 19. For each kabupaten included in the Project, the

BAPPEDA II concerned will prepare a consolidated list of subproject proposals for the kabupaten and there after, submit proposals to bupati concerned who will in tern submit these to the provincial governor concerned.

Sch. 5, para. 14

Complied with.

20. All concerned dinas at the kabupaten level will implement the subprojects approved by DGHS or ADB, in coordination with public works department (DPU) II, bupati of the kabupatens, heads of kecamatans and desas and the community-based organizations concerned.

Sch. 5, para. 15

Complied with.

21. BAPPEDA I will appraise the subproject proposals forwarded by the bupati concerned in coordination with the central government representative at the provincial level, public works department (DPU) I, dinas kesehatan I, and any other agency concerned, and submit the subproject proposals to DGHS.

Sch. 5, para. 16

Complied with.

22. DGHS will obtain ADB’s prior approval for (i) the first subproject proposal in each province covered under the Project; (ii) any subproject proposal, which includes a WSS scheme costing $500,000 or more. DGHS will submit to ADB an application for the approval of each subproject before requesting any withdrawals in respect of such subproject.

Sch. 5, para. 18 (a)

Complied with. ADB approval obtained for first subproject proposal in each province as required. No subproject proposal cost $500,000 or more.

23. DGHS may approve all other subproject proposals but will inform ADB promptly after approval. DGHS will submit to ADB a summary description of each subproject approved before requesting withdrawal.

Sch. 5, para. 18 (b)

Complied with. Summary descriptions of each subproject approved by DGHS were submitted to ADB prior to requesting withdrawal.

24. DGHS shall submit to ADB for its review and approval the initial environmental examination (IEE) and environmental impacts assessment (EIA), if required, of the first subproject proposal in each province covered under the project and any subproject proposal estimated to cost the equivalent of $500,000 or more. DGHS will ensure that the IEE is cleared by the BAPPEDA.

Sch. 5, para. 19

Not required because projects had no significant detrimental effect on the environment.

Appendix 7 39


Covenant

Reference in Loan

Agreement

Status of Compliance 25. DGHS will ensure that each proposed

subproject meets the following criteria before the proposal is approved by DGHS or ADB, (i) project community to be served by the subproject lacks basic and reliable water supply service; (ii) IEE and EIA, if required, for the subproject, including formulation of mitigation measures, will have been prepared; (iii) WSS scheme will have demand surveys, use the appropriate technology, and represent the least cost solution; (iv) project community will be responsible for the operation and maintenance (O&M) costs and capital cost recovery of the water supply facilities to be constructed/rehabilitated; (v) project community to be served by the subproject will have poor sanitary conditions; (vi) project community will be responsible for the O&M costs and part of construction cost of the sanitation facilities to be provided.

Sch. 5, para. 17

Partly complied with. DGHS checked each subproject proposal for compliance with specific criteria prior to approval and forwarding to ADB. Demand surveys were not adequate, and many facilities were inappropriate and did not represent the least cost effective solution. ADB approved the first subproject in each province.

26. The Borrower will ensure that the project communities are responsible for the O&M costs and capital cost recovery of the water supply facilities constructed/rehabilitated as follows: (i) O&M costs will be covered in part by contributions from the project community; (ii) part of the capital costs will be provided by the project community in cash and/or kind; (iii) for private household connections, users will pay the full costs of the connection including service pipes.

Sch. 5, para. 20

Project communities are directly responsible in lesser developed village (DT) and growth center village (DPP) locations and through regional water supply enterprise (PDAM) operations in district administrative center (IKK) locations.

27. The Borrower will ensure that the project communities provide for the O&M costs and part of the construction costs of the public and private sanitation facilities.

Sch. 5, para. 21

Complied with.

Training 28. Within 6 months after the effective date,

DGRCD and DGRD will develop training programs on community management and furnish them to ADB for approval.

Sch. 5, para. 22

Not complied with. The programs are yet to be submitted to ADB.

Appendix 7 41

42 Appendix 7


Covenant

Reference in Loan

Agreement

Status of Compliance 29. Within 6 months after the effective date,

DGCDCEH will develop a training program on hygiene and sanitation education, and water quality surveillance, and furnish it to ADB for approval.

Sch. 5, para. 23

Not complied with although such training programs were allegedly completed in 1997. The programs are yet to be submitted to ADB.

Benefit Monitoring and Evaluation 30. Within 6 months after the effective date, DGHS

will establish a benefit monitoring and evaluation system, subject to the approval of ADB.

Sch. 5, para. 25

Complied with. Project benefit monitoring and evaluation report submitted to ADB.

Project Review 31. DGHS will evaluate the benefits of the Project

after the Project has been completed in accordance with a schedule and term of reference to be agreed upon by DGHS and ADB.

Sch. 5, para .26

Not yet due.

32. Comprehensive review of project implementation will be carried out by the Borrower and ADB in the third year of the project implementation.

Sch. 5, para. 27

Complied with.

33. The Borrower and ADB will carry out midterm review of all aspects of the Project in the third year of the project implementation, or any date as may be agreed upon by the parties.

Sch. 5, para. 28

Complied with.

34. The Borrower and ADB will consult each other on a regular basis regarding the progress of Borrowers Sixth Five-Year Development Plan (REPELITA VI) for the Rural Water Supply and Sanitation Sector, including issues and strategies concerning the sector.

Sch. 5, para. 29

Complied with.

42 Appendix 7

Appendix 7 39

CONSULTING SERVICES Tasks Input Cost

(Person months)

Rupiah (million)

$ (‘000)

Other (‘000)

Technical Assistance

Preparation and evaluation of SPAR and DED for Project

Preparation and Implementation for FY1995/96. 45 838.8 TW 1,649.6

Project Preparation Implementation and Supervision (PPIS). Assistance to Provincial Officials in Preparation of SPAR and DED and in Monitoring Civil Works Constructions.

2,645 3,845.2

Assistance to Provincial Officials to prepare SPAR and DED for

Additional Locations. 653 5,967.9

Project Management

Project Preparation Implementation and Supervision (PPIS). Assisting the Executing Agency in Programming, Budgeting, Reporting and Coordinating Project Activities. Assisting in Training Preparation and Implementation and on Benefit and Monitoring Programs. Assisting in Project Management.

269 3,570.0 106.9

Secretarial Technical Assistance to CPAO, in Administration and

Finance. 111 852.9

Assistance to KPS in Project Management and Monitoring

Progress of Activities in the Concerned Kabupaten. 1,440 4,388.0

Assistance to CPAO in Evaluating the Project Implementation of

the Years 1995–1997. 30 2,746.4

Technical Assistance to KPS in Project Management and

Monitoring Progress of Activities in the Concerned Kabupaten.1,137 6,089.6

Assistance to CPAO for the Analysis of the Impact of the

Institutional Rules on RWSS Sustainability. 24 84.5

PCR-RWSS Project, Assistance to CPAO 18 147.6 Assistance to CPAO in Evaluating the Project Implementation 30 199.5 Advisory to Kabupaten Project Secretaries 1,115 5913.6 Financial Auditing 195 766.1 Training

Administration of Foreign Aid Programs and Procurement Procedures.

2,685.0

Guidelines for Operation and Maintenance 81.7 Operation and Maintenance for Operators 433.8 Hygiene Sanitation Education and Water Quality Control 126.3

Community Development Improvement for Village Institutional

Strengthening 666.6

Community-Based Management Approach for District

Coordination Teams 1,452.0

CPAO= central project administration office, DED= detailed engineering design, KPS=kabupaten pengelola sarana (kabupaten project secretariat), PCR=project completion report, PPIS= project preparation implementation and supervision, RWSS=rural water supply and sanitation, SPAR= subproject administration report

Appendix 8 43

44 Appendix 7

FINANCIAL AND ECONOMIC EVALUATION

A. Feasibility Study

1. Financial Analysis 1. The feasibility study assed four subprojects, in (i) Tambang in Riau Province; (ii) Sungai Baung with Aro, Singoan, and Olak in Jambi Province; (iii) Sungai Kakap in West Kalimantan Province; and (iv) Sungkai with Batu Balian and Pasar Lama in South Kalimantan Province. All four subprojects were piped schemes to be operated by regional water supply enterprises (PDAMs). The feasibility study assumed a tariff increase of 5% each year. A collection efficiency factor of 95% was adopted based on assumed successful institutional and community programs. The results of the financial and economic analysis of the feasibility studies are shown in Table A9.1.

Table A9.1: Feasibility Study Financial and Economic Analysis Scheme Type of

Scheme Constr.

(‘000 Rp)HC

(no.)PH

(no.)Av.tariff /

Customer (Rp1996)

FIRR EIRR

Sungai Baung Deep Aquifer 1,038 309 30 334 3.46 18.7Sungkai Spring. 1,118 231 15 401 -3.31 12.6Tambang River, WTP

15l/s 1,698 314 20 451 -10.40 11.2

Sungai Kakap Deep Aquifer, 15l/s

1,657 281 27 494 1.68 14.6

Av.= average, Constr.= construction, EIRR=economic internal rate of return, FIRR=financial internal rate of return, HC= house connection, PH=public hydrant, WTP= water treatment plant Source: PMDU Riau, PMDU Jambi, PMDU West Kalimantan, PMDU South Kalimantan

2. Economic Analysis

2. Economic Analysis was carried out by comparing cost and benefit streams. The cost streams included the financial costs (investment and operating) converted to economic costs by applying a factor of 85% and converting to current prices over 20 years. 3. Two sources of benefits were considered. The first was the resource cost reductions arising from savings in time, health benefits, and monetary benefits such as fuel costs for boiling water. The second benefit was the net increase in water consumption and was estimated as the consumers’ willingness to pay. B. Project Benefit Monitoring and Evaluation Report 4. The project benefit monitoring and evaluation (BME) report presented analysis of the financial benefits of the district administrative center (IKK) systems and economic benefits of the water and sanitation facilities constructed in lesser developed villages (DTs) and growth center villages (DPPs). Using an approach similar to that used in the feasibility studies an estimate of the FIRR and EIRR was calculated for 10 of the IKK schemes (Table A9.2). In these calculations, half of the unaccounted for water was deemed to be used beneficially. The return on these schemes is quite variable. While only 3 of the 10 schemes had positive FIRR values

44 Appendix 9

Appendix 7 39

and 4 had positive EIRR values, the results are generally similar to those of the feasibility study. The feasibility study recommended the raising of tariff rates and the post project results again emphasized the importance of this to sustain the project’s water supply systems. Example of typical PDAM income statements are include in tables A9.3 and A9.4 (pp. 49-50) for PDAM Tabalong and PDAM Tirta Sakti (Kerinci). 5. The BME also presented EIRR estimates for 37 DTs and DPPs. Actual construction costs and estimated operation and maintenance (O&M) costs (10% of construction costs) were deflated to current prices at the time of construction and converted to economic prices. Economic benefits were assumed to be equal to the willingness to pay, estimated as 4% of household income. This method yielded estimates of EIRR varying from.-85% to 260%.

Table A9.2 : Estimated FIRR and EIRR for 10 IKK Schemes Scheme Type of

Scheme Constr.

(‘000 Rp)HC

(no.)PH

(no.)Basic Tariff

(Rp/m3)

FIRR %

EIRR%

Bahorok WTP 10 l/s 625,341 300 8 465 -10.0 -5.8Betung Deep Well 5 l/s 392,620 98 8 370 -43.9 -41.8Darit WTP 2.5 l/s 231,013 170 7 235 -42.4 -21.7Indralaya WTP 10 l/s 1,021,979 929 11 649 11.4 14.9Koto Anau WTP 10 l/s 215,924 52 4 300 18.4 32.4Koto Petai WTP 10 l/s 723,036 849 2 350 2.3 13.9Kuala Begumit Deep Well 5 l/s 653,765 168 2 465 -2.6 -2.6Limaupit Spring 5 l/s 464,579 53 5 328 -33.5 -13.1Mata Payang WTP 10 l/s 540,219 138 7 465 -7.7 -0.7Palupuah WTP 10 l/s 512,106 14 2 465 -9.4 5.4EIRR=economic internal rate of return, FIRR=financial internal rate of return, HC=house connection, PH=public hydrant, WTP=water treatment plant Source: the Government’s PCR C. Economic Analysis of the Whole Project 6. Economic analysis of individual subprojects provides variable results and gives unclear indication of the economic value of the whole Project. A total project model was developed using data from the feasibility study, the project completion report, and the BME study. 7. It was assumed that the water and sanitation facilities constructed were all new systems and did not extend existing systems. Although the final year of the Project included the rehabilitation and expansion of some existing systems, it was assumed that all these systems were not operational at that time and so had no existing capacity and any previous expenditure was treated as sunk costs. For this reason it was also assumed that the “without project” situation is the same as the “before project” situation in the absence of any other interventions. 8. Project costs were converted to domestic currency rupiah for a base year of 2000. Domestic currency costs were standardized using the gross domestic product (GDP) deflator, while foreign currency costs were standardized using the World Bank (WB) manufactures unit value index, and converted at the average exchange rate for year 2000. Based on the data in the Government’s project completion report, total costs were distributed across the water supply and sanitation outputs, divided between the IKK, DPPs and DT systems, and increased by the project overhead costs, which included institutional and community programs. Ongoing O&M costs for piped systems providing house connections were calculated as a proportion of initial

Appendix 9 45

46 Appendix 7

construction costs using data from the sample of 13 IKK systems. For collected water, operational costs were considered to be the ongoing cost of collection of water plus an allowance for maintenance. As indicated in the feasibility studies, the economic life was deemed to be 20 years for piped systems and 7 years for nonpiped systems.

9. Financial prices were converted to economic prices using an exchange rate factor of 1.06, and a labor rate factor of 0.65 and taxes were assumed to be 10%. For construction, O&M 60% of nonlabor costs were considered to be traded and 40% to be nontraded. No impact on overall water resources is expected, so no allowance for opportunity cost was included. The only case where this might not be true is for pumped systems, but because of the rural context it is considered unlikely that this will affect the current analysis. The project environmental impact is considered to be benign to slightly positive and so there has been no consideration of environmental costs. 10. Project benefits were calculated in terms of incremental and nonincremental outputs of the scheme. The rural nature of the schemes means that the majority of users have access to a water source such as a river or irrigation system, or in many cases an existing spring or well. While in some cases dry season access to water is reduced, generally people in rural areas collect water rather than purchase from a vendor. For this reason the cost of existing supplies is predominantly valued in terms of time taken to collect water. Time to collect water is very variable but from the results of the feasibility studies it is assumed that households were spending around 1.25 hours per day collecting water. In addition, people carry washing to water sources and bathe in rivers. The BME report indicates for that for DTs and DPPs, water consumption rose from below 30 liters/ capita/ day (lcd) to 74 lcd for piped systems and to 60 lcd for nonpiped water supplies. The difference may be explained in terms of the water used at site for the nonpiped system. A reasonable assumption is that water used on site is around half the quantity collected, which means that effective use for both piped and nonpiped systems is around 75 lcd, but for nonpiped systems around 15 lcd is used at the source. 11. A range of technical options was used during the Project, including rain water collectors, hand pumps, shallow dug wells, deep wells, and piped systems ranging from simple gravity fed systems with public hydrants, to systems distributing pumped and treated water to house connections. To simplify analysis, these were reduced to two cases: (i) systems where consumers were still required to collect water, called “collected water;” and (ii) systems that delivered water to house connections, called “delivered water.” As this is a post project analysis, it was assumed that the technical option for each situation was the least cost option. 11. The demand for collected water (Table A9.5) was calculated in terms of increase in consumption proportional to reduced distance and time to collect. The results of the BME indicate that collection of water from these systems doubled and so it is reasonable to assume that the Project halved the distance and time to collect water. The nonincremental demand for water is the amount of water used prior to the Project, which is estimated to be 30 lcd collected and 15 lcd used at site. The incremental demand for collected water is estimated to be 30 lcd. 12. The demand for delivered water varied across the IKK, DPP, and DT systems because of factors such as supply continuity, price, and availability of convenient alternative sources. No clear relationships emerged but across all piped systems a value of around 75 lcd has been adopted as a representative estimate. The nonincremental demand is the same as that for collected water.

46 Appendix 9

Appendix 7 39

Table A9.5: Incremental and Nonincremental Demand Demand Collected Water

(lcd) Delivered Water

(lcd) Without Project In house use 30 30 At site use 15 15 Non Incremental Demand 45 45 With Project

In house use 60 75 At site use 15 0 Incremental Demand 30 30

lcd=liters/ capita/ day 13. The value of collected water (and water used at site) is based on the value of time spent on water collection. In 2000, minimum wage rates varied throughout the provinces of the project area, but were generally around Rp300,000 per month. There is a range of approaches for converting this to the value of time spent on domestic work, and for this analysis the financial cost was calculated from the minimum wage rate, as suggested in the Asian Development Bank (ADB) Handbook for the Economic Analysis of Water Supply. The economic cost was calculated by applying a shadow wage rate factor of 0.65. The feasibility study estimated that around 1.25 hours per day is spent on water collection. Assuming 8 hours of domestic work per day, the economic cost of water collection each year before the project is estimated to be Rp365,625 ($40.63 with $1=Rp9,000). Data from the BME shows that consumption doubled after the Project so it appears that people use saved time to collect more water, thus effectively lowering the price per meter cubic (m3). 14. The value of nonincremental water is calculated as the resource cost saving on the nonincremental supply of collected water and water used at site. Collected water costs Rp365,625 per year. The average household size is 5.9, and so the amount collected per year is 64.6 meter cubic (m3) making the cost of collection Rp5,660/m3. The value of water used at the site is deemed to be half that of water carried and has a value of Rp2,830/m3.. The weighted average value of nonincremental water for both delivery systems is Rp4,726/m3.. 15. The value of incremental water was estimated at the average demand price (Table A9.6), which was approximated by the average of current and future costs of water supply in financial prices for collected water, and in terms of the weighted average value of the tariff for delivered water..

Appendix 9 47

48 Appendix 7

Table A9.6: Value of Incremental and Nonincremental Demand Demand Collected Water

Rp/m3 Delivered Water

Rp/m3 Without project In house use 5,660 5,660 At site use 2,830 2,830 Value of non incremental water 4,726 4,726 With project

In house use 4,353 536 At site use 2,177 0 Value of incremental water 3,635 536

Rp/m3=rupiah per meter cubic 16. Sanitation benefits were based on assumed savings in health care costs, using the same approach as the feasibility studies. The amount assumed was 25% of the amount used in the feasibility studies, standardized to year 2000 rupiah and converted to economic costs. Only 25% of health cost saving was used as it was assumed that other health savings were attributable to water supply improvements. 17. Calculation of the EIRR yielded the results shown in Table A9.7 These results indicate the returns that would have occurred if project outputs were as reported in the Government’s project completion report. Table A9.8 shows the EIRR the Project assuming that the population benefiting from water supply facilities is reduced by 50% and the population benefiting from sanitation facilities is reduced by 80%.

48 Appendix 9

Appendix 7 39

Table A9.3: Income Statement for the Year Ended 31 December 2001

(PDAM Tirta Sakti, Kerinci, Jambi) (in million rupiah) 2001 2000 Change %Revenue - water sales 3,430 2,700 730 21- non water sales 923 831 92 10

Total 4,353 3,531 822 19 Expenses - Water Source 110 84 26 24- Operations 972 792 180 18- Transmission and distribution 1,320 1,538 (218) -17

Total 2,402 2,414 (12) -1 Gross Operating Margin 1,951 1,117 (834) 43 Administration 2,417 1,371 (1,046) 43 Net Operating Margin (466) (254) (212) 45 Other Income & Expenses - Other income 113 86 27 24- Other expenses (1) (1) 0 -16

Total 112 85 27 24 Net Profit/Loss (353) (169) 184 52PDAM= regional water supply enterprise Source: PDAM Tirta Sari, Jambi

Appendix 9 49

50 Appendix 7

Table A9.4 Income Statement

for the Year Ended 31 December 2001 (PDAM Tabalong, Tabalong, Kalimantan Selatan)

(in million rupiah) 2001 2000 Change %Revenue - water sales 1,175 1,019 156 13- non water sales 403 353 50 12

Total 1,578 1,372 206 13 Expenses - Water Source 516 404 112 22- Operations 356 304 52 15- Transmission and distribution 307 307 0 0

Total 1,179 1,015 164 14 Gross Operating Margin 399 357 42 11 Administration 572 546 26 5 Net Operating Margin (173) (189) 16 9 Other Income & Expenses - Other income 3 5 (2) -67- Other expenses (1) 0 (1) 100

Total 2 5 (3) -117 Net Profit/Loss (171) (184) 13 7.6PDAM = regional water supply enterprise Source: PDAM Tabalong

50 Appendix 9

Appendix 7 39

Table A9.7 Economic Internal Rate of Return for Reported Facilities Completed

Gross Economic Benefits Resource Costs Net Water Sanitation Water Sanitation Economic IKK DPP DT TOTAL IKK DPP DT TOTAL Benefits Year (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) 1996 0 0 0 0 0 33806 7848 18714 20021 80389 -803891997 10678 11444 51213 855 74190 126459 36888 103359 34646 301351 -2271621998 21356 22887 102426 1710 148379 72557 31907 106875 1591 212931 -645521999 32034 34331 153639 2564 222569 52583 34801 129173 6952 223509 -9412000 42712 45774 204852 3419 296758 98378 52964 187878 11051 350271 -535122001 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852002 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852003 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852004 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852005 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852006 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852007 53390 57218 256065 4274 370948 4497 38702 169263 0 212463 1584852008 53390 0 0 0 53390 4497 0 0 0 4497 488932009 53390 0 0 0 53390 4497 0 0 0 4497 488932010 53390 0 0 0 53390 4497 0 0 0 4497 488932011 53390 0 0 0 53390 4497 0 0 0 4497 488932012 53390 0 0 0 53390 4497 0 0 0 4497 488932013 53390 0 0 0 53390 4497 0 0 0 4497 488932014 53390 0 0 0 53390 4497 0 0 0 4497 488932015 53390 0 0 0 53390 4497 0 0 0 4497 488932016 53390 0 0 0 53390 4497 0 0 0 4497 488932017 53390 0 0 0 53390 4497 0 0 0 4497 488932018 53390 0 0 0 53390 4497 0 0 0 4497 488932019 53390 0 0 0 53390 4497 0 0 0 4497 488932020 53390 0 0 0 53390 4497 0 0 0 4497 48893 EIRR 18%DPP=growth center village, DT=lesser developed village, EIRR=economic internal rate of return, IKK=district administrative center

52 Appendix 7

Table A9.8 Economic Internal Rate of Return for 50% Reduced Population for Water and 80% Reduced for Sanitation

Gross Economic Benefits Resource Costs Net Water Sanitation Water Sanitation Economic IKK DPP DT TOTAL IKK DPP DT TOTAL Benefits Year (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) (Rp mill.) 1996 0 0 0 0 0 33806 7848 18714 20021 80389 -803891997 5339 5722 25607 399 37067 126459 33105 86432 34646 280642 -2435751998 10678 11444 51213 799 74133 72557 24340 73023 1591 171512 -973781999 16017 17165 76820 1198 111200 52583 23452 78394 6952 161381 -501812000 21356 22887 102426 1597 148267 98378 37831 120173 11051 267433 -1191662001 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182002 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182003 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182004 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182005 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182006 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182007 26695 28609 128033 1996 185333 4497 19786 84632 0 108915 764182008 26695 0 0 0 26695 4497 0 0 0 4497 221982009 26695 0 0 0 26695 4497 0 0 0 4497 221982010 26695 0 0 0 26695 4497 0 0 0 4497 221982011 26695 0 0 0 26695 4497 0 0 0 4497 221982012 26695 0 0 0 26695 4497 0 0 0 4497 221982013 26695 0 0 0 26695 4497 0 0 0 4497 221982014 26695 0 0 0 26695 4497 0 0 0 4497 221982015 26695 0 0 0 26695 4497 0 0 0 4497 221982016 26695 0 0 0 26695 4497 0 0 0 4497 221982017 26695 0 0 0 26695 4497 0 0 0 4497 221982018 26695 0 0 0 26695 4497 0 0 0 4497 221982019 26695 0 0 0 26695 4497 0 0 0 4497 221982020 26695 0 0 0 26695 4497 0 0 0 4497 22198 EIRR 4%DPP=growth center village, DT=lesser developed village, EIRR=economic internal rate of return, IKK=district administrative center

Appendix 10 53

PROJECT PERFORMANCE RATING

1. A technique of relative weighting of criteria values was used to extrapolate the results of project performance. Applying ADB’s project performance audit report (PPAR) rating methodology, the Project is less than successful. The assessments include project relevance, efficacy, efficiency, sustainability, and institutional development and other impacts. Table A10.1 provides the overall assessment of project performance. Tables A.10.2-A.10.4 describe the assessment of project relevance, efficacy, and efficiency. Table A.10.5 describes the assessment of project sustainability. Table A.10.6 describes the assessment of project institutional development and other impacts.

Table A10.1: Overall Assessment Criterion Weight Explanation of Performance Rating

Description Rating Valuea

A. Outcome Assessment

1. Relevance 20% Project objectives remained relevant throughout project implementation and when the project completion report was prepared. Project design lacked adequate detail of implementation processes and risk, and changes should have been made at midterm review but were not.

Relevant 2

2. Efficacy 25% While reported outputs exceeded the planned outputs, the project completion report mission concluded that actual outputs were substantially less, and that achievement of project outcomes was seriously impacted by implementation issues such as quality of construction, choice of technology and location, and institutional and community support. It is likely that this will result in a limited achievement of project goals.

Less efficacious

1

3. Efficiency 20% The project recorded low economic internal rate of return (EIRR) and the commercial parts of the project recorded low financial internal rate of returns (FIRRs). Generally the efficiency of project management and implementation, including recruitment of consultants and funding, impacted on the achievement of project outcomes.

Partly efficient

1

B. Sustainability 20% The main limitations to sustainability are high connection charges that limit demand, low tariffs that limit cash flow, and poor outcomes in location and choice of facilities and support of communities. On the other hand it is unlikely that the Government will allow regional water supply enterprises (PDAMs) to fail.

Less likely 1

C. Institutional Development and Other Impacts

15% Institutionally the Project had little impact. Social impact in villages where user groups have been sustained will be positive but this appears to be a minority of project villages.

Little 1

Overall Assessment

Less than successful

1.2

a Highly successful: rating >2.5 and no criteria <2; Successful: 1.6 ≤ rating ≤ 2.5 and no criteria <1; Less than successful: 0.6≤ rating ≤1.6 and number of criteria < 1 should not exceed 2; Unsuccessful: rating < 0.6

54 Appendix 10

Table A10.2: Assessment of Relevance

Criterion Explanation of Performance Rating Valuea 1. Relevance of project preparation

to project output at the time of approval

The project design was relevant to the intended outputs but failed to adequately address implementation process and the risks of implementation process failure

Relevant 2

2. Relevance of project output to achieve project goals and purposes at the time of approval

Increased access to water and sanitation facilities is widely accepted as highly relevant to the project goals of improving health and social welfare. The outputs related to institutional strengthening and community participation are less well understood and could have been more fully described in the project design.

Relevant 2

3. Priority in the context of the Government’s development strategy at the time of approval

Water and sanitation was a high priority within the Government’s development strategy but rural water and sanitation was accorded a lower priority than urban water. Although accorded a high priority, water supply and sanitation programs have failed to achieve intended results and lessons learned have not had a major impact on implementation results. This indicates a lower priority in real terms than that stated in planning documents.

Relevant 2

4. Priority in the context of ADB’s development strategy for Indonesia at the time of approval

Relevant but not highly relevant. Relevant 2

5. Priority in the context of the Government’s development strategy at the time of evaluation

Water and sanitation remains a priority within the Government’s program. Not change from approval.

Relevant 2

6. Priority in the context of ADB’s development strategy for Indonesia at the time of evaluation

ADB’s focus on poverty alleviation and social benefits has increased since the time of approval.

Highly relevant

3

7. Priority in the context of one or more of ADB’s strategic objectives at the time of evaluation

Relevant to several strategic objectives Highly relevant

3

8. Appropriate changes made at midterm review to make the project more relevant

No substantive changes were made despite clear evidence that they were required.

Irrelevant 0

Average Value and Rating Relevant 2 ADB=Asian Development Bank a highly relevant = 3, relevant = 2, partly relevant = 1, irrelevant = 0.

Appendix 10 55

Table A10.3: Assessment of Efficacy Criterion Explanation of Performance Rating Valuea 1. Achievement of most project

physical outcomes Due to changes in exchange rates during the Project the value of foreign inputs increased and consequently the number of facilities planned for construction was increased in 1998. The Government’s PCR states that more facilities were constructed than originally planned. However, field assessment indicates that reported outputs need to be assessed in terms of outcomes such as increased access to facilities. The effective number of facilities has been reduced by factors such as inappropriate choice of technology or locations resulted in little or no use; poor construction reducing facility life to close to zero; poor community participation and support resulting in lack of affordability of operation and maintenance or unwillingness to maintain facilities. Project outputs have been over reported and some facilities were never built.

Less efficacious

1

2. Achievement of most project intangible outcomes

The intangible outputs of the Project included institutional development and community strengthening. The effectiveness of these outputs is measured in terms of reduction of implementation risks. Clearly, these outputs have not generally produced the desired outcomes. Field assessment indicated that the results varied across the Project.

Less efficacious

1

3. The likelihood of project outcomes leading to project goals

Where project outcomes have been achieved as planned, it is likely that these outcomes will lead to achievement of the project goals. For the total Project, it is unlikely that sufficient outcomes have been achieved to contribute to achievement of project goals.

Less efficacious

1

Average Value and Rating

Less efficacious

1

PCR=project completion report a highly efficacious = 3, efficacious = 2, less efficacious = 1, inefficacious = 0.

56 Appendix 10

Table A10.4: Assessment of Efficiency

Criterion Explanation of Performance Rating Valuea Efficiency of Investment 1. EIRR > 12 percent The Project as planned had a likely EIRR of

around 18%. Reduced achievement of outcomes has resulted in an EIRR around 4%.

Inefficient 0

2. FIRR > weighted average cost of capital

While some IKK schemes will achieve FIRR’s greater than zero, most are likely to continue the record operational losses. This has been caused by failure to follow the project design criteria, such as choice of target population, high connection prices, and low tariffs.

Inefficient 0

Efficiency of Process 1. Manner of ADB’s internal

processing of the Project Despite concerns about project implementation during the Project, no major changes were made. Funds continued to be disbursed despite noncompliance with some loan covenants. Some NOLs were withheld when procurement issues were identified. However availability of funding was adequate.

Partly efficient

1

2. Organization and management of executing and implementing agencies

Major issues arose due to lack of coordination between agencies, including the construction of facilities before preparation work was completed.

Partly efficient

1

3. Effectiveness of project management

Project management focused on the management of inputs and did not adequately consider the achievement of outcomes. The Government’s PCR does not adequately report the outcomes of the project. The BME system did not collect useful information.

Partly efficient

1

4. Efficiency in recruiting consultants and of procurement

Poor timeliness in consultant inputs affected project implementation.

Partly efficient

1

5. Timely and adequate availability of counterpart funding

A difference in timing of funding to different agencies seems to have affected project implementation.

Partly efficient

1


Partly efficient

0.7

ADB=Asian Development Bank, BME=benefit monitoring and evaluation, EIRR=economic internal rate of return, FIRR=financial internal rate of return, NOL=no objection letter, IKK=district administrative center, PCR=project completion report. a highly efficient = 3, efficient = 2, partly efficient = 1, inefficient = 0.

Appendix 10 57

Table A10.5: Assessment of Sustainability

Criterion Explanation of Performance Rating Valuea 1. Availability of adequate and

effective demand for project services or products

Where facilities have been constructed in accordance with design criteria and institutional and community support has been adequate, demand should be adequate and effective. A proportion of IKK systems has been built in towns with insufficient populations to provide adequate demand, and in some cases high connection fees are limiting demand. Poor location, choice of technology, and community support for some nonpiped water facilities and sanitation facilities has also limited effective demand.

Less likely 1

2. Probable operating and financial performance of the operating entity and the ability to recover costs

The operating entities are the PDAMs and the village facility management groups. A number of PDAMs are performing adequately but unless tariffs rise or current financial support continues their financial performance will be affected. Facility management groups did not exist in some villages visited but in some cases the village leader has taken charge of operations.

Less likely 1

3. Probability of the existence of appropriate maintenance policies and procedures

PDAMs generally appear to be performing an adequate level of maintenance, although some are impacted by poor plant design and limited income. Where village facility management groups exist, or where the village leader has taken charge of maintenance, nonpiped systems are also being maintained.

Less likely 1

4. Probability of funds availability (cash flow) for continued operation, maintenance, and growth requirements

Currently, most small PDAMs appear to be supported by government assistance. Tariffs will need to rise to provide adequate cash flow.

Less likely 1

5. Probable continued availability of required skills

Additional programs are planned to raise the management and operation skill levels in PDAMs. Community skill levels will remain the same.

Most likely 3

6. Probable availability of appropriate technology and equipment to operate the project

Replacement of taps on public hydrants is a continuing issue but is being address through a number of community initiatives. PDAM plant operation is not likely to be affected by nonavailability of technology or supplies unless supply conditions deteriorate.

Less likely 1

7. Probable availability of enabling environment in which the Project is operating at the time of evaluation

The current operating environment will likely continue.

Less likely 1

8. Government ownership and commitment to the Project

PDAMs are being supported. Less likely 1

9. The extent to which the operations affect the environment and renewable or nonrenewable resources

Generally operation of water and sanitation facilities are relatively benign to the environment. In some cases water sources may be depleted due to over-extraction. Sludge disposal needs to be handled appropriately.

Likely 2

58 Appendix 10

10. The extent to which community participation and beneficiary incentives are adequate to maintain the project benefits

Support to some communities has been adequate. Generally for growth centers and disadvantaged villages, community involvement has included expanding and modifying systems that were inadequate or poorly constructed. Where facilities are well located and there is real demand, communities will continue to receive project benefits.

Less likely 1


Less likely

1.3

IKK=district administrative center a most likely = 3, likely = 2, less likely = 1, unlikely = 0.

Appendix 10 59

Table A10.6: Assessment of Institutional Development and Other Impacts

Criterion Explanation of Performance Rating Valuea Institutional Development 1. The country’s formal laws,

regulations, and procedures The Project has not impacted formal laws, regulations, or procedures. The opposite is true: the formal procedures have impacted on the intended implementation procedures of the Project.

Negligible 0

2. The people’s informal norms and practices

Some villages responded to the demonstration sanitation facilities and changed their sanitation practices.

Little 1

3. Organizational strengthening The Project has brought no change in organizations. Significant organization change has taken place within project agencies but none of this can be attributed to the Project. Some strengthening of kabupaten organizations has occurred from their greater participation in the last two years of the Project.

Little 1

4. Institutional skill levels and capacities

PDAM skill levels have been raised. Training has had some positive effects.

Moderate 2

5. Participatory attitudes of society Communities have responded to the Project where there is clearly perceived benefit, such as taking over operation of small piped systems or extending systems to allow house connections.

Moderate 2

6. Sector policy framework It is unlikely that the Project will have an impact of sector policy

Negligible 0

Other Development Impacts 1. Impacts on poverty Public facilities in disadvantaged villages will

have a positive effect on poverty. Better sanitation facilities and better health outcomes will be positive for poor families. High connection charges to piped systems limit the impact on poverty of IKK systems

Moderate 2

2. Impacts on the environment Properly operated sanitation facilities should have a net positive impact on local environments.

Moderate 2

3. Impacts on social organization Sustained operation of water user groups and facility management groups in some villages will have a positive impact on social organization.

Moderate 2

4. Impacts on political developments There have been no impacts on political development

Negligible 0


Little

1

IKK=district administrative center, PDAM=regional water supply enterprise a substantial = 3, moderate = 2, little = 1, negligible = 0.

60 Appendix 10

PROJECT BENEFITS 1. The Government project benefit monitoring and evaluation report analyzed project performance in terms of achievement of outcomes. Outcome targets were quantitatively expressed as benchmark values and for a sample of locations achievement against the benchmarks were assessed by surveying project beneficiaries. 2. The project objectives were expressed in terms of a hierarchy of desired service levels. The hierarchy is shown in figure A11.1.

Figure A11.1: Hierarchy of Project Objectives

1. To provide safe, adequate and reliable WSS services toselected low income communities through community basedarrangements, and 2. To support hygiene and sanitation education, increasingwater quality surveillance and community managementactivities in project areas

WATER SANITATION

SERVICE LEVEL

SERVICEIMPACT

USAGE LEVEL

SERVICELEVEL

SERVICEIMPACT

USAGE LEVEL

Population served

Coverage of PH

Coverage of HC

Communal latrines (CL)

Investment per person

Increase in toilet use

Consump- tion by PH

Distance to collect

No of family latrines (FL)

People using CL

Decrease in health costs

Consump- tion by HC

Time to collect

Populationserved

People using FL

Water borne diseases

Information distribution

Communiy awareness

Billing efficiency

Operation ration

PDAM revenue

Water borne disease

Continuity of supply

Average Consumpt.

60 Appendix 11

Appendix 10 61

3. A technique of relative weighting of parameter values was used to extrapolate the results of individual questions to higher objective levels. Performance was measure in terms of a benchmark value. The adopted benchmark levels are shown in Table A11.2

Figure A.11.2: Project Benchmark Level

Service Level Useage Level Service Impact A. Water Supply

No of people served

IKK DPP DT

7000 1200 600

Total consumption of house connection

90 Time to collect water <10mn

Service coverage of house connections

calc.

Total consumption of public hydrant

30 Distance to collect water

<

Service coverage of public hydrant

calc Average water consumption

calc Continutiy of supply 24 hrs

Water borne disease frequency

0%

PDAM revenue

calc

Operation Ratio

100%

Investment per capita: IKK

RWC Handpump Dug well

DPP pipe sys.

116,68775007500

20,00011,667 Billing efficiency

100%

B. Sanitation Population served

DPP DT

1200 600

No.of users of family latrines

10 Freqency of waterborne disease

0%

No. of family latrines constructed

50 No.of users of communal latrines

600 Descrease in cost for health

No.of communal latrines constructed

2 Decrease in non toilet use.

100%

Community awareness

100%

No.of people receiving information

100%

Investment per capita FL CL

School latrine

10020832083

CL=communal latrine, DPP=growth center village, DT=lesser developed vilage,

4. The results of the benefit anaysis are shown in Figures A11.3-11.14. These figures show the percent of locations where various levels of achievement were exceeded. These figures show that rarely were benchmark levels achieved and in most cases more than half of locations failed to achieve 50% of the target for the objective.

Appendix 11 61

62 Appendix 10

Figure A.11.4: DT Water Useage

-102030405060708090

100

0% 25% 50% 75% 100%

Level of target achieved

% o

f loc

atio

ns

Figure A.11.3: DT Water Service Level

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.5: DT Water Impact

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.6: DPP Water Service Level

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Appendix 10 63

Figure A.11.7: DPP Water Useage

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

nsFigure A.11.8: DPP Water Impact

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.9: IKK Water Service Level

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.10: IKK Water Useage

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

64 Appendix 10

Figure A.11.11: IKK Water Impact

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.12: Sanitation Service Level

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.13: Sanitation Useage

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Figure A.11.14: Sanitation Impact

-102030405060708090

100

0% 25% 50% 75% 100%


% o

f loc

atio

ns

Appendix 9 65

LESSONS LEARNED

Issue Lesson Recommendation 1. Organizational Structure of the Project a. Structure of the Central Project Administration Office Role of the central project administration office (CPAO) was to manage and coordinate the activities of all implementing agencies (Agency for National Planning (Bappenas), Directorate General of Communicable Disease Control and Environmental Health (DGCDCEH), Directorate General of Rural Community Development (DGRCD), Directorate General Rural Development (DGRD), Directorate General of Human Settlements (DGHS)]. Only staff from DGHS were located in the CPAO and the CPAO was located in DGHS. This limited the project authority of the CPAO and limited its coordination ability.

Coordination between agencies needs to be effective for project processes to be effective. A single agency project management unit is limited in its authority over other agencies, and its ability to act uniformly across project requirements. The project management entity needs to work closely with implementing agencies and coordinating agencies

For large projects, a separate project management entity should be established with an adequate and representative staffing structure. In multiagency projects, the project management entity should not be placed within any single agency and should be allowed to work uniformly across the project.

Two consultants with similar roles assisted the CPAO. This cause communication and coordination problems.

Unnecessary duplication of consulting services will limit the effectiveness of project implementation.

The project management entity should contract consultancy services as required and in accordance with procurement procedures agreed between the Asian Development Bank (ADB) and the Government.

The main focus of the CPAO was on management and reporting of finances, and this is clearly seen in the monthly and quarterly reporting which focuses on inputs such as contracts and expenditure. There was very limited output or outcome reporting.

Generally, the purpose of a project is to achieve the desired objectives at minimum cost. To do this, project management must be focused on attainment of outcomes, which are achieved through the production of the project outputs. At the same time financial management needs to be good enough to ensure that outcomes are achieved for minimum cost. If the monitoring systems of the project do not adequately report outcomes and outputs and adequately account for inputs against outcomes, the project will not

The Government and ADB ensure that project entities adopt monitoring and accounting procedures that adequately measure outcomes and outputs, and account for inputs against outputs and outcomes.

66 Appendix 9

LESSONS LEARNED

Issue Lesson Recommendation be efficacious or efficient.

b. Coordination Between Agencies Despite the best efforts of many individuals, coordination between agencies was a continuing issue throughout the project, causing delays and impacting on community motivation. Implementation schedules agreed at the beginning of the project were not complied with. Construction activities were started prior to preconstruction training because funds for training were made available to DGRCD 18 months after funds for construction works. Similarly, training for local government officials was delayed until the latter part of the second project year.

Project design needs to adequately describe proposed implementation processes so that project scheduling identifies the correct relationships between activities. Project management and control needs to ensure that the implementation processes described in the design are adhered to, and that precedent activities are completed before subsequent activities are commenced. Project entities need to be backed by sufficient authority to resolve identified scheduling issues.

ADB and the Government ensure that implementation processes are adequately described in project design, and that implementation agencies understand their responsibility to adhere to schedules. Project management entities should adopt management systems that ensure implementation proceeds according to schedule, including adherence to activity precedence.

c. Coordination Between Central and Local Offices Coordination between levels of government was affected by consistently late reporting from local government, causing monitoring to be less effective.

No project management system will work effectively if information is not available or is not timely.

Similarly, subproject appraisal reports (SPARs) and detailed engineering designs (DEDs) were prepared by national consultants contracted by DGHS, which created inefficiencies in the design process. This continued after decentralization. Decentralization was introduced by government in 1999. This changed the focus of implementation management from level I to level II government with disruptive impact because of the changes to project responsibility, changes to project staffing and the need to train new staff.

If design and implementation assistance is not contracted at the most appropriate level, unrealistic communication and coordination requirements are created. A major change in the project environment should have initiated a review of project implementation risk and identification of appropriate responses.

Design and implementation assistance should be contracted at the lowest level at which adequate skills exist and at which adequate governance capabilities exist. Project management entities should operate a risk management system and major changes in the project’s implementation environment should initiate risk reviews and responses. A risk management system should identify and monitor key risks associated with assumptions in the project logical framework. The risk response mechanism should be based on monitoring systems that provide information relevant to key risks.

Appendix 9 67

LESSONS LEARNED

Issue Lesson Recommendation The complexity of management and coordination arrangements and the added difficulties of adjusting to decentralization resulted in a number of issues of coordination between central and local levels:

• Handover of some district administrative center (IKK) systems to regional water supply enterprises (PDAMs) was delayed by many months, causing some to need rehabilitation of electro-mechanical components that had never been used, with cost of up to 20% of the original construction cost.

The project design included adequate focus on commissioning and handover procedures. These were not followed, with the result that project resources were required to rehabilitate unused facilities. Similarly, population criteria for IKK systems were not complied with, and as a result PDAMs saw many IKK systems as loss making burdens. Greater involvement of PDAMs may have reduced this problem.

Project commissioning and handover procedures should be adequately documented and agreed by commissioning and receiving entities before construction is commenced. Greater involvement in planning and design activities by PDAMs is required. Consideration should be given to supporting PDAMs as project clients, and allowing them to let contracts for the investigation, design, and implementation of facilities that they will be required to operate and maintain.

• Information flow between local government and consultants preparing SPARs and DEDs resulted in wrong decisions.

Local governments need to be actively involved in the selection of consultants and should be supported through training to develop the necessary skills to procure and manage consulting services.

Projects should include training of local government staff to allow them to adequately manage the procurement of consultants. To avoid problems with assessment of skill levels, a certification system is required to prequalify consultants.

• Rapid changes in staffing and poor selection of candidates resulted in training having reduced effectiveness, and this resulted in errors in tender documents and bid evaluation. In some cases ADB refused to issue no objection letters (NOLs) and project implementation was delayed.

Training was not undertaken with adequate manuals of project procedures, and project system documentation was not easily accessed by staffing administering project activities. Where staff turnover is high, training without adequate systems development and documentation will be ineffective.

Training in project procedures should be subsequent to the establishment and adoption of documented project procedures. Incoming staff should be trained by outgoing staff (or by supervisors) in the use of project procedures so the benefits of training are sustained.

d. Coordination Between Agencies and Consultants In the first year of the project, some of the SPARs and DEDs, which should have been

Scheduling integrity needs to be maintained and if necessary contract control procedures

Project management systems should include adequate contract control provisions, and

68 Appendix 9

LESSONS LEARNED

Issue Lesson Recommendation completed within the first 6 months, were delivered at around 10 months. Some project managers had already commenced work and SPARs and DEDs were revised to reflect works already completed. Delays in delivery of DEDs were frequent throughout the project. At the end of the project, works were abandoned because late delivery of DEDs left insufficient time for completion.

used to ensure that contract outputs are adequate and delivered on time.

these should be used where scheduling integrity is threatened by poor performance of consultants.

2. Capability of Local Government Officials a. Technical Capabilities It was intended that appropriate training would boost technical capabilities at the kabupaten level of government. However, frequent changes of project staff at the local level have limited the effectiveness of training. This resulted in a lack of local capacity to pick up errors in SPARs, DEDs and village actions plans (VAPs) related to poor location, poor technology choice, and poor designs.

Technical capabilities at the kabupaten level will probably never be permanently boosted through project training. Project training should be used to impart skills specifically needed for the project. Overall technical capability is a longerterm human resource development issue and should be addressed through adequate position description and hiring procedures, adequate salary levels, and use of consultants where appropriate.

Project training should not try to resolve issues such as lack of availability of qualified staff. Where longer-term human resource development is required, projects should include activities to improve recruitment procedures for permanent staff. Where increased skill levels are required on a short-term basis, agencies should make appropriate use of consultants.

b. Local Consultant Support Local consultants supported kabupaten project staff, and their role included endorsement of work progress for contract payments. This was done without sufficient site inspection to consider the quality of work completed. Funds for site supervision were insufficient.

Improved quality of project outputs will rely on better use of local consultants to inspect contractors’ work, and proper use of contract control procedures to ensure the adequacy of completed work. Use of consultants for contract management is not effective if insufficient time and resources are allowed for their work.

Project management systems should include contract administration procedures that ensure only work completed to specification is paid for. Budgets for local consultants should be sufficient to ensure adequate inspection of contractor’s work.

Appendix 9 69

LESSONS LEARNED

Issue Lesson Recommendation c. Contract Bidding and Evaluation Contract procurement at the local level resulted in (i) wrong decisions regarding the method for procurement [(international shopping (IS) versus local competitive bidding (LCB)], (ii) mistakes in bid requirements such as time for submission of bids, (iii) failure to award contracts to the lowest bidder, and (iv) awarding contracts to non compliant bidders.

Reasons why contract management issues occurred, range from insufficient contract administration skills through to possible corruption. Issues such as these should have been identified during project audits along with likely causes. Issues to do with inadequate skill levels should have been addressed through training and systems documentation.

Project audits should include investigation of procurement procedures and identification of cause of problems, along with appropriate recommendations for action. Preconstruction information and training for communities should clearly describe the quantity and quality of facilities that the community will receive, and they should be empowered to demand an adequate result.

d. Construction Supervision The responsibility for construction supervision lay with the public works department (DPU) Tk II, which did not have adequate skills or resources to provide adequate supervision.

A significant level of project resources was applied to the procurement of consultants for planning and design, but a large proportion of this was probably wasted due to inadequate provision for construction supervision.

Consulting services should be applied where short-term human resource shortages or inadequate skill levels are identified. The total amount of consulting services should be distributed across all areas of identified human resource deficiency.

3. Quality of Implementation a. Quality of Planning and Design The quality of planning resulted in (i) unreliable water sources, (ii) poor quality water sources, (ii) financially unsustainable systems due to lack of population, (iv) technology choices that are not the least cost option, (v) technology choices that could not be operated or maintained, or (vi) no prospect of cost recovery due to lack of affordability of chosen option.

Most of the identified issues were adequately described in the project design and occurred because the criteria and implementation processes of the design were not followed. The tendency of implementing agencies to vary the project design should be controlled within the project management system by procedures that explicitly describe and approved design variations.

Project management systems should include adequate procedures to control and approve design variations. Project implementing agencies should not have the sole authority to vary design criteria.

70 Appendix 9

LESSONS LEARNED

Issue Lesson Recommendation b. Choice of Technology Generally technology choice was constrained to a set of standard modules. Government national policy requires designers to use standard package plants for water treatment plans (WTPs) below 40 liter/second. While this maintains a higher standard for the completed WTP, it also results in some WTPs that are not suited to the specific local raw water characteristics. For lesser developed village (DT) and growth center village (DPP) standard modules were used and very little adaptation was undertaken. One exception to this was where standard modules were incompletely implemented because contract bid prices were inadequate.

The adoption of standard modules was intended to control costs and overcome skill deficiencies at the local level. While this was achieved in most cases, the use of standard designs is ineffective where (i) the standard design is inadequate, or (ii) local conditions fall outside the design parameters of the standard. The design consultants should have checked this. .

Design consultants who do not check the adequacy or appropriateness of standard designs for project application should be held accountable for losses incurred. Where facilities are required for applications that don’t match the standard design parameters, the project management system should include procedures for the adoption of non-standard designs.

c. Quality of Constructed Works Quality of constructed works suffered from (i) inadequate supervision, (ii) unskilled workers, and (iii) cost cutting due to financial pressures. Electric and mechanical components in IKK WTP are of adequate quality. Works carried out in DTs and DPPs by Swakelola (an implementation system method intended to involve communities in project implementation, whereas the government provide materials and training, and the community constructed the works) and local contractors lacked adequate supervision and are not to the national design standard but consistent with local practice. Cost cutting resulted in reduction in technical standards (i.e., reduced inputs, such as size and quality of inputs) and the scope of implementation (i.e., reduced outputs).

The project design included adequate consideration of issues related to the quality of constructed works but these were not implemented. Contract management procedures were not sufficiently robust to allow appropriate contract variations or to ensure delivery of contracted outputs. The use of community labor in a contract framework was not successful and is not consistent with best practice in community participation.

Project management systems should ensure that contract specifications are consistent with intended project outcomes. Project management systems should include contract administration systems that ensure that works are completed according to contract specifications. Community participation should not be reduced to the hire of community workers by contractors, but should focus on incorporation of community needs in project designs.

Appendix 9 71

LESSONS LEARNED

Issue Lesson Recommendation d. Quality of Service The outcome of the project is measured in terms of the quality of services and products delivered by the constructed systems. Due to the choice of water sources, the quality of raw water, and the operating characteristics of the pipe systems (e.g., not 24 hour pressure) it is unlikely that many of the systems will deliver “safe” water, and so do not reach the standard set in the project design.

The linkages between project activities and desired project outcomes were not strong enough and monitoring of outcome achievement was not sufficiently timely to allow modification of implementation processes.

Benefit monitoring and evaluation should be carried out throughout the project implementation period, and should be based on a logical framework that identifies the linkages between inputs, activities, outputs, and outcomes. Risk management should involve clear identification of project assumptions about the linkages between outputs and outcomes, and should be used to actively respond to inaccuracy or inadequacy in project assumptions. See also previous recommendations about project management systems and design variations.

4. Effectiveness of the Institutional and Community Support a. Effectiveness of Institutional Support The effectiveness of institutional support for the project is measured in terms of the efficacy of production of outputs. In the preceding sections a number of significant issues have been identified in the production of project outputs, and these indicate that institutional support was lacking. That these issues continued to occur throughout the Project despite significant training expenditures indicates the limited success of the training component. Poor choice of trainees, poor targeting of training, and constant reassignment of staff are some of the reasons for this failure.

Institutional support includes both formal and nonformal components. Formal regulatory changes were not included in the project design, but it was assumed that formal project management procedures would be adequate to achieve project outcomes. This proved not to be true. Informal aspects of agency performance should have been more adequately addressed in training. As a general observation across all agencies, where local officials were motivated, training was more successful and project outcomes were more positive. Motivational training may have improved achievement of project outcomes.

Future projects designs should identify key institutional constraints in addition to skill and training issues. These should include formal regulatory issues, formal management system issues, and informal attitudinal and behavioral issues.

72 Appendix 9

LESSONS LEARNED

Issue Lesson Recommendation Institutional activities were largely limited to training and did include systems development. Lack of strong and specific project management procedure development hindered achievement of project outcomes. A project of this size and complexity should have had much better project management.

b. Community Support The bottom-up approach of the project design relied on support to communities to allow them to participate in the planning phase of the Project. This can be measured in terms of the quality and relevance of the village action plans. Anecdotal evidence suggests that village action plans were commonly the work of agency staff and were delivered to communities. Similarly, where community support was provided through training, it sometimes occurred after construction.

Working with communities requires specific skills complementary but vastly different to the skills required for technical tasks. Use of specialists in community participation would enhances project delivery. Bottom-up planning for a project assumes that communities have a real voice and that voice is heard and acknowledged by authorities. The role of community advocacy is to make sure that a community’s voice is faithfully represented in project planning and implementation. This will only happen if adequate time and resources are allocated to community support.

A greater proportion of project resources should be allocated to ensuring sufficient levels of community support. Future projects should involve local community organizations that faithfully represent the interests and aspirations of the community.

Appendix 9 73

LESSONS LEARNED

Issue Lesson Recommendation c. Community Participation and Contributions No reports on the community management program were submitted by DGRCD community management program (PMD) during the 5 years of the Project. Community involvement within the Project did not reach the levels proposed in the project’s design. This resulted in (i) failure of the bottom-up planning approach; (ii) continuing issues in use of community labor; (iii) nonestablishment of users management groups (UPS) and facility user groups (KPS); and (iv) systems that cannot be maintained, operated or afforded. In the case of household sanitation facilities, the Project was to provide construction materials and the householders were to construct them. In some cases this was not done due to lack of interest or lack of material and resources.

Lack of monitoring of community participation resulted in the community support activities not contributing fully to the Project. Lack of a logical framework that identified the linkages between community participation and Project outcomes caused a lack of understanding of the importance of community participation. Poor levels of community participation had a direct impact on the performance of the Project.

Future projects should clearly identify the value of community participation, and monitoring systems should be established that ensure that community participation is reported as a critical project output.

d. Operation, Maintenance and Cost Recovery The major issue for PDAMs will be the extent to which they will need to subsidize IKK systems that are financially inefficient. Cases were identified where PDAMs had dropped such schemes, and in other cases IKKs had been incorporated into a single system to reduce fixed operating costs. For DPP and DT, ongoing operation seems to be heavily influenced by the interest and ability of the village leader.

PDAMs need to be closely involved in investigation and design decisions, which will impact on their future operating viability. Alternative management arrangements for piped systems for small communities are feasible if communities are sufficiently motivated and resourced.

PDAMs are the main client for larger piped water supply systems and should be directly involved in all planning and design decisions. Community alternatives to PDAMs should be considered for small community piped systems. Management systems for these alternatives should be developed in consultation with the community and should reflect true cost recovery.

Documents

ASIAN DEVELOPMENT BANK · 2014-09-29 · asian development bank pcr:ino 26314 project completion report on the rural water supply and sanitation sector project (loan 1352-ino) in