Technical Assistance Consultant’s Report · This consultant’s report does not necessarily reflect the views of ADB or the Government ... ECNEC Executive Committee of the National

This consultant’s report does not necessarily reflect the views of ADB or the Government concerned, and ADB and the Government cannot be held liable for its contents. (For project preparatory technical assistance: All the views expressed herein may not be incorporated into the proposed project’s design.

Technical Assistance Consultant’s Report Project Number: 39469 Jan 2010

Bangladesh: Strengthening Results-Based Monitoring and Evaluation Project (Special Fund Resources of Japan)

i

Table of Contents Abbreviations ...............................................................................................................................ii

1. Foreword ................................................................................................................................. 1

2. Technical Progress to Date ...................................................................................................... 2

Impact Level ................................................................................................................................ 2

Outcome Level............................................................................................................................. 4

Output 1 ...................................................................................................................................... 6

Output 2 ...................................................................................................................................... 9

Output 3 .................................................................................................................................... 11

Output 4 .................................................................................................................................... 15

Output 5 .................................................................................................................................... 18

Activities with Milestones ......................................................................................................... 19

Consultant Arrangements ......................................................................................................... 21

Appendix 1: Individual Consultant ToRs.................................................................................... 22

Appendix 2: List of Reference Documents ................................................................................ 28

Appendix 3: Recommendations for Follow up Support to IMED and FAPAD ........................... 29

Appendix 4: Model Audit Report............................................................................................... 40

ii

Abbreviations ADB Asian Development Bank AMC Annual Maintenance Contract AMS Audit Management System ASICT Assistance to SICT CPAM Computerisation of Project Accounting Manual CPTU Central Procurement Technical Unit DF Data Facilitator DMF Design and Monitoring Framework DP Development Partner DPP Development Project Proforma EA Executing Agency ECNEC Executive Committee of the National Economic Council FAPAD Foreign Aided Project Audit Directorate GoB Government of Bangladesh IAE International Audit Expert IMED Implementation Monitoring and Evaluation Division LAN Local Area Network LM Line Ministry M&E Monitoring and Evaluation MoLGRD&C Ministry of Local Government, Rural Development & Cooperatives MoPME Ministry of Primary and Mass Education MTBF Medium Term Budget Framework NAS National Audit Specialist NSAPR National Strategy for Accelerated Poverty Reduction PSC Project Steering Committee PMIS Project Management Information System PPR Public Procurement Regulation RBM Results Based Management RBME Results Based Monitoring and Evaluation SICT Support to ICT Task Force SP Strategic Plan SPPP Strengthening Portfolio Performance Project SWAp Sector Wide Approach TA Technical Assistance TL Team Leader UNDP United National Development Program WAN Wide Area Network

1

Foreword This Final Report is a contractual requirement and reports on project completion against the modified Design and Monitoring Framework (DMF). The DMF was modified under Variation Order 5 with the main changes being to the MIS Component (Output #3). These changes are explained in detail under “Output 3”. The original DMF is set out on pages 6 – 8 of the Technical Assistance Report (TAR 39469). The Final Report follows the submission of the Draft Final Report submitted in May 2009. The Draft Final Report was discussed in the Project Steering Committee meeting of 15 June 2009. In that meeting the following decisions were made:

A letter should be issued to FAPAD and IMED co-ordination by the project authority asking their grievance/opinions about the performance of consultants and quality of materials supplied. Agency/Firm responsible for supplying low quality of training materials, furniture, equipments to FAPAD requested to take necessary actions in this matter. A PIC may be organized to review the progress of works done by MIS consultants immediately. A proposal may be sent to ADB requesting to organize training courses on MIS user with the fund remaining utilized. A proposal may also be sent to ADB with recommendation to initiate a pilot project for sustaining the exisiting works done so far and to improve RBME system usable to all Agencies/Ministries

When the Draft Final Report was submitted there was one activity under the FAPAD component to be completed. This has since been completed and all activities are now complete.

2

Technical Progress to Date Results in terms of technical progress to date are assessed against the Design and Monitoring Framework (DMF) revised under Variation Order #5. Progress at the level of impact, outcome and output are discussed below. While impact level results might not be expected to be achieved by the end of the project it is nevertheless useful at this stage to assess the likelihood of results being accomplished at this level.

Impact Level Design Summary Performance

Indicators/Targets Data Sources/Monitoring Mechanisms

Assumptions and Risks

Improved accountability and enhanced performance in delivering development programs toward poverty reduction

Increase in transparency and accountability in development project management. RBM for outcome measurement widely practiced

Project audits, reports and reviews

Government continues to pursue RBM approach and recognizes the linkages between RBM and improved poverty reduction outcome of development projects Risks Project not implemented on schedule

ANTICIPATED PROJECT IMPACT The impact of the RBME project largely depends on the design and implementation of a second phase of the project. The RBME Project has focused on the development and agreement of a Five-year Strategic Plan for IMED based on RBM principles. However, RBME was never tasked with implementing the Strategic Plan. While some aspects of the plan have been piloted, a full roll out of the strategy is envisaged in the follow up project. The moderate successes of the pilot initiatives indicate that if the Plan were successfully implemented then there would certainly be an “Increase in transparency and accountability in development project management” and RBM would be an integral part of the management of the investment portfolio in terms of measurement against development outcomes or results. For example, if DMFs (or Logframes) were used rigorously in project management (as specified in the IMED Strategic Plan), then there would be clear and measureable relationship between inputs and project outcomes. Moreover, with specific measurable

3

outcomes project directors and executing agencies would be held more accountable for delivering results.

RECOMMENDATIONS TO IMED TO MAXIMISE EARLY IMPACT This brief section suggests what IMED can do to increase the impact of the project between the end of this project and the indicative beginning of the follow on project (possibly in July 2010). IMED had a high degree of ownership of the RBME project during its first year (mid-2007 to mid-2008) culminating in the formal approval of the IMED Strategic Plan in August 2008 by the then Finance and Planning Advisor. The degree of ownership declined to some degrees in the second half of the project indicated by the limited response of IMED to the Strategic Plan pilot initiatives developed by the consultants, and by the disappointing attendance by IMED officers in the RBME Training Programme (around 50%). The impact of this project largely depends on the success of the follow on project and in turn the success of the follow on project depends on IMED’s commitment to implementing the Strategic Plan. In this respect IMED could review the Strategic Plan, the Recommendations for Follow on Support to IMED and FAPAD and decide how the next project could be effectively managed and owned by GoB. The IMED should consider the effectiveness of the project management arrangements under RBME and how these could be improved. Therefore the role and function of the Project Director, The Deputy Project Director, Project Steering Committee and the Technical Working Committees should be reviewed. With this suggestion, the TA consultants are not making any judgements about the effectiveness of these arrangements but merely recommending that they be reviewed so that project management arrangements under the potential follow on TA can sustain - throughout the follow on project - the excellent levels of management and ownership that existed under the first half of RBME.

4

Outcome Level Design Summary

Performance Indicators/Targets

Data Sources/Monitoring Mechanisms


Improve organizational capacity of IMED and FAPAD through qualitative changes in their structure, functions, reporting relationships, and monitoring, evaluation, and audit capacity based on the principles of RBM

Changes in IMED locus, functions and structure; IMED MIS expanded and FAPAD MIS established; and capacity developed among IMED, FAPAD, and selected line ministry/EA staff; M&E capacity in IMED, and audit capacity of FAPAD enhanced

Capacity assessment reports

Prevalence of political will to undertake change, continuous dialogue with stakeholders throughout the process, staff motivation in IMED and FAPAD maintained, and consultants’ performance satisfactory Risks Absence of timely Government decisions

END OF PROJECT STATUS At the end of the project there is reasonable indication of sustainable changes in IMED and FAPAD. Many of the changes required to move towards an RBM approach to M&E are outlined in the IMED Strategic Plan. This Plan was approved by the immediate past Finance and Planning Advisor in August 2008. The Plan was originally intended to come on-line in July 2008 and run to June 2013. However, IMED preferred to pilot a limited number of the initiatives set out in the Plan and seek further support from ADB for a full roll out of the Plan. Full implementation of the Plan is therefore more likely to begin in July 2010 (The ADB has given an initial indication that further support to IMED and FAPAD is likely to start up at that time). Consequently, through approval of the Plan, qualitative changes in structure, function, reporting relations and M&E have been embedded in Government policy and capacity in that sense has improved. On the other hand since a full roll out of the Plan has not begun then these capacities have not had a substantial operational effect. In FAPAD, there has been less of an emphasis on RBM or performance based auditing, as the early Capacity Development and Training Plan for FAPAD noted that without a development of fundamental audit capacity a results based approach to auditing (performance auditing) would not be possible. Consequently organizational capacity has

5

been built in FAPAD through strategic, operational and practical support to the Directorate. The FAPAD MIS has been fully established by setting up a network of PCs with server and the development of the FAPAD Audit Management System to a stage where audit managers will be able to use it to track the progress of project audits. The IMED PMIS has been expanded by adding additional input forms and reports according to IMED needs. (More details of this are included under Output 3) Extensive training programmes for both IMED and FAPAD and LM and EAs have been completed for over 250 officials. (More details of training for IMED and FAPAD are included under Output 2 and Output 4)

REVIEW OF ASSUMPTIONS AND RISKS Despite changes in the consultant personnel, the team has continued to carry out the necessary analyses, present concise and relevant reports on time, engage with stakeholders and maintain a good relationship with IMED and FAPAD. IMED have commented on the performance of the consultants indicating that the absence of the Deputy Team Leader from January 2008 and the earlier replacement of two other national consultants has hampered progress. While IMED’s concerns and perceptions are well taken there was no substantial evidence that this has been true. Nevertheless, the fact that IMED carefully monitored consultant inputs represents a high degree of ownership of the project and a laudable interest in value for money. Government decision making has generally been timely. FAPAD has been particularly proactive in making use of the limited resources available under this project. IMED made rapid progress on the consultation process (known as the “Road Map”) to deliver the IMED Strategic Plan culminating in the endorsement of the Plan by the Project Steering Committee (PSC) and ultimately by the Finance and Planning Advisor. This represents a very significant achievement for IMED. Even prior to the approval of the IMED Strategic Plan, IMED made thoughtful decisions regarding selection of the pilot initiatives from the Strategic Plan. However, once the pilot initiatives (detailed under Output 1) had been completed, IMED were less decisive in putting these into practice preferring to wait for further TA support for full implementation of the Strategic Plan.

6

Output 1 Design Summary Performance



An agreed-upon strategic plan for IMED containing a well defined institutional development action package in relation to its functions, staffing, structure, appropriate locus, and reporting relationships

IMED strategic plan and M&E manual prepared; strategy for wider “whole of government engagement with IMED delineated; best practice examples understood by the stakeholders; and M&E frameworks established

Reports and TA review missions

Government decisions are made on time Line ministries/EAs continue to engage with IMED Sustained interest from development partners Ministries and EAs continue to support changes

PROGRESS TO DATE The Five-year Strategic Plan (SP) was agreed within IMED, agreed by the RBME Project Steering Committee and ultimately officially approved by the then Finance and Planning Advisor in August 2008. Divided into short, medium and long-term goals to be accomplished with five years, each goal is supported by specific objectives. Institutional development involving functions, staffing, structure, locus (areas of authority and influence) and reporting relations are implied within the goals themselves. The Strategic Plan states, Short-term goals focus on building internal capacity to define project outcomes and impacts, to collect and verify information efficiently and, most importantly, to use this information as the basis of analysis. In brief this period of the strategy is about measurement and analysis; it is concerned with establishing IMED’s value and unique contribution within Government. Setting goals and measuring progress towards those goals are the foundation for a results based approach to monitoring and evaluation. Building on but not replacing earlier goals, the medium-term goals center around using organizational capacity in measurement and analysis to coordinate resource allocation and support policy making. By this stage IMED should have established enough credibility to coordinate the creation of a national set of performance indicators that take into consideration the Medium-Term Budgetary Framework (MTBF), the National Strategy for Accelerated Poverty Reduction (NSAPR) and the Annual Development Plan (ADP) sector plans. This agreed set of national indicators represents the fundamental step towards RBM.

7

At the same time IMED will continue to build more advanced skills in large scale evaluation. This will undoubtedly require external assistance and will also involve the creation of a national consultative committee on evaluation drawn from across government, academia and civil society. Expanding its reporting brief to even higher levels and to more exacting standards, IMED will be able to comment on the results of public sector investment. In the longer term IMED will respond to a greater public demand for information on public sector performance. It is anticipated that citizens will hold government accountable for performance and IMED will play a key supporting role in providing information to the public in a clear and transparent manner. The results based approach needs to be institutionalized through the passing of appropriate legislature and the reform of laws that hamper public access to information. Institutional changes are implicit within the specific objectives of the plan and focus more on functions and less on structure and staffing arrangements. There is a strong emphasis on an expanded locus and a greater focus on “value-added” in reporting relationships in the plan. Specifically in the short-term the SP indicates that the Evaluation Wing of IMED will extend its function start to review all the DMFs (Logframes) of projects presented to the Planning Commission for review. Moreover, it is proposed that IMED will be required to sign off on all DMF (Logframe) indicators before Development Project Proformas (DPPs) can be approved. These are subtle yet substantial changes in the locus and function of the IMED. The IMED SP implies many other changes in locus and thus in the reporting relationships with other parts of Government. For example it is proposed that in the medium-term IMED play a coordinating role between the Finance Division, the Planning Commission and line ministries to prepare and ensure a common set of government-wide results. IMED Strategic Plan therefore explains how IMED proposes to engage with “whole of government”. In conclusion, the Plan refines and expands the M&E function of IMED and extends its area of authority and influence to project design, coordination of results based initiatives across government and to government policy on public sector performance. That is to say the Plan calls for a “whole of Government engagement”. Structure is addressed not by suggesting different configurations of IMED Sectors or directorates but through the rearrangement of reporting functions. Nevertheless, the Plan does contain a proposal for IMED field offices in each of the Divisions (although initially this would be piloted in just one division). An M&E Manual for IMED was drafted and presented to IMED in March 2009. This Manual is not intended to be a comprehensive manual on IMED project monitoring and evaluation but rather contain the key elements of a results based approach to monitoring and evaluation in keeping with the overall ambitions of the RBME project. The Manual therefore pulls together and compiles the pilot initiatives of RBME. These include i) the M&E Frameworks for the Primary and Mass Education, and the Local Government Sectors, ii) the Sector Performance Report Formats based on those Frameworks, iii) the Project Rating System and Guidelines for Output to Outcome

8

Reviews and iv) Guidelines for Project Impact Assessment. Finally, the way these processes fit together to form a complete information system is suggested in Information Flows for Effective RBME. Separate and extensive documentation of M&E best practices in four countries, (including strategies, structures, functions, methods, report formats and guidelines) was collated and extensively analysed and debated by stakeholders as an input into the strategic planning process in January and February 2008. This material remains available as a continuing resource to IMED and other stakeholders.

REVIEW OF ASSUMPTIONS AND RISKS Overall assumptions have held true. Government decision making was more punctual in the first half of the project with the approval of the IMED Strategic Plan in August 2008. In the latter half of the project IMED has been slower to approve some of the pilot initiatives of the project. For example, even though Results Based Reporting Formats have been developed for the Primary and Mass Education Sector and the Local Government Sectors, IMED has not decided to use these formats in any way. Line ministries are reasonably engaged with IMED as evidenced by feedback from several ministries on IMED SP. The two key ministries (MoPME and MoLGRD&C) were very engaged in the development of the M&E Frameworks for the pilot sectors. Efforts to further engage line ministries were taken by redesigning the RBME Training Programme to include a significant number of trainees from line ministries. Development Partners’ (DP) interest is moderate to high with three key DPs (DFID, EC and UNDP) attending a focus group discussion on IMED SP in late 2007. Cooperation with the SIDA supported RBME project in the DPE was productive and led to the development of the IMED Results Based Reporting Format in line with the SIDA project’s Annual Performance Report. Regular contact has also been kept with the DANIDA Agricultural Support project component based in the Planning Commission as they develop their M&E Framework for the Agriculture Sector.

WORK TO BE COMPLETED (IF ANY) None

9




A capacity needs assessment and implementation of a capacity development and training plan that will support the implementation of the IMED strategic plan

Capacity development and training plans developed and implemented on time Line ministries, EAs and other stakeholders becoming more involved in M&E and RBM; IMED acting as mentor and partner

Reports and TA review mission

Ministries and EAs continue to support changes Staff is available for training Staff is eager to learn new techniques in M&E and understand RBM

PROGRESS TO DATE A capacity needs assessment carried out through interviews, workshops and review of work samples (e.g. monitoring reports) began in August and September 2007. It was necessary to make progress on the strategic planning process before a Capacity Development and Training Plan could be drafted. That is to say it was essential to understand the capacities required by IMED to implement the Strategic Plan and incorporate these in the Training Plan. Accordingly a Draft Capacity Building and Training Plan (IMED) was submitted on 1 January 2008. (The Project Director accepted the draft and did not request any changes so this document became the final approved Capacity Building and Training Plan). Initially it was expected that the Training Program, as defined in section 6 of the Capacity Building and Training Plan (IMED), would start in Mid-2008. However, a number of factors led to delivery of the program between January and March 2009. First, the IMED Strategic Plan was not formally approved by the then Finance and Planning Advisor until August 2008. This added legitimacy to the RBME Training Program as it was now derived from official Government policy. Second, in order to further involve LMs and EAs it was decided to redesign and extend the program to more participants. The program was amended from three batches of a nine-day training course for IMED, to nine-batches of a 4-day training course for IMED, LMs and EAs. Thus the targeted number of participants grew from around 60 to around 180. The decision to train more people in the same time meant that course contents were prioritised so that the most critical training objectives from the originally planned nine-day program were retained in a revised four-day course. During this expanded training programme IMED officers were to act a resource persons led by the TA consultants to deliver training to line ministries and agencies. Consequently, a short mentoring/training of trainers course was planned for up to eight IMED officers to take place between the IMED batches and the ministry batches to enable the IMED resource people to co-train

10

the RBME Training Program. These co-trainers could have potentially become a permanent IMED resource for the entire Government for additional RBME training leading to IMED acting as a mentor and partner to LMs and EAs in the design of project M&E. To fund this expanded programme underutilized funds of $30,000 from the equipment budget line were transferred to training. This was proposed in Variation Order #5 and was approved by ADB on 8 November 2008. Therefore by the end of 2008 the RBME Training Programme had policy support from GoB and funding support from ADB. As with any training program a number of difficulties arose during implementation. The main concerns were the attendance level of the IMED officers in the first three batches and the availability of IMED officers to act as co-trainers for ministry batches. In fact only 27 IMED officers participated in the training when around 50 were expected. Competing work tasks, overall inadequate staffing levels and the demands of the new government were all factors in preventing trainees from participating in the full four-day course. Sadly, the proposed co-trainers could not spare time to receive the dedicated “training of trainers” course that would have equipped them with the skills to be competent RBME co-trainers. Nevertheless, four IMED officers were highly engaged (even though for limited amount of time) in the training for the ministries. In total eight batches of training were delivered with 129 participants. There were 47 trainees from IMED and the remaining 82 were drawn from across government ministries and agencies. The seventh and eighth batches were taught simultaneously with the eighth being trained by three of the graduates of the mentoring/training of trainers programme. The training conducted by the graduates was a very significant achievement for IMED.

REVIEW OF ASSUMPTIONS AND RISKS The assumptions held true to different degrees. The assumption that “Ministries and EAs continue to support changes” was valid. To some extent this was manifested in the near 100% attendance levels of trainees in the batches designed for LMs and EAs. In fact these participants were far more engaged in the classroom, and seemed to be more technically informed about their respective projects. The assumption that “Staff is available for training” did not hold up entirely for IMED. As written above only 27 out of a possible 50 IMED officers were able to attend the training and even these were sometimes called away from the training for “important” tasks by their immediate supervisors. The assumption that “Staff is eager to learn new techniques in M&E and understand RBM” is still valid but they were sometimes sceptical about the level of support they would get to really assert their legitimate role within ministry and agency projects.


11




3. Enhancement of the IMED MIS within the framework of the Government’s comprehensive e-governance strategy

Consensus reached on MIS expansion to support implementation of IMED strategic plan and to establish linkages with other stakeholders A conceptual and functional design for upgrading existing MIS through web interfaces with other automated systems in the government (such as CPTU) prepared on time Modifications to PMIS’s computer program made to produce required output reports

TA review missions Implementation of MIS expansion documented Personnel trained and capacity enhanced

Government decisions on MIS expansion and line ministry connections are made on time Personnel are available to handle mentoring functions DPs collaborate in the process of change management in MIS. Availability of funds for equipment procurement and operations and maintenance from the Government of Bangladesh and DPs Availability of maintenance support Government involvement in implementation

PROGRESS TO DATE As fully explained, and approved, in both the Inception and Interim reports, there have been substantial changes to the approach taken for Output 3 from the project’s original design. These changes were primarily due to external factors that arose prior to the start of the project and during its execution. At the project’s inception there was an incompatibility between the existing client server architecture in the PMIS system and the Web-based approach planned in the project’s

12

original design. As such, the original idea of simply migrating the application to the Internet was impractical given the resources available. Furthermore, the consultants discovered that a UNDP-funded project, ASICT, also had a mandate to develop a completely new MIS for IMED. In evaluating the feasibility of establishing “electronic links” to other targeted agencies selected by IMED, the consultants determined that only one of the four organisations had an automated system capable of establishing such a link. Even then, the link was not sensible given the current status and problems of PMIS. Because of ASICT’s plans, it was agreed that the RBME consultants would work closely with the ASICT to delineate responsibilities and to ensure the efforts of both projects dovetailed with each other. A key agreement, documented with signed minutes of a meeting between RBME, ASICT and IMED representatives, was that RBME would focus on design aspects, while ASICT would procure a vendor to implement the design; RMBE would later verify that the system met the design requirements. A timeline for key events was also agreed. The RBME project met the agreement with a conceptual design presented to the third meeting of the Technical Committee on 5 December 2007. The RBME consultants also assisted ASICT by the developing a Terms of Reference for their tendering process. However, by the mid-point of the project it became apparent that ASICT was falling behind schedule to such an extent that the new MIS for IMED could not be completed during the RBME project period. According to the original timeline, work on the new system was to begin by March 2008. However, the vendor, Domain Technologies Ltd., was not selected until November 2008 and initiated work in December 2008. Furthermore, the PMIS is only one of many systems that fall within ASICT’s mandate to develop for the Planning Commission. By the MIS Expert’s second review mission in July/August 2008, the initial plans had to be re-evaluated. As such, it was agreed by both the IMED and ADB to focus RBME resources on “upgrading” the PMIS by fixing existing problems in the current PMIS and to develop new output reports as immediately required by IMED. This effort remained in line with ASICT’s work as the modification and repairs would eventually be adopted in the new system. On the other hand, the modifications also provided valuable insurance in case ASICT’s system failed to meet IMED’s needs. The consultant’s Terms of Reference for the Output 3 were modified to reflect these changes. The revised ToR was included and approved in the project’s Interim Report/Variation Order #5. A series of workshops/meetings were then conducted by the MIS Experts to determine the modification and repairs required. These were documented and approved by IMED during a meeting with the MIS Technical Committee on 3 August 2009. It was decided during the meeting that Ms. Rashida Khatun, Director, Co-ordination & MIS Sector (IMED) would oversee the consultants’ work. She subsequently appointed Ms. Halima Khatun, Data Entry/Control Supervisor (IMED), to work with the consultants and provide necessary guidance.

13

The project also arranged, at the request of IMED, to deliver comprehensive training of key officers in modules of Visual Basic, SQL Server, and Crystal Reports. This training was conducted by throughout the third quarter of 2008. The aspects of the training are documented in the project’s Interim Report. The PMIS repairs and upgrade were undertaken by the project’s national MIS expert, Mr. Md. Ahmedur Rahman Talukder, from November 2008 through April 2009. A complete list of modifications and new reports is provided in the table below. These modifications were reviewed during the third mission of the International MIS Planning Expert. Database 1. Developed diagram and Relations New Modified Tables 2. Sectoral_Block_Allocation Modified Store Procedure 3. rpt_Monthly_Ministry_Progress Modified Forms 4. Sectoral Block Allocation Progress Modified 5. Project Information Entry Screen Modified 6. Monthly Ministry Wise Progress Modified 7. Updated Monthly Ministry wise progress Report Form Modified 8. Sectoral Block Allocation Progress Modified Reports 9. Ministry Division Wise Expenditure Statement and Project Competition Status

Modified

10. Ministry/Division wise Expenditure Statement and Project Completion Status Report

Modified

11. Updated Comparative Statement on Monthly ADP Implementation Progress

Modified

12. Monthly Ministry Wise Progress Report Modified Modifications made by RBME to the IMED PMIS The MIS Experts also developed an updated version of the PMIS’s Technical Manual. The manual has been updated to reflect the new environment and changes made since the original SPPP project. It is intended for IMED MIS staff, and provides guidance on how to maintain and support the system. The revised PMIS Technical Manual is available on request. On 5 April 2009 the consultants met with ASICT’s National Project Coordinator, Mr. Shaheeduddin Ahmad, to review the progress and plans of ASICT and to discuss follow-on support from the RBME project. The ASICT expect to complete the revised PMIS in July 2009, and then to provide additional institutionalization support, including training, for another year. The software vendor will also provide a warranty and technical support for two more years.

REVIEW OF ASSUMPTIONS AND RISKS It appears that only some of the assumptions for this output have held true.

14

The IMED has been clear about its needs or intentions for the IMED MIS expansion evidenced by of the discussions and decisions made in several RBME MIS Technical Committee meetings. However, it is not clear if any hard connections between ministries are being made by other projects. While the UNDP/ASICT project was cooperative in terms of the acceptance of the RBME conceptual design UNDP/ASICT moved according to its own timeline and fell increasingly behind the RBME schedule. Consequently, ASICT did not establish an effective web-based MIS within the timeframe of the RBME project. Ultimately RBME and ASICT were able to cooperate but not fully collaborate because ASICT was subject to the procurement regulations of UNDP. Similarly without the concurrent implementation of RBME and ASICT, funds for procurement of a new MIS for IMED were delayed (in terms of the RBME timeframe).


15




Implement a plan for enhancing FAPAD’s audit management

Based on review and appraisal of earlier support to FAPAD, capacity development and training plans developed and implemented; and PC-based audit tools developed, and PC network initiated in FAPAD and CAG for better audit management

Reports and TA review mission Stakeholders’ involvement in audits and overall performance of the audit process demonstrated

Government decisions are made on time Personnel are available to handle mentoring Staff members are available for training Staff members are eager to learn new techniques

PROGRESS TO DATE A comprehensive review of earlier support to FAPAD was undertaken and a Capacity Building and Training Plan (FAPAD) presented and agreed by FAPAD in December 2007. The Capacity Building and Training Plan was comprehensive and included the following elements:

Training and development in project audit management Development of the Audit Management System (AMS) Establishment of a Local Area Network to support the use of the AMS

The development of capacity in project audit management covered planning, execution and reporting. The methods of capacity development included formal classroom training and coaching and support of pilot audits designed to demonstrate new learning. As a first step in RBME reviewed the new Project Audit Manual (PAM) of December 2007. Although the PAM included operational guidelines there were a number of issues that had not been fully explained. For example, the development of a sector level operational plan and audit follow-up activities are not described in the PAM. Detailed explanations on such issues were developed and presented to FAPAD in Guidelines for Project Audit under the Project Audit Manual (Septermber 2008). Subsequently a 5-day training programme on PAM was conducted in July 2008 and March/April 2009 in five batches for a total of 155 FAPAD officials. The training topics were selected to meet the need of FAPAD after extensive consultation and included subjects not covered in the PAM such as project formulation, approval and revision processes, basic concepts of civil works, GoB project fund release process, ADB and

16

WB project financial procedures and PPR-2008,. The Directors and Deputy Directors of FAPAD were selected as resource persons for PAM to better enable them to understand and put into practice the new method of the PAM. Further details of the training are included in Report on Initial Two Batches of Training for FAPAD Officials on Project Audit Manual. The next critical intervention in the development of project audit capacity involved the execution of pilot audits. To this end a model audit plan and a model audit report (format) was developed in December 2008. Using these models two pilot audit were conducted with FAPAD audit teams to strengthen capacity and validate the new PAM. Field work for the first pilot audit on “Rural Development Project: Greater Noakhali & Chittagong Districts” was completed in December 2008/January 2009 with the FAPAD audit teams. The International Audit Management Expert and the National Audit Specialist worked with these audit teams. The second pilot audit on “Nishargo Support Project” was conducted in April 2009 with support from the NAS. The audit report on “Nishargo Support Project” has been submitted to FAPAD and presented on 30 April 2009. The audit report for “Rural Development Project: Greater Noakhali & Chittagong Districts” was submitted on 7 September 2009 and is appended to this report. As well as developing skills and knowledge in good audit practice capacity development for FAPAD has focused on IT including audit management software and supporting hardware. The Foreign Aided Project Audit Monitoring System (FAPAMS) software was developed under the CPAM Project in 2006. This was designed to meet all the requirements of project monitoring and follow up of audit activities. This project was predicated on the proper functioning of the Audit Monitoring System (AMS). This software would be at the heart of a PC-based audit tool. However, as highlighted in the FAPAD Capacity Building Report the AMS had never been tested. Extensive testing in 2007 by RBME consultants revealed substantial flaws in the software rendering the AMS inoperable. FAPAD with support from RBME successfully re-engaged the AMS vendor to repair or provide a functioning version of the AMS. To date the vendor has made substantial progress and the AMS is now operational. A detailed technical manual for AMS operation was developed by the CPAM project but it did not describe how the AMS would be integrated into FAPAD’s existing management structure such as specific responsibilities of each sector/section, procedure of ensuring data entry, monitoring the quality of data entry, management of data, security of data, purposes of each report. In short the manual did not explain how FAPAD could fully benefit from use of the AMS. Consequently, under the RBME Guidelines for Foreign Aided Project AMS (April 2009) were developed. To ensure the continued functioning of the AMS FAPAD had to enter into an Annual Maintenance Contract (AMC) with the vendor. Therefore a draft AMC was developed by the National Audit Specialist and presented to FAPAD. FAPAD is currently in negotiations with the vendor. FAPAD also officially requested from the RBME Project Director funding for the first year of an AMC from the GoB portion of the TA.

17

To further embed the AMS in FAPAD a “Help Desk” facility was established under RBME consisting of five Data Facilitators (DF) engaged initially for 3 months. The DFs entered data from past and ongoing audits, solve the day-to-day problems faced by the FAPAD officials in entering data and ensured data quality. They also provided one-to-one support to five Sectors/Sections to ensure FAPAD made full use of the AMS facilities. Overall responsibility for the AMS was handed over in April 2009 to an “AMS Champion” (an officer appointed by FAPAD to oversee the AMS and other computer related work). He will be responsible for and switching on and off the system each day, solving day-to-day problems, regular data back-up, cleaning viruses, repair and maintenance of hardware, monitoring data quality, ensuring report generation and use, weekly/fortnight data entry progress reporting to DG, and training on FAPAMS for new officers/staff. This substantive input will fully institutionalise and integrate the AMS into FAPAD. The design of a LAN to support the AMS and other office functions was completed and included in the FAPAD Capacity Building Plan. Consequently, a server, several PCs and other network hardware were purchased and set-up in March 2009. At this stage the network is fully functional.

REVIEW OF ASSUMPTIONS AND RISKS FAPAD have been very supportive of the RBME project. This is indicated by participation of FAPAD officers in the RBME workshops and meetings, timely feedback on the draft Capacity Building and Training Report and the formation of the FAPAD Technical Committee for RBME. FAPAD officers participated fully in the Training Programme on the Project Audit Manual both as trainees and resource people. In this respect the assumptions have held true. Assumptions regarding the status of existing AMS software and hardware should have been included in the DMF.


18




Recommendations prepared for subsequent support (investment and/or TA as required) to IMED and FAPAD

Recommendations for subsequent support, in phases as appropriate, prepared based on an assessment of the achievement and impact of outputs 1–4

Project final report Political will on role of IMED in M&E of PRS, and FAPAD for financial and performance audits, continues

PROGRESS TO DATE Details of Progress on this output are in Appendix 3: Recommendations for Follow up Support to IMED and FAPAD – Proposals for RBME2

19

Activities with Milestones 1. Improve the organizational capacity of IMED

Status

1.1. Strategic plan for IMED prepared (within 3 months from mobilization), and approved by the Government (within 6 months from mobilization)

IMED SP drafted in December 2007 (4.5 months from mobilization), IMED approved SP in second PSC meeting in February 2008 (6.5 months from mobilisation). Finance and Planning Advisor approved on 18 August 2008 (13 months from mobilisation). Completed.

1.2. Capacity development and training plans supporting the strategic plan for IMED prepared (within 3 months from mobilization), and approved by the Government (within 6 months from mobilization);

Capacity Development and Training Plan for IMED drafted in January 2008 (5.5 months from mobilisation). Delay resulted from need to sequence Capacity Development Plan with IMED SP. This was specified in Inception Report. Government are satisfied with Plan. Completed.

1.3. Capacity development and training plans for IMED implemented (within 22 months from mobilization);

Training for IMED started in January 2008 and continued to March 2009 (20 months from mobilisation). Majority of training delivered from January 2009 to March 2009 Completed.

1.4. Consensus reached on MIS expansion to support IMED strategic plan and establish linkages with other stakeholders (within 6 months from mobilization); and

Initial consensus reached on proposed new web-based system designed (conceptually) by RBME to be implemented by ASICT in December 2007 (5 months after mobilization) in 3rd meeting of Technical Committee on MIS. IMED keen to repair existing PMIS established under SPPP. Consensus again reached in 5th meeting of TC on alternate plan to improve existing PMIS in August 2008 (12 months after mobilization). Completed (ie consensus reached).

1.5. MIS expansion to support IMED strategic plan and linkages with other stakeholders established and functioning (within 18 months from mobilization).

MIS component was reformulated. MIS has been enhanced rather than expanded. Enhanced is defined as production of new reports, repair of database and modification of key input forms. This enhanced PMIS will be basis of expanded system to be developed by ASICT (according to ASICT timeframe). Milestone completed in April 2009 (21.5 months from mobilization) Completed.

2. Improve the organizational capacity of FAPAD

Status

20

2.1. Capacity development and training plans for FAPAD prepared(within 2 months from mobilization), and approved by the Government (within 4 months from mobilization);

Capacity development and Training Plan for FAPAD prepared by November 2007 (4 months from mobilization) and approved by FAPAD in December 2007 (5 months from mobilization). Completed.

2.2. Capacity development and training plans for FAPAD implemented (within 22 months from mobilization)

FAPAD training plan began in July 2008 and continued to April 2009. (21.5 months from mobilization) Completed.

2.3. PC-based audit tools developed and used, and PC network established and used by key managers of FAPAD and CAG (within12 months from mobilization).

The AMS is now functioning; PCs and server have been purchased under GoB component and LAN established (20 months after mobilisation). Getting key managers to use network and AMS has been handed over to AMS Champion assigned by FAPAD (21 months after mobilisation). Completed.

3. Future Support Status 3.1. A plan with recommendation for subsequent support (investment and/or TA as required) to IMED and FAPAD (within 22 months from mobilisation)

“Recommendations and Plan for a Second Phase of Support in RBME to IMED and FAPAD” submitted in March 2009 (20 months from mobilization) Completed.

21

Consultant Arrangements

CHANGES IN CONSULTANTS TO DATE Before the inception of the project the International Team Leader was replaced at the request of the consultants citing extreme personal circumstances of the original team leader. The replacement Team Leader is Mr Phillip Choudhury. In November 2007 the National MIS and Training Specialist and the National Audit Specialist were changed at the request of the consultants. The replacements were Mr Ahmed Zafor Sadeq Bhuiyan and Mr Shamsuzzaman respectively. Again in November 2008 Mr Ahmed Zafor Sadeq Bhuiyan was replaced by Mr Ahmedur Rahman. In January 2008 the Deputy Team Leader retired from the position. The remaining time of the DTL was reassigned to the TL after agreement with IMED and ADB. In October 2008 the International Audit Management Expert retired from the position for reasons of ill-health and was replaced by Dr Sahabub Alam.

UTILISATION OF CONSULTANTS

Consultant Total MM Used MM Remaining MM

Team Leader 10.25 10.25 0.00

Capacity Development Expert 5.00 5.00 0.00

MIS Planning Expert 2.00 1.60 0.40

Audit Management Expert 2.25 2.25 0.00

Deputy Team Leader 4.50 4.50 0.00

Capacity Development Specialist 8.00 8.00 0.00

MIS Specialist 17.00 17.00 0.00

Audit Specialist 16.00 16.00 0.00

National

International

Position at October 2009

22

Appendix 1: Individual Consultant ToRs Each of the ToRs of the individual consultants has undergone some small modifications. These are detailed in Variation Orders #4 and #5. Therefore, progress against these revised ToRs is given here. In the following tables:

The first column, “Activity” is drawn from the revised ToR The second column, “Int/Nat” describes whether the national or international consultant has responsibility for this activity (or both) The fourth column, “Reference” states the document where this can be verified or more details found.

23

TOR FOR TEAM LEADER (INT) AND DEPUTY TEAM LEADER (NAT): OUTPUT 1 AND OUTPUT 5 1. Activity 2. Int/Nat 3. Status 4. Reference (i) Lead the consultant team; I Complete (ii) Conduct, in conjunction with the team, a series of participatory strategic planning workshops involving IMED staff and other stakeholders;

I/N Complete Draft Final IMED Strategic Plan

(iii) Review all relevant reports including recommendations of the Strengthening Project Portfolio Performance (SPPP) Workshop of January 2006 and IMED Road Map prepared in 2004;

I/N Complete Inception Report

(iv) Lead the review and appraisal of past support to the Foreign Aided Projects Audit Directorate (FAPAD), and support the development of modern audit management tools for FAPAD;

I Complete Capacity Development and Training Plan (FAPAD)

(v) In line with the outcome of the participatory workshops, prepare, involving IMED counterparts and project team, a strategic plan document delineating short-,medium- and long-term actions for IMED, which will form the basis of a specific divisional plan for IMED;

I Complete Final IMED Strategic Plan

(vi) Collect and disseminate best practices on monitoring and evaluation (M&E) and results-based management (RBM) from the region and globally (such as India and Malaysia) through participatory workshops that will serve as inputs to the development of the IMED reform recommendations;

I Complete Global Practices in M&E (4 Models: Vietnam, Thailand, India, Malaysia)

(vii) Mentor IMED staff in the preparation of strategic plans as a capacity development activity;

I Complete Presentations/Coachingsessions on “Organizational Strategy” and “SWOT Analysis”

(viii) Following the series of workshops and in conjunction with the national institutional strengthening, strategic planning and result-based management specialist, recommend an appropriate reform package delineating IMED’s structure, functions, mandate, locus, reporting relationship, and potential linkages with line ministries and EAs;

I/N Complete Final IMED Strategic Plan

(ix) Consult various development partners and other stakeholders regularly on the project activities and seek suggestions for inclusion in the recommendations for IMED reforms;

I/N Complete Presentation to DPs

(x) Maintain regular communication with the Asian Development Bank’s Bangladesh Resident Mission;

I Complete

(xi) In conjunction with the national institutional strengthening and result-based management specialist, prepare a plan for implementing reforms;

I Complete Final IMED Strategic Plan

(xii) Conduct high-level workshops with regard to IMED reforms and I Complete Presentation to Seminar at

24

1. Activity 2. Int/Nat 3. Status 4. Reference strategic plan; Dhaka Sheraton in February

2008 (xiii) Design, through a participatory capacity development exercise, two sets of result based M&E framework relevant to social sector and infrastructure projects that can measure outcomes and impacts, as well as support the monitoring of the Millennium Development Goals and poverty reduction strategy;

I Complete IMED Draft M&E Manual

(xiv) Advise and provide mentoring to IMED on change and result management with regard to IMED’s reforms as and when required;

I/N Complete

(xv) Prepare reports in conjunction with the consultant team; I/N Complete Final Report (xvi) Prepare a draft capacity development and training plan based on a review of previous training, the absorptive capacity of IMED and the skills and knowledge required to implement the IMED Strategic Plan;

I/N Complete Capacity Building and Training Plan (IMED)

(xvii) Draft guidelines for conducting output to outcome reviews; I Complete IMED Draft M&E Manual (xviii) Draft guidelines for project impact evaluations; I Complete IMED Draft M&E Manual (xix) Draft a project rating system; I Complete IMED Draft M&E Manual (xx) Design a special report to ECNEC; I Complete IMED Draft M&E Manual (xxi) Review of information needs for effective results-based project monitoring at different levels of the government, including IMED, and suggest measures to establish an adequate information flow;

I Complete IMED Draft M&E Manual

(xxii) Draft recommendations for subsequent support (investment and/or TA as required) to IMED and FAPAD.

I Complete Recommendations and Plan for a Second Phase of Support in RBME

25

TOR FOR CAPACITY DEVELOPMENT EXPERT (INT) AND CAPACITY DEVELOPMENT AND TRAINING SPECIALIST (NAT): OUTPUT 2 1. Activity 2. Int/Nat 3. Status 4. Reference (i) Design and conduct a series of capacity development and training activities in M&E, RBM, change management, and other relevant areas;

I/N Complete Output 2 Completion Report

(ii) Assess the appropriateness of twinning arrangements with external agencies overseas and establishment of a special training and research unit within IMED;


(iii) Identify areas of training and conduct special workshops on result-based M&E that draw upon current global best practices;


(iv) Based on capacity development and training plans for IMED, prepare training curriculum for line ministry and department staff

I/N Complete RBME Workbook

(v) Develop appropriate mentoring tools for IMED and line ministry staff. I/N Complete Output 2 Completion Report (vi) Assist the Team Leader in collecting and disseminate best practices on M&E and RBM from the region and globally through workshops that will serve as inputs to the development of the IMED reform recommendations

I/N Complete Global Practices in M&E (4 Models: Vietnam, Thailand, India, Malaysia)

(vii) Assist the Team Leader in conducting a series of participatory Strategic Planning workshops involving IMED staff and other stakeholders

I/N Complete Draft Final IMED Strategic Plan

26

TOR FOR MIS PLANNING EXPERT (INT) AND MIS/TRAINING SPECIALIST (NAT): OUTPUT 3 1. Activity 2. Int/Nat 3. Status 4. Reference (i) Conduct participatory workshops involving IMED and other stakeholders, such as the line ministries, relevant EAs, development partners, and ICT Task Force

I/N Complete First Mission Report of MIS Planning Expert

(ii) Review all relevant reports including recommendations of SPPP and support to ICT Task Force Program Project (SICT) so as to avoid duplication of efforts and repetition of earlier activities;


(iii) Prepare an MIS Operation and Expansion Plan in close conjunction with the government counterparts and development partners and the ICT Task Force;


(iv) Prepare a conceptual and functional design for upgrading the existing MISs;

I/N Complete Interim Report (Appendix 8)

(v) Review required hardware and network set-up for FAPAD’s proposed computerized audit monitoring system, and propose upgrading, if required;

I/N Complete Capacity Building and Training Plan (FAPAD)

(vi) Develop training materials for the upgraded MIS; and I/N Complete Interim Report (Appendix 10)

(vii) Help procure necessary hardware and software to upgrade the MIS and networks (Note: Most of the funding for MIS hardware and software will be provided by development partners and the Government of Bangladesh; procurement under ADB funding will be identified by the MIS planning expert).

I/N Complete Tender Schedule

(viii) Make necessary modifications to PMIS’s computer program to produce the output reports required by IMED.

I/N Complete Final Report

(ix) Write a technical manual that explains the upgraded PMIS and provides detailed instructions on maintaining the computer program. The manual will include an entity-relationship diagram illustrating the new database structure.

I/N Complete PMIS Technical Manual

(x) Provide recommendations, if any, for improving the current PMIS including detailed descriptions of suggested functions and modifications to be considered in future versions of the system.

I/N Complete Recommendations and Plan for a Second Phase of Support in RBME

27

TOR FOR AUDIT MANAGEMENT EXPERT (INT) AND AUDIT SPECIALIST (NAT): OUTPUT 4 1. Activity 2. Int/Nat 3. Status 4. Reference (i) Review the impact of all project support to FAPAD, including the SPPP, and the FAPAD component of the World Bank-supported Computerization of Project Accounts Manual, Development of Asset Management, and Reforming Project Audit Methodology Project;


(ii) Conduct trial pilot audits with the FAPAD audit team to strengthen capacity, validate and, if necessary, propose revisions to the audit manual;

I/N Complete Local Inspection Report on the Accounts of Nishorgo Support Project Local Inspection Report on the Accounts of Rural Development Project: Greater Noakhali & Chittagong Districts

(iii) Prepare guidelines in applying the revised audit manual; I/N Complete Guidelines for Project Audit under PAM

(iv) Prepare recommendations regarding FAPAD’s audit focus for foreign-aided projects, develop a model audit report and a model audit plan;

I/N Complete Model Audit Plan Model Audit Report (Format)

(v) Support the development of the capacity development and training plans; and


(vi) Implement the capacity development and training plans to enhance FAPAD staff capacity in a range of areas required to support the audit process.

I/N Complete Report on Initial Two Batches of Training for FAPAD Officials on Project Audit Manual

(vii) Institutionalise the use of AMS in FAPAD including advising senior management

N Complete

(viii) Negotiate and draft an Annual Maintenance Contract with AMS vendor N Complete Draft Annual Maintenance Contract

(ix) Develop operations manual for use of AMS N Complete Guidelines for Foreign Aided Project Audit Management System

(x) Mentor AMS administrator and “champion”. N Complete FAPAD office order for taking over the charge of AMS.

(xi) Monitor data quality of new projects entered into AMS N Complete FAPAD office order for taking over the charge of AMS

28

Appendix 2: List of Reference Documents

PREVIOUSLY SUBMITTED The following documents have been previously submitted Inception Report Strategic Plan Interim Report IMED Draft M&E Manual Global Practices in M&E in four volumes Capacity Building and Training Plan IMED Capacity Building and Training Plan FAPAD

AVAILABLE ON REQUEST The following documents are available on request: RBME [Training] Workbook PMIS Technical Manual Guidelines for Project Audit under PAM Draft Annual Maintenance Contract [for Audit Management System] Guidelines for Foreign Aided Project Audit Management System

29

Appendix 3: Recommendations for Follow up Support to IMED and FAPAD – Proposals for RBME2

Introduction This document constitutes the recommended design for a further TA project to the IMED and FAPAD. These recommendations represent one of the outputs of the Results Based Monitoring and Evaluation Project (RBME, ADB TA 4880). This version of the recommendations is best seen as a project design that supports the implementation of the goals and objectives of the IMED Strategic Plan that was completed and approved under RBME. This document should be discussed at the next meeting of the Project Steering Committee. This document starts with a brief description of past and current ADB TA to IMED and FAPAD and goes on to briefly assess the achievements of this current support. Based on this assessment a further TA is recommended to further support the implementation of the IMED Strategic Plan and the further capacity development of FAPAD. The duration of this TA could be between 18 and 24 months and therefore of similar duration to the current phase of support. However, the exact duration of the project and the financial and human resources required are to be decided.

Strengthening Project Portfolio Performance The proposed follow up project to RBME would be the third TA to the IMED and FAPAD. ADB financed the TA for Strengthening Project Portfolio Performance (SPPP, TA 3336-BAN), which was implemented from 12 December 1999 to 31 August 2006. The SPPP addressed the implementation weaknesses in IMED and FAPAD, specifically of ADB’s portfolio of projects in Bangladesh. Executed by IMED of the Ministry of Planning, the SPPP focused on three critical areas (i) streamlining managerial and technical capacities in IMED, (ii) establishing an operational portfolio performance monitoring system through a management information system (MIS), and (iii) strengthening the project accounts management system and audit capacity of FAPAD.

Results Based Monitoring and Evaluation ADB, under the Japan Special Fund, has financed the Results Based Monitoring and Evaluation (RBME) Project which began on 15 July 2007 and will complete on 15 July 2009. As with the preceding SPPP, RBME addressed capacity issues in IMED and FAPAD but rather than just focusing on ADB projects sought to improve the monitoring and evaluation of all development projects. Again executed by IMED, RBME has five outputs: (i) the development of a strategic plan for IMED, (ii) design and implementation of a capacity development plan for IMED, (iii) the further development of the Project Management Information System (PMIS) set up under SPPP, (iv) a plan for enhancing FAPAD’s audit capacity, and (v) recommendations for a further period of support. Overall the intended outcome of RBME was to introduce a results based approach to monitoring and evaluation.

Brief Assessment of Achievements of RBME The recommendations for subsequent support are based on a quick assessment of achievements of the current phase of support. These achievements are described below and a fuller analysis of project results will be included in the Final Report.

30

IMED Strategic Plan (RBME Output 1) A Five-year Strategic Plan for IMED was developed through extensive consultation with internal and external stakeholders. The Plan consisting of ten goals each with supporting specific objectives was approved in June 2008 by the then Finance and Planning Advisor. The Plan is a comprehensive scheme to transform IMED from an organization primarily concerned with measuring project inputs to one focused on monitoring and evaluating projects against results specified in project planning documents. Under RBME some of the activities implied by the short-term objectives were piloted. These pilot initiatives included the development of M&E Frameworks for two sectors (Primary and Mass Education, and Local Government) and the drafting of sector performance report formats for these two sectors. The Frameworks were developed in conjunction with representatives from the ministries and agencies of the two sectors. These initiatives were seen as successful by the participants but IMED, however, has yet to formally issue the Sector Performance Report Formats to the respective ministries. Other pilot initiatives included the development of a Project Rating System and Guidelines for Output to Outcome Reviews. These were drafted and submitted to IMED for discussion but no comments have been received from IMED at time of writing. Despite the initiatives having varied levels of uptake, the RBME Project Director suggested that a follow up project be formulated to fully support the short and possibly medium-term goals and objectives of the IMED Strategic Plan.

Capacity Development and Training (RBME Output 2) A four-day training course in RBME was delivered in eight batches to IMED officers and officers from the Planning Commission, Planning Division, line ministries and agencies. The training course focused on the development of accurate project logframes and their use a tool for monitoring and evaluation. The strength of the course was the examination of real projects in which the participants were currently or had been involved in. These real examples were worked through in the classroom with participants using the principles taught to review and improve their project logframes. The success of this training programme was demonstrated in the clear improvements in the quality of the logframes judged against the guidelines provided by the trainers.

MIS (RBME Output 3) While it was initially expected that RBME would work closely with the UNDP funded ASICT project to develop a new MIS for IMED, it soon became apparent that the two projects could not be synchronised with ASICT’s timeline being constantly extended. It was therefore decided to repair the existing IMED PMIS and upgrade it. In this case “upgrade” involved developing new reports and reformatting existing reports according to IMED needs, developing new input screens to gather the necessary data for these reports and ensuring the generation of existing reports by repairing or debugging broken elements of the database. Good progress has been made with the development of revised and new input forms and the completion of the output reports will be finished by the end of March 2009. Achievements also include the delivery of a comprehensive training on MIS for key officers and staff members of IMED consisting of modules in Visual Basic, SQL Server and Crystal Report. This has given key staff an understanding of the function of a large, complex project database.

31

FAPAD (RBME Output 4) Under the RBME project initiatives to strengthen FAPAD’s audit capacity included (i) developing model audit plans; (ii) carrying out pilot audits based on these audit plans; (iii) producing model audit reports; (iv) developing guidelines for the use of the new FAPAD Audit Manual; and (v) the repair and development of an Audit Management System (AMS) to plan and keep track of audits and personnel assigned to these audits. Model Audit Plans and Model Audit Report Formats have been developed and used during a Pilot Audit led by the international and national consultants. Guidelines for use of the new FAPAD Audit Manual have been developed and a training program to explain the Guidelines has been delivered to two batches of auditors with a further three batches to take place in March 2009. The AMS has been repaired and developed and real project data has been entered to ensure that it functions as intended. The AMS still has a few minor glitches but is functional. FAPAD will have set up a LAN so that the AMS can be a management tool but FAPAD has not been provided with the resources to enter current and past project data into the system. Building on these pilot initiatives it is now recommended that FAPAD incorporate these approaches across all audit work through extended training, coaching and on-the-job support in modern audit practice. Moreover, partially integrating FAPAD’s work with IMED would provide for a wider range of information on which to draw conclusions about project progress. Such integration is limited to sharing of information on projects and joint inspections for large projects. The audit reports produced by FAPAD and the monitoring reports produced by IMED would continue, however, to be entirely separate documents.

32

The Recommended Technical Assistance

Impact and Outcome The outcome of the recommended TA is a results based approach to project auditing, monitoring and evaluation. Specifically by the end of the project IMED & FAPAD are auditing, monitoring and evaluating projects against results specified in DMFs (Logframes), sector M&E frameworks and the National Strategy for Accelerated Poverty Reduction (NSAPR). Ultimately this approach will lead to a clearer understanding of their relationship between project inputs and eventual results. The impact is therefore the capacity to verify the cost benefit analyses developed during the project formulation phase. Consequently the Government will have a more precise understanding of its returns on investment.

Methodology and Key Activities Outputs 1 to 5 below will provide IMED with a coherent scheme for (i) identifying results, (ii) providing support to the design of monitoring and evaluation frameworks, (iii) capturing planning and monitoring information in an electronic database, (iv) planning and executing project inspections and evaluations and (v) reporting progress against results to policy makers. Output 6 will provide FAPAD with enhanced audit management capacity by partially integrating its inspections with IMED’s monitoring work.

• Output 1: Design and Monitoring Frameworks (Logical Frameworks) are Formulated for all ADP Projects

• Activities:

• Write guidelines on formulation of Design and Monitoring Frameworks (to include sample frameworks for at least two sectors). GoB undertakes to issue manual to all ministries.

• Draft an order instructing that all DPPs must include Design and Monitoring Frameworks with indicators to the standards specified in the guidelines. GoB undertakes to enforce compliance with guidelines.

• Provide on-the-job support to Evaluation Wing and relevant IMED sectors to review all new Design and Monitoring Framework indicators. GoB undertakes to give written feedback on indicators to Project Evaluation Committees.

• Draft order requiring IMED to sign off on Design and Monitoring Framework indicators before DPP can be processed. GoB undertakes to issue and enforce order.

• • Output 2: Sector M&E Frameworks designed for 12 key sectors •

Activities:

• investigate and document current roles and responsibilities and processes for sector monitoring and evaluation.

• Recommend appropriate delineation of roles and responsibilities between executing agencies, line ministries, IMED, Planning Commission, Finance Division and

33

Development Partners in the processes of project and programme monitoring and evaluation. GoB undertakes to negotiate and agree clear roles and responsibilities between key government actors.

• Hold participatory workshops to draft sector performance measures for key sectors. GoB undertakes to approve sector frameworks for ADP, NSAPR and MTBF monitoring.

• Draft a report format for ministries to report against sector performance on an annual basis to IMED. GoB undertakes to approve formats and issue order requiring ministries to complete and submit formats.

• Output 3: Results Based MIS1 •

Activities: • Restructure IMED PMIS to include quantitative and qualitative fields on project results

(outcome and impact) as defined in Design and Monitoring Framework. • Design and develop specific output reports to provide results-based monitoring

information to IMED management. • Provide training to IMED on the use of the reports. • Provide additional training to both IMED and executing agencies on PMIS’s RBME-

related features to ensure that all users understand the requirements of the system. • • Output 4: Results Based Monitoring and Evaluation guidelines followed and plans

implemented

1 A number of institutional and technical activities have also been suggested for the IMED PMIS and these are detailed below

34

Activities: • Draft annual plan for selected output to outcome reviews (detailed project monitoring).

GoB undertakes to allocate human and financial resources to implement plan. • Conduct pilot project output to outcome reviews (detailed project monitoring) conducted

with IMED officers. • Conduct pilot project impact evaluations with IMED officers. • Design quality assurance system for in-house and outsourced reviews and evaluations.

GoB undertakes to review system and implement if approved.

• Output 5: Model Performance Reports for NEC, ECNEC Activities:

• Revise, develop and calibrate project rating system (drafted under RBME) that will allow ranking of projects and aggregation of project performance to an overall portfolio performance for any sector or ministry. GoB undertakes to review system and implement if approved.

• Draft Sector/Ministry Project Portfolio Performance Reports for four sectors. GoB undertakes to formulate Sector/Ministry Project Portfolio Performance Reports for remaining sectors.

• Analyse contribution of Annual Development Programme (ADP) to National Strategy for Accelerated Poverty Reduction (NSAPR). GoB undertakes to submit analysis to NEC/ECNEC for review.

• Output 6: FAPAD Audit Capacity Enhanced

Activities:

• Draft an annual audit plan using the AMS application. • Plan selected joint audit and monitoring inspections with IMED using model audit plans

developed under RBME • Undertake six pilot audits and coach FAPAD auditors to write audit reports in line with

model audit report formats developed under RBME • Incorporate model audit plans and reports into Guidelines for Audit Manual

Future Capacity Building and Training Capacity building and training for IMED in results based monitoring and evaluation and for FAPAD in project auditing was successfully carried out under RBME. Training should continue along similar lines in any subsequent support. However, it is recommended to send officials from IMED and FAPAD on study tours to Sri Lanka, Malaysia, Vietnam and India in addition to providing local training. Moreover, in a follow up phase there should be greater emphasis on coaching (“mentoring”) IMED officers so that they are able to provide training on RBME to officers joining IMED and to officers from line ministries and agencies that are involved in project design, monitoring and evaluation.

Institutional Issues A strategic shift towards results based monitoring and evaluation had a number of institutional implications. In general this approach requires greater cooperation with other key parts of the Government that are currently undertaking initiatives in results based budgeting, managing for development results and the monitoring of poverty reduction. In particular the Finance Division has a results framework for each designated ministry in the Medium-Term Budget Framework

35

and there are M&E Frameworks for each sector in the National Strategy for Accelerated Poverty Reduction. Consequently, it is important that IMED’s results based monitoring and evaluation function be executed in consonance with these other initiatives. To a large extent the Strategic Plan has been designed so that IMED makes a unique contribution to results based monitoring and evaluation and managing for development results rather than duplicating other efforts. Nevertheless, the execution of the Plan will require careful negotiations and agreements on roles and responsibilities. Clearly IMED’s key reporting relationship for results information will be to NEC and ECNEC.

Implementation Arrangements The recommended implementation arrangements are in line with those for RBME i.e. IMED will be the EA for the TA, and the support to FAPAD is provided as a separate module under the TA, with FAPAD as Implementing Agency. The secretary of IMED will appoint a senior project director (PD), at no less than director general rank, to act as full- time PD for the TA. The FAPAD component will be implemented in close coordination with CAG. A core group of qualified technical staff (minimum one per consultant) within IMED and FAPAD will serve as counterparts to the TA consultants. The counterparts must have background in (i) capacity development and training, (ii) institutional development and change management, and (iii) policies and strategic planning and (iv) MIS/ICT systems development, corresponding to the skill areas of the international and national consultants. A high-level TA steering committee, chaired by the IMED secretary, with the office of the PD serving as the secretariat, will be constituted, involving FAPAD, Planning Commission, Finance Division, key line ministries/agencies, ADB, World Bank, UNDP, DFID, and other partners development partners presently supporting or planning support to IMED and FAPAD, to provide policy guidance and supervise the TA activities. The steering committee will meet at a minimum every second month to review consultants’ reports, and evaluate project activities and project implementation arrangements.2 For a second phase of RBME a greater onus is placed on GoB to review, approve and implement the recommendations, processes and systems proposed by the TA. These undertakings are spelled out explicitly in the TA activities describe above. 2 ADB 2006. Technical Assistance to the People’s Republic of Bangladesh for Results Based Monitoring and Evaluation(Finance by the Japan Special Fund)

36

Additional Institutional and Technical Activities for MIS Institutional Activities:

• Analyze the work flow requirements of the web –based PMIS and make recommendations towards improving IMED’s use the system. The recommendations should include an analysis of both current and recommended work flows, and a detailed description of the changes required.

• Strengthen, through training, the use of the system by IMED’s monitoring officers. Review their current use of the system and analyze problem areas. Provide recommendations on improving the use of the system by the officers.

• Review the organizational responsibilities and tasks assigned to IMED’s MIS section and recommend changes to reflect the operations of the new system. Review job descriptions and staff responsibilities. Develop training courses, as necessary, in topics that are required by the system.

• Undertake interviews and surveys with selected executing agencies to determine what improvements need to be made to the PMIS from their perspective. Compile this feedback and prepare recommendations for changes.

• Analyze the data flows between IMED’s system and the databases of other related organizations on the Planning Commission’s campus and determine whether or not the data exchanges are sufficient and what modifications are required, if any.

Technical Activities: • Conduct an information systems audit that include both a general control review

(physical asset and information security, user access, usage policies, etc.) and an application control review (data integrity, database relations, report correctness, data completeness, etc.). Recommend changes to existing system (new reports, new procedures, improved security, etc.).

• Review the PMIS Technical and User Manuals and make changes as necessary.

Follow up TA Project: The above recommendations for IMED are to increase the institutional capacity due to impact of the project implementation. However further detailed studies should be conducted during implementation of the follow up TA Project, which is expected to be undertaken by IMED to fulfill the requirement of future need.

37

Design and Monitoring Framework Design Summary Performance



Impact Relationship between project inputs and development results defined

Information available to recalculate post fact cost benefit ratios

IMED project completion reports and impact evaluations. FAPAD audit reports

Outcome By the end of the project IMED & FAPAD are auditing, monitoring and evaluating projects against results specified in DMFs (Logframes), sector M&E frameworks and the NSAPR

Impact of projects assessed against specifications in project DMF (Logframe) Sector performance reports in included in “ADP Implementation Progress Review” FAPAD audits report on project efficiencies, effectiveness and economy

Project impact evaluations “ADP Implementation Progress Review” FAPAD audit reports

Validity of evaluations is sufficient to draw reasonable conclusions about development results.

Outputs 1. Design and Monitoring Frameworks (Logical Frameworks) are reviewed for all ADP Projects

DMF written to specifications of official guidelines included in all new DPPs

DPPs

Ministries accept IMED guidance and feedback on project DMFs

2. Sector M&E Frameworks

Impact, outcome and outputs specified for 12 sectors in line with MDGs and NSAPR2

IMED ADP Review of Progress

IMED is given authority to facilitate the formulation or review of existing frameworks

3. Results Based MIS Progress MIS Reports IMED develops capacity (or

38

information on project results retrievable from PMIS Information on sector results available from PMIS

contracts capacity) to maintain and develop MIS

4. Annual Plan for verification of project results

Sample of ADP projects identified. Schedule of M&E events formulated Resources allocated (internal and outsource) to plan QA scheme for outsourced verification formulated.

IMED Annual Work Plan IMED budget Contracts with suppliers of M&E services

IMED executes plan

5. performance reporting system to NEC/ECNEC

Performance report formats drafted for 12 sectors Project rating system calibrated Flow of reporting information with responsibilities described.

Minutes of NEC/ECNEC meetings

IMED is authorized to implement reporting system

6. Auditing and Monitoring better integrated

IMED and FAPAD have joint plans for project inspection. Audit and project monitoring information shared between IMED and FAPAD

Correspondence between DG FAPAD and Secretary IMED

IMED and FAPAD formalize limited joint working arrangements

Milestone Reports Inputs

39

IMED tbd FAPAD tbd

ADB tbd GoB tbd

Government Undertakings (Listed in detail under each activity where appropriate)

40

Appendix 4: Model Audit Report On The Financial Statements Of “Rural Development Project: Greater Noakhali And Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong And Cox's Bazar Districts)” Financed By Idb And Gob For The Year 2007-2008 Foreign Aided Projects Audit Directorate (FAPAD) Audit Complex, Segunbagicha, Dhaka PART-I: Executive Summary Report: 1. The Project to which the Audit Report relates to is "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)”, a project administered by Local Government Division, Ministry of Local Government, Rural Development and Co-operatives; executed by Local Government Engineering Department; and funded by Islamic Development Bank (IDB) and the Government of Bangladesh (GOB). The Project was commenced 1 July 2002 (2002-03); and is expected to be completed by 30 June 2010 (2009-10). The main objectives of the Project are raising income of the rural people; and thereby, reducing rural poverty by developing rural roads, bridges and culverts; growth centres and markets; landing stations (ghats); and tree plantations. 2. The FAPAD audited, in accordance with International Standards of Auditing (ISAs), the financial statements of "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)” financed under BD-0133 by Islamic Development Bank (IDB) and the Government of Bangladesh (GOB) for the year ended 30 June, 2008 prepared by the management of the Project in conformity with the International Public Sector Accounting Standards (IPSASs); and the Project Accounting Manual issued by the Ministry of Finance, Government of the People’s Republic of Bangladesh. 3. In FAPAD’s opinion, except for the differences in (a) the civil work expenditures in the project financial statements and those in the statements of the district offices; (b) the unspent balance in the project financial statements and those in the statements of the district offices; and (c) RPA fund receipt in the project financial statements and those in the Withdrawal Application Forms (form A14) (i) the project financial statements properly present the expenditures and receipts for the year ended 30 June, 2008; and (b) in all material respects, the expenditures and receipts have been applied to the purposes intended by the project documents and conform to the authorities which govern them. 4. During FAPAD’s audit, its Audit Team have come across material matters, which they believe need the personal intervention of the Project on an urgent basis. These, amongst others, are:

The Project incurred a loss of Tk 92,87,120.28 (Taka ninety two lacs eighty seven thousand one hundred twenty and poisha twenty eight) only due to non-deduction of liquidated damage at 10% of the contract value from the final bills of the Contractors as detailed in Audit Para No 1 under Part II.B.I. The Project incurred a loss of Tk 82, 69,222 (Taka eighty two lacs sixty nine thousand two hundred twenty two) only due to non-forfeiture of (a) Performance Security on cancellation of Work Order; (b) Security Deposit on abandonment of Work; (c) Tender Security on failure to sign Contract within stipulated time; and (d) Performance Security on non-execution of Work as detailed in Audit Para No 2 under Part II.B.I.

41

The Project paid an excess payment of Tk Tk. 12,82,038 (Taka twelve lac eighty two thousand thirty eight) only to Messrs Ahsan Enterprise & Chowdhury Builders (JV), Contractor against civil works beyond estimate approved by the Project Director as detailed in Audit Para No 3 under Part II.B.I.

The Project incurred a loss of Tk 30,27,936 (Taka thirty lacs twenty seven thousand nine hundred thirty six) only due to inclusion of 4% income-tax deductible at source in the Cost Estimate of the Work; and then, payment of the same to the Contractors notwithstanding that such income-tax, being part of the Contractors’ profit, has already been included in the Cost Estimate of the Work as profit as detailed in Audit Para No 3 under Part II.B.I.

The Audit Team have observed defective/incomplete works in all the works physically verified by the Team Members, indicating serious weakness over control and completion of such works; and corrupt practices associated therewith as detailed in paragraph 2.D.1 under Part II.B.II.D.

42

PART-II: Audit Reports: A. Short-Form Report: Auditors’ Opinion To The Secretary Local Government Division Ministry of Local Government, Rural Development and Co-operatives Government of the People’s Republic of Bangladesh Dhaka, Bangladesh We have audited the accompanying financial statements of "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)” financed under funded by Islamic Development Bank (IDB) and the Government of Bangladesh (GOB) for the year ended 30 June, 2008 prepared by the management of the Project in conformity with the International Public Sector Accounting Standards (IPSASs;) and the Project Accounting Manual issued by the Ministry of Finance, Government of the People’s Republic of Bangladesh. These financial statements are the responsibility of management. Our responsibility is to express an opinion on these financial statements based on our audit. We conducted our audit in accordance with International Standards on Auditing (ISAs). Those Standards require that we plan and perform the audit to obtain reasonable assurance about whether the financial statements are free of material misstatement. An audit includes examining, on a test basis, evidence supporting the amounts and disclosures in the financial statement. An audit also includes assessing the accounting principles used and significant estimates made by management, as well as evaluating the overall financial statement presentation. We believe that our audit provides a reasonable basis for our opinion. The Financial Statements of the Project for the year showed Tk. 47,95.73 lakh as civil work expenditure; whereas actual expenditures in the district offices aggregated Tk. 40,60.26 lakh, resulting in an excess of expenditure over actual expenditure at Tk 7,35.47 lakh (Tk.47,95.73 lakh-Tk.40,60.26 lakh). Further, there was no unspent balance of RPA fund in the Financial Statements of the Project for the year; but the district offices disclosed an aggregate amount of Tk. 4,32.85 lakh as unspent. Moreover, the Financial Statements of the Project for the year showed RPA fund receipt at Tk. 25,56.72 lakh, compared to Withdrawal Application Forms (form A14) aggregating Tk. 25,58.00 lakh, resulting in an under-statement of RPA fund receipt by Tk. 1.28 lakh (Tk. 25,58.00 lakh-25,56.72 lakh). These differences could not be reconciled by the Project. In our opinion, except for the differences referred to above, The project financial statements properly present the expenditures and receipts for the year ended 30 June, 2008; and In all material respects, the expenditures and receipts have been applied to the purposes intended by project documents and conform to the authorities which govern them. Opinion Status: Qualified-except for

43

(Nazrul Islam Azad) Date: 7 September 2009 Deputy-Director For the Director-General Foreign Aided Project Audit Directorate, Dhaka

44

B. Long-Form Report: Ref No 830/FAPAD/Sector-2/IDB/2007-2008/ __ Date 7 September 2009 To The Secretary Local Government Division Ministry of Local Government, Rural Development and Co-operatives Government of the People’s Republic of Bangladesh Dhaka, Bangladesh. We have audited the financial statements of "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)” for the year ended on 30 June, 2008 and have issued our report on 7 January 2009. During our audit, we have come across matters, which we believe need your personal intervention on an urgent basis. These are:

PART-I. Audit Observations that shall be brought to the attention of CAG/PAC:

Audit Para No 1:

Description of the Audit Observation :

The Project incurred a loss of Tk 92,87,120.28 (Taka ninety two lacs eighty seven thousand one hundred twenty and poisha twenty eight) only due to non-deduction of liquidated damage at 10% of the contract value from the final bills of Messrs Techbay International, Isha international, Dilder Associates and Mohammad Younus & Brothers (Pvt) Ltd, Contractors against civil works although completion of the works were delayed upto 750 days in violation of Clause 49.1 of the General Conditions of Contract (GCC). Messrs Md. Shahidur Rahman Pramanik, XEN, LGED, Noakhali; Md. Faruk Ahmed and Md. Harun-ur- Rashid, XEN, LGED, Laxmipur; Shaik Mozakka Jaher, XEN, LGED, Cox’s Bazar and Mr. Md. Shahjahan Mollah, XEN, LGED, Chittagong were in charge during the period; and responsible for not deducting the aforesaid liquidated damage. Reply of the Auditee :

The aforesaid officials of the Project cited time extensión for “limited working period” or “site land dispute” as the specific reason for not deducting the aforesaid liquidated damage from the final bill of the aforesated Contractors. Auditors’ Comment on the Auditee’s Reply :

The aforesaid officials of the Project could not satisfactorily explain the reasons for not deducting the aforesaid liquidated damage from the final bills of the aforesated Contractors. Auditors’ Recommendation :

45

The aforesaid loss to the Project occured due to the aforesaid officials’ non-compliance of the rules; and accordingly, should be recovered from them unless they themselves can realise the aforesaid liquidated damage from the aforesated Contractors.

Audit Para No 2:


The Project incurred a loss of Tk 82,69,222 (Taka eighty two lacs sixty nine thousand two hundred twenty two) only due to non-forfeiture of (a) Performance Security on cancellation of Work Order; (b) Security Deposit on abandonment of Work; (c) Tender Security on failure to sign Contract within stipulated time; and (d) Performance Security on non-execution of Work. Mr Md. Shahjahan Mollah, XEN, LGED, Chittagong was in charge during the period; and responsible for not forfeiting the aforesaid Performance/ Tender Security. Reply of the Auditee :

The aforesaid official of the Project did not explain the reason for not forfeiting the aforesaid Performance/Tender Security. Auditors’ Comment on the Auditee’s Reply :

The aforesaid official of the Project could not satisfactorily explain the reason for not forfeiting the aforesaid Performance/Tender Security from the Contractors. Auditors’ Recommendation :

The aforesaid loss to the Project occured due to the aforesaid official’s non-compliance of the rules; and accordingly, should be recovered from him unless he himself can realise the amount from the concerned Contractors.

Audit Para No 3:

Description of the Audit Observation : The Project paid an excess payment of Tk Tk. 12,82,038 (Taka twelve lac eighty two thousand thirty eight) only to Messrs Ahsan Enterprise & Chowdhury Builders (JV), Contractor against civil works beyond estimate approved by the Project Director. No approval was obtained from the Project Director for the alteration to the approved estimate prior to execution of the work in terms of paragraph 56 of the CPWD Code. Mr. Nurul Huda was XEN, LGED, Feni during the

46

period; and responsible for not obtaining the approval for the aforesaid alteration in an appropriate manner. Reply of the Auditee :

The aforesaid official of the Project cited that approval of the Contract on a “net basis (net of value of the old materials)”, instead of on a “gross basis” as the specific reason for not obtaining the approval for the aforesaid alteration in an appropriate manner. Auditors’ Comment on the Auditee’s Reply :

The aforesaid official of the Project could not satisfactorily explain the reasons for not obtaining the approval for the aforesaid alteration in an appropriate manner. Auditors’ Recommendation :

The aforesaid loss to the Project occured due to the aforesaid official’s non-compliance of the rules; and accordingly, should be recovered from him unless he himself can either realise the aforesaid excess payment from the aforesated Contractor or obtain the approval for the aforesaid alteration by the Project Director in an appropriate manner.

Audit Para No 4:


The Project incurred a loss of Tk 30,27,936 (Taka thirty lacs twenty seven thousand nine hundred thirty six) only due to inclusion of 4% income-tax deductible at source in the Cost Estimate of the Work; and then, payment of the same to the Contractors notwithstanding that such income-tax, being part of the Contractors’ profit, has already been included in the Cost Estimate of the Work as profit. Therefore, such income-tax has in effect been paid twice to the Contractors. Messrs Md. Harun-ur-Rashid and Faruk Ahmed were XEN, LGED, Laxmipur were in charge during the period; and responsible for inclusión of such income-tax in the Cost Estimate of the Work; and thereafter, payment of the same to the Contractors. Reply of the Auditee :

The aforesaid official of the Project explained that such income-tax was included in the Cost Estimate of the Work as per Rate Schedule approved by the Project Director; and thereafter, paid to the Contractors. Auditors’ Comment on the Auditee’s Reply :

47

The aforesaid official of the Project could not satisfactorily explain the reason for inclusión of such income-tax in the Cost Estimate of the Work; and thereafter, payment of the same to the Contractors. Auditors’ Recommendation :

The aforesaid loss to the Project occured due to the aforesaid official’s non-compliance of the rules; and accordingly, should be recovered from them unless they themselves can realise the amount from the concerned Contractors.

48

PART-II: Section A. Comments and observations on the accounting records, systems and controls that were examined during the course of the audit: 2.A.1 We have observed that the Project has not established and maintained records adequately to identify and control the goods of the Project on a perpetual basis. We recommend that such records should be established and maintained by the Project/LGED on a perpetual basis. 2.A.2 We have observed that the control over the fixed assets of the Project was not satisfactory as inventory thereof was not made, and annual physical verification thereof was not carried out, by it. We recommend that such inventory and annual physical verification thereof should be introduced by the Project/LGED. Section B. Comments on compliance of the covenants on the financing agreement; and internal and external matters affecting such compliance: 2.B.1 We have observed that all goods financed out of the proceeds of the Loan were not insured or cause to be insured by the Project in terms of Section 7.07 of the Loan Agreement between the Government of Bangladesh (GOB) and Islamic Development Bank (IDB). We recommend that such insurance policies should be taken or cause to be taken by the Project/LGED on a perpetual basis. 2.B.2 We have observed that the Project has not established and maintained records adequately to identify the goods financed out of the proceeds of the Loan in terms of Section 7.04 of the Loan Agreement between the Government of Bangladesh (GOB) and Islamic Development Bank (IDB). We recommend that such records should be established and maintained by the Project/LGED on a perpetual basis. 2.B.3 We have observed that the Project/LGED has not ensured that the Contractor in the Project takes insurance cover in the joint name of LGED and the Contactor as per Clause 24.1 of the General Conditions of the Contract, thereby benefitting the Contractor by the amount of insurance premium against such insurance policy; and depriving the Government of 15% VAT on the insurance premium on such insurance policy. We recommend that the Project/LGED should ensure that such insurance policy is taken by the Contactor as per the Contract. Section C. Matters which might have a significant impact on the implementation of the Project: We have noted that poor planning and implementation of works by the XEN of the Project/LGED, caused, amongst others, by poor monitoring by the Project Director, led to underutilisation and also misuse of funds in the Project. We recommend that planning, implementation and monitoring needs to be further strengthened in the Project/LGED. Section D. Comments and recommendations on economy, efficiency, and effectiveness in the use of resources: 2.D.1 We have carried out physical verification of certain works completed in the Project on a sample basis during the course of our audit. The results of such verification are as follows:-

49

Work Order No. LGED/XEN/Cox/GNCP-15/2005/735 dated 12.3.2005 issued in favour of Messrs FCS Consortium for improvement of Islamabad-Gomatali road development work at a contract value of Tk.2,47,49,388.53 under Package GNCP-42 completed on 30.6.07: (a) Defective road development work in a number of places; (b) Inferior quality bricks were used in road (as compared to 1st class bricks which should not have been accepted by the LGED and supplied by the contractor); and (c) Earth filling was not done in the case of culverts. Work order no LGED/XEN/Cox/GNCP-2/2004/2784 dated 26/10/2004 issued in favour of M/S Royal Cox Associate at a contract amount of Tk.48,82,520.00 for Construction of 90.1m bridge on Bakkhali Khal completed on 15-06-2006: (a) Brick soling work was incomplete; (b) Finishing was incomplete; and (c) Name plate and signboard were not supplied. Work Order No. LGED/XEN/Cox/GNCP-27/2005/1365 dated 19.05.2005 issued in favour of M/S Didar Associate & Brothers Const Consortium at a contract amount of Tk. 88,97,418.14 for Improvement of Jalabad-Pokkhali Road and Development of Khrushkul Bazar Work under Package no.GNCP-41 completed on 30.06.2007: Defective road development works in a number of places; Inferior quality bricks were issued in road development works; Earth filling was not done in the cases of culvert; and Growth centre is either not used or used as for purpose not meant for. 31.13.m Bridge constructed over the Baminala Khal at ch. 4100 m at cost of Tk. 31.10 lac under Package no. GNCP-37 in Lohagara Upazila completed on 24.06.07: People do not get benefit from the Bridge as the approach road (about 30m) from the both sides was not included in the estimate and so constructed. Work Order No. LGED/XEN/Feni/2835 dated 03/10/2005 issued in favour of M/S Ahsan Enterprise & M/S Chowdhury Builders (JV) at a contract amount of Tk.3,74,27,869/- (after recast Tk.2,52,07635/-) for Construction of several culverts and roads under Package no.GNCP-12 completed on 22.11.2007: Defective road development works in two places; Inferior qualitative seal coat works in one place; and Road damage in another place. Work Order no. LGED/XEN/Feni/D (GNCP)-79/08/1028 dated-12.03.2008 issued in favour of M/S Alahi Enterprise at a contract amount of Tk.92,76,539/- for improvement of several roads under Package no. GNCP/07-08/Wo3 completed on 30.04.2008: Road improvement completion certificate was issued on 30.04.2008 & final bill amount of Tk. 71,97,328/- was paid accordingly but the portion Part-2 (Scheme code no. A-3029-08-10026); Part-3 (Scheme no. 43029-08-27) & Part-4 (Scheme code no.43029-08-10028) of ch.4020m and 5138m location box culvert were not been constructed.

50

Work Order no LGED/XEN/LAX/2007/2436 dated 08.11.07 issued in favour of contractor M/S Jahangir Alam at a contract amount of Tk.79,97,381.00 for Development of TT & DC Road at Raipura Upazilla under Package no. GNCP/01/2007-08-2 completed on 08.06.2008: Defective road development work in two places. Work Order no. LGED/XEN/GNCP (IDB)/Sen/S-3/2004-05/3248 dated 11.09.2005 issued in favour of M/S Tech Bay International at a contract amount of Tk.2,16,02,784.25 for Samirmunshi-Chelonia via Lemunia Road development work under Package no.GNCP/01/2004-05 completed on 10.12.2007: Defective road development work in three places; Inferior quality bricks (as compared to 1st class bricks); Earth filling work was not done in the case of two culverts. Work Order no LGED/XEN/NoA/Com-sub/s-01/2007-08/809 dated 17.02.08 issued in favour of M/S Javed- Bachchu(JV) at a contract amount of Tk 2,88,45,782.00 for Shamsuddin, Hafezia and Meherunessa Road Development work under Package no. 06/07-08 said to have been completed on 30.4.08: (a) Slaughter House at Chawdhury Hat that was claimed to have been completed was in deplorable condition; (b) Bridge over Dadna khal was not constructed although completion certificate mentions that the aforesaid work has been completed on 30.04.08 and progress report for the month of June, 2008 also records that 100% work has been completed. Work Order no LGED/XEN/LAX/GNCP/2005/2322 dated 2/11/2005 issued in favour of M/S Isha International at a contract amount of Tk.2,64,14,071.46 for Construction of Bridge at Char Banshi Char Ruhita at Raipur Upazila under package no. GNCP-22 claimed to have been completed on 11/6/2008: (a) Earth filling work not completed; (b) Single layer brick soling not completed; (c) Providing name plate not completed; There is no approach road enabling the people to use the road; and Full and final bill was paid vide bill no.60/2007-08 dated 11.06.2008 despite non-completion of full works. Such defective/incomplete works observed by us in all the works physically verified by us indicate serious weakness over control and completion of such works; and corrupt practices associated therewith. We recommend that effective steps should be taken by the Project/LGED to (a) rectify such defective/incomplete works; (b) identify and bring the officials of the Project associated with such works to book; and (c) introduce measures for effective control and completion of such works. 2.D.2 Based on the Financial Progress Reports received by us from XEN, Noakhali, Feni and Cox’s Bazar (no report was received by us from XEN, Chittagong despite repeated requests from us; and the report received from XEN, Laxmipur was erroneous), we have observed that the Cost Inefficiency Percentage stood at 15.82%, whereas Cost Variance (Underutilisation) Percentage came to 54.43%, as follows:-

Amount in Lakh Taka

51

Projected

Revised

Projected

Revised Actual

Appropriate

Sl No Item of Work

Unit

Unit Cost

Unit Cost

Total Cost

Total Cost

Total Cost

Total Cost

a) Upazilla Road Development km 30.00 31.91

5910.00

7658.25

2046.68

1634.40

b) Construction of Bridge/Culvert on Upazilla Road m 1.30 1.47

975.00

1344.13

766.45

706.407

c) Union Road Development km 20.00 21.75

2500.00

3328.48

4547.47

3794.72

d) Construction of Bridge/Culvert on Union Road m 1.00 1.26

1500.00

2307.18

223.40 232.95

e) Growth Centre/Market Development

No 20.00 20.00

800.00

980.00 28.79 40

f) Construction of Union Parishad Bhaban

No 30.00 38.19

570.00

878.35 0.00 0.00

g) Tree Plantation km 0.50 0.50 25.00 30.50 0.00 0.00

h) Construction of Landing Stage No 15.00 15.00

150.00

180.00 0.00 0.00

12430.00

16706.90

7612.79

6408.48

Section E. Comments and recommendations on achievement of the planned results of the Project: Based on the Physical Progress Reports received by us from XEN, Laxmipur, Noakhali, Feni and Cox’s Bazar (no report was received by us from XEN, Chittagong despite repeated requests from us), we have observed that the achievement of the planned results of the Project is not satisfactory, as follows:-

Planned Actual AchievementSl No Item of Work Unit Output Output % a) Upazilla Road Development km 240 75 31.30%

b) Construction of Bridge/Culvert on Upazilla Road m 917 666 72.67%

c) Union Road Development km 153 205 133.75%

d) Construction of Bridge/Culvert on Union Road m 1833 283 15.45%

e) Growth Centre/Market Development No 49 2 4.08% f) Construction of Union Bazar No 23 0 0.00% g) Tree Plantation km 61 0 0.00% h) Construction of Landing Stage No 12 0 0.00%

The results indicate that the Project/LGED had (a) more interests in costlier road works than socially desirable works like Growth Centre, Union Bazar, Landing Stage (Ghat) and Tree Plantation; and (b) poor monitoring over planning and implementation of works. We recommend

52

that monitoring over planning and implementation of works in the Project/LGED need to be strengthened so as to achieve value for money. Section F. Any other matters considered pertinent: 2.F.1 We have observed that despite non-completion of the work against Work Order No LGED/XEN/NoA/GNCP/Begum/S-01/2008/2829 dated 16.09.2007 issued to the Contractor, Hossain and Brothers for Development of Sirajmiahpool (Chowmuhuni) Talala-Hazipur UP Office Road at a contract amount of Tk.2,18,97,932.05 (Package no. 01/2007-08), the Project has released the performance security (i,e, Bank Gurantee No. 242/2007 dated 10.09.2007 for Tk 21,90,000) on 26.11.08, violating Clause 67.2/67.3 of the General Conditions of the Contract and putting LGED at performance risks. We recommend that the Project should not release performance security unless all incomplete/defective works are completed by the Contractor to the satisfaction of the Project/LGED; and warranty (defect liability) period of the works have expired. 2.F.2 We have also observed that Contract against Work Order No 4827 dated 12/10/05 issued to the Contractor, Messrs Techbay International for Tk.1,24,13,769 was signed by XEN, LGED, Chittagong on 25.10.05 although the tender validity thereof expired on 30.09.05. We recommend that the Project should either extend the validity of a tender or not issue Work Order/enter into a Contract after expiry of the validity of a tender. 2.F.3 We have further observed that accrued bank interest at Tk. 2,25,526 and forfeited security deposit at Tk. 1,11,307, totalling Tk Tk. 3,36,833 was not deposited by the Project into the Government Account. We recommend that such accrued interest and forfeited deposit should deposited by the Project into the Government Account before the expiry of the concerned financial year. PART-III. Carryover unresolved Audit Observations: We have noted that 17 Audit Paras having financial involvement aggregating Tk 4,44,60,000 for the years ended 30 June 2003 through 2007 have been settled till 30 June 2008; whereas 32 Audit Paras having financial involvement aggregating Tk 16,36,26,000 for the years ended 30 June 2004 through 2007 have not yet been settled by the Project, as detailed below:-

Year of audit

Nos. of Paras

Financial involvement of Audit Paras

Nos. of Paras settled

Financial involvement of Audit Paras settled

Nos. of Outstanding paras

Financial involvement of Audit Paras outstanding

2002-03 06 49,40,000 06 49,40,000 - -

2003-04 10 3,93,44,000 06 369,84,000 04 23,60,000

2004-05 08 6,73,03,000 03 23,78,000 05 6,49,25,000

2005-06 10 8,49,33,000 02 1,58,000 08 8,47,75,000

2006-07 15 1,15,66,000 - - 15 1,15,66,000

53

Total 49 20,80,86,000 17 4,44,60,000 32 16,36,26,000

With respect to our current and carry over recommendations, we suggest that an implementation timetable be prepared and followed. We look forward to receiving your response at your earliest convenience. Thank you Yours truly (Nazrul Islam Azad) Deputy Director For Director General Foreign-Aided Projects Audit Directorate Phone: 935630 SECTION-III: Project "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)” Statement of Cash Receipts & Payments and Changes in Cash Balances For the Year Ended 30 June, 2008 (Figure in lakh taka)

Receipts Notes Current Period

Previous Period

Cumulative

1. Contribution from GOB 3 22,98.25 150,46.90 173,45.15 2. Loan from lender/donor 4 25,56.72 42,83.50 68,40.22 3. Other Receipts 5 0 0 0 Total Receipts for the year 48,54.97 193,30.40 241,85.37 Payments

1. Land Acquisition/Development 0 0 0 2. Civil Works 47,95.73 191,70.25 239,65.98 3. Equipment-L/C Value 0 0 0 4. Equipment-IDVAT 0 0 0 5. Equipment-Other Costs 0 0 0 6. Other Fixed Assets 5.00 47.47 52.47 7. Consulting Services 36.80 62.29 99.09 8. Personnel Costs 11.67 20.29 31.96 9. Interest, Fees and Charges 0 2.24 2.24 10. 11. 12. Other Costs 6.99 26.64 33.63 Total Payments for the year 48,56.19 193,29.18 241,85.37 Excess of Receipts over Payments/ (Payments over Receipts) for the year

(1.22) 1.22 0

54

Cash Opening Balance 6 1.22 0 1.22 Cash Closing Balance 6 0 1.22 1.22 Note- The heads of cash receipts and payments shall be in conformity with the Project Documents/Aid Agreement. Head of Accounts of Project Head of Project

55

Project "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)”

Statement of Comparison of Budget with Actual For the Year Ended 30 June, 2008

(Figure in lakh taka)

Receipts Notes

Actual Amounts

Original Budget

Final Budget

Variance Amounts

1. Contribution from GOB 3 22,98.25 41,79.00 31,26.75 8,28.50 2. Loan from lender/donor 4 25,56.72 18,21.00 25,58.00 1.28 3. Other Receipts 5 0 0 0 0 Total Receipts for the year 48,54.97 60,00.00 56,84.75 8,29.78 Payments

1. Land Acquisition/Development 0 0 0 0 2. Civil Works 47,95.73 59,11.00 56,15.20 822.57 3. Equipment-L/C Value 0 0 0 0 4. Equipment-IDVAT 0 0 0 0 5. Equipment-Other Costs 0 0 0 0 6. Other Fixed Assets 5.00 5.00 5.00 5.00 7. Consulting Services 36.80 35.00 36.80 0 8. Personnel Costs 11.67 42.00 42.00 30.33 9. Interest, Fees and Charges 0 0 0 0 10. 11. 12. Other Costs 6.99 7.00 7.00 0.01 Total Payments for the year 48,56.19 60,00.00 57,06.00 852.91 Excess of Receipts over Payments/ (Payments over Receipts) for the year

(1.22) 0 (21.25) (23.13)

Cash Opening Balance 6 1.22 0 0 1.22 Cash Closing Balance 6 0 0 0 (21.91) Note- The heads of cash receipts and payments shall be in conformity with the Project Documents/Aid Agreement. Head of Accounts of Project Head of Project

56

Project "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)” Notes to Financial Statements for the Year Ended 30 June 2008

1. Status and Activities of the Project The Project to which the Audit Report relates to is "Rural Development Project: Greater Noakhali & Chittagong Districts (Feni, Laxmipur, Noakhali, Chittagong and Cox's Bazar Districts)”, a project administered by Local Government Division, Ministry of Local Government, Rural Development and Co-operatives; executed by Local Government Engineering Department; and funded by Islamic Development Bank (IDB) and the Government of Bangladesh (GOB). The Project was commenced 1 July 2002 (2002-03); and is expected to be completed by 30 June 2010 (2009-10). The main objectives of the Project are raising income of the rural people; and thereby, reducing rural poverty by developing rural roads, bridges and culverts; growth centres and markets; landing stations (ghats); and tree plantations. 2. Accounting Policies The financial statements are prepared in conformity with the International Public Sector Accounting Standard (IPSAS): “Financial Reporting under the Cash Basis Accounting” and other IPSASs issued by the International Public Sector Accounting Standards Board (IPSASB); and also Project Accounting Manual issued by the Ministry of Finance, Government of the People’s Republic of Bangladesh (GOB); 3. Contribution from GOB

Funds are allocated by the GOB to cover GOB’s share of eligible project expenditures, as specified in the Project Proforma and in the Annual Development Programme (ADP) for each year of the project. Any allocated ADP funds that have not been expended by the end of fiscal year lapse and must be returned to GOB. GOB contributions to the project since inception are as follows:

(In lakh taka) Current Period Previous Period Cumulative

Disbursement by GOB

31,26.75 150,68.59 181,95..34

Less: Refund to GOB

828.50 21.69 850.19

Total 22,98.25 150,46.90 173,45.15 4. Loan from lender/donor The (name the lender/donor) has provided funds to the project to cover its share of eligible project expenditures. These funds, which must be repaid to (name lender/donor) after the conclusion of the project, have been drawn by the project in accordance with the withdrawal procedures as follows:

(In lakh taka) Current Period Previous Period Cumulative

57

Initial Deposit (Advance) 0 0 0 DPA (Direct Payment) 0 0 0 RPA (SOE Procedure) 0 0 0 RPA (Non SOE Procedure)

25,56.72 42,83.50 68,40.22

Others 0 0 0 Total 25,56.72 42,83.50 68,40.22

5. Other receipts Other receipts consist of the following: Current Period Previous Period Cumulative

0 0 0 0 0 0 Total 0 0 0 6. Cash The project maintains two bank accounts: an imprest bank account, or revolving fund, which is kept at Bangladesh Bank to hold funds advanced by the lender/donor, and an operating account, which is kept at a commercial bank to hold funds advanced by GOB as well as funds transferred from the imprest bank account,. Year end cash balances were as follows.

(In lakh taka)

Current Period Previous Period Cumulative Current Period

Imprest Account 0 0 0 Operating Account (RPA) 0 1.22 0 Operating Account (GOB) 0 0 0 Total 0 1.22 0 Head of Accounts of Project Head of Project Note-The figures against the previous period include cumulative figures, instead of periodical figures.