Evaluation of the Bank’s Country Strategy and Program 2005 ...idev.afdb.org/sites/default/files/documents/files... · This report summarizes the findings, lessons and recommendations

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Malawi: Evaluation of the Bank’s Country

Strategy and Program 2005–2016Summary Report

Independent Development Evaluation

From experience to knowledge... From knowledge to action... From action to impact

African Development Bank

October 2018

IDEV conducts different types of evaluations to achieve its

strategic objectives

Thematic Evaluations Project Cluster Evaluations

Regional Integration Stra

tegy

Evaluations

Project Perfo

rmance Evaluations

(Public Secto

r)Impact Evaluations

Project Performance Evaluations

(Private Sector)

Coun

try S

trate

gy E

valu

atio

n

Evaluation Syntheses

Corporate Evaluations

Sect

or E

valu

atio

ns

Coun

try

Stra

tegy

Eva

luat

ions

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Malawi: Evaluation of the Bank’s Country

Strategy and Program 2005–2016Summary Report

Independent Development Evaluation

From experience to knowledge... From knowledge to action... From action to impact

African Development Bank

October 2018

© 2018 African Development Bank Group All rights reserved – Published October 2018

Malawi: Evaluation of the Bank’s Country Strategy and Program 2005–2016 – Summary Report IDEV Country Strategy Evaluation, October 2018

Disclaimer

Unless expressly stated otherwise, the findings, interpretations and conclusions expressed in this publication are those of the various authors of the publication and are not necessarily those of the Management of the African Development Bank (the “Bank”) and the African Development Fund (the “Fund”), Boards of Directors, Boards of Governors or the countries they represent.

Use of this publication is at the reader’s sole risk. The content of this publication is provided without warranty of any kind, either express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, and non- infringement of third-party rights. The Bank specifically does not make any warranties or representations as to the accuracy, completeness, reliability or current validity of any information contained in the publication. Under no circumstances including, but not limited to, negligence, shall the Bank be liable for any loss, damage, liability or expense incurred or suffered which is claimed to result directly or indirectly from use of this publication or reliance on its content.

This publication may contain advice, opinions, and statements of various information and content providers. The Bank does not represent or endorse the accuracy, completeness, reliability or current validity of any advice, opinion, statement or other information provided by any information or content provider or other person or entity. Reliance upon any such opinion, advice, statement, or other information shall also be at the reader’s own risk.

About the AfDB

The overarching objective of the African Development Bank Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction. The Bank Group achieves this objective by mobilizing and allocating resources for investment in RMCs and providing policy advice and technical assistance to support development efforts.

About Independent Development Evaluation (IDEV)

The mission of Independent Development Evaluation at the AfDB is to enhance the development effectiveness of the institution in its regional member countries through independent and instrumental evaluations and partnerships for sharing knowledge.

Independent Development Evaluation (IDEV) African Development Bank GroupAvenue Joseph Anoma, 01 BP 1387 Abidjan 01, Côte d'IvoirePhone: +225 20 26 20 41 E-mail: [email protected] idev.afdb.org

Design & layout: CRÉON – www.creondesign.net

ACKNOWLEDGMENTS

Task Manager Khaled Ibn Waleed Samir Hussein

Team Members Mirianaud Oswald Agbadome, Bilal Bagayoko

Consultants Lattanzio Advisory

External peer reviewer James C. McDavid

Internal peer reviewer Foday Turay

Reference Group Jonathan Banda, Investment Officer; Kelvin Banda, Principal Country Economist; Philip Boahen, Chief Agriculture Policy Economist; Emmanuel Chisesa, Senior Procurement Operations Officer; Eyerusalem Fasika, Principal Country Program Officer; Fenwick Kamanga, Principal Governance Expert; Vinda Kisyombe, Agricultural Economist; Davies Makasa, Principal Transport Engineer; Peter Mwanakatwe, Chief Country Economist; Delight Ngwira, Disbursement Assistant; Benson Nkhoma, Senior Country Program Officer; Peter Rasmussen, Principal Country Economist

Knowledge Management Officers Jayne Musumba, Principal Knowledge Management officerAminata Kouma, Junior Consultant Knowledge Management and Communications

Special Thanks to Andrew Mwaba and Frank Mvula, the Bank's former Resident Representatives in Malawi for their support and contribution during different stages of the evaluation. Staff of development partners, representatives and external stakeholders, clients and partners and private sector clients for their time for interviews, arranging site visits, and providing on evaluation findings.

Division manager Madhusoodhanan Mampuzhasseril (OIC)

Evaluator General Karen Rot-Münstermann (Acting)

ContentsAbbreviations and Acronyms vExecutive Summary 1Management Response 9

Introduction 23

The Development Assistance Context of Malawi 24

Malawi Government Development Strategies 26

AfDB’S Strategic Response for Malawi (2005–2016) 28Overall Strategy 28AfDB Portfolio 31

Evaluation Findings 35Relevance 35Effectiveness 37Sustainability 38Crosscutting issues 42Efficiency 44Knowledge and Policy Advice 44Partnerships, Harmonization and Leverage 48Managing for Results 49Lessons Learned and Knowledge Management 52

Lessons Learned and Recommendations 54Lessons Learned 54Recommendations 54

Annexes 57

Contents

List of figuresFigure 1 Evolution of the pillars of the Bank’s CSPs for Malawi (2005–2016) 29Figure 2 Amounts (UA million) committed and disbursed by year in Malawi (2006–2016) 30Figure 3 AfDB commitments by sector in Malawi (2006–2016) 31Figure 4 AfDB CSPs and Malawi MGDS priority areas 36

List of tablesTable 1 Destination of Bilateral Aid 2007–15 24Table 2 Main donors and their disbursements in US$ over the last three years (2014–2016) 24Table 3 Government of Malawi Receipts 2013–2016 24Table 4 Evolution of net ODA received as a percentage of GNI-Malawi 25Table 5 Malawi Government’s strategies and priority areas 26Table 6 AfDB Commitments by status and sector 31Table 7 Rating scale and description 35

List of boxesBox 1 List of ESW conducted during the evaluation period 32

vAbbreviations and Acronyms

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Abbreviations and Acronyms

ADF African Development Fund

AfDB African Development Bank Group

AISP Agricultural Infrastructure Services Project

CABS Common Approach to Budget Support

CARLA Climate Adaptation for Rural Livelihoods and Agriculture

CSF Community Storage Facilitiest

CSP Country Strategy Paper

DCAFS Donor Committee on Agriculture and Food Security

DfID Department for International Development (UK)

DPs Development Partners

ESIA Environmental and Social Impact Assessment

ESMP Environmental and Social Management Plan

ESW Economic and Social Work

EU European Union

EQ Evaluation Question

GBS General Budget Support

GDP Gross Domestic Product

GoM Government of Malawi

GPRSG Governance and Poverty Reduction Support Grant

ICSP Interim Country Strategy Paper

IDEV Independent Development Evaluation function of the AfDB

JC Judgment Criteria

JIU Joint Country Assistance Strategy papers

KM Knowledge Management

LED Local Economic Development

M & E Monitoring and Evaluation

MCs Market Centers

MDGs Millennium Development Goals

MGDS Malawi Growth and Development Strategy

MTR Mid-Term Review

NIP National Irrigation Policy

NWDP National Water Development Project

OECD Organization for Economic Cooperation and Development

OSBP one-stop-border-posts

PBO Program Based Operation or Policy Based Operation

PBS Protection of Basic Services

PCR Project Completion Report

PD Policy Dialogue

PFEMRP Public Financial and Economic Management Reform Program

PFM Public Finance Management

PIU Project Implementation Unit

PSD Private Sector Development

RBM Results Based Management

REC Regional Economic Communities

RFSSP Restoration of Fiscal Stability and Social Protection

RISP Regional Integration Strategy Paper

RWSSI Rural Water and Sanitation Supply Initiative

SCPMP/SCMP Smallholder Crop Production and Marketing Project

SIVAP Smallholder Irrigation and Value Addition Project

SVIP Shire Valley Irrigation Project

SWAp Sector-wide Approach to Programing

UK United Kingdom

WATSAN Water and Sanitation

WB World Bank

Phot

o: ©

AfD

B

1Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Executive Summary

Introduction

This report summarizes the findings, lessons and recommendations of the independent evaluation of the African Development Bank’s (AfDB or, the Bank) Country Strategy and Program in the Republic of Malawi during 2005–2016. The main objective of the evaluation is to inform the next Country Strategy Paper (CSP) and to contribute to both accountability and learning in the Bank as well as in countries where the Bank’s program is similar to that of Malawi. The evaluation exercise focused on both strategic and individual interventions in all sectors (Social, Transport, Water and Sanitation or WATSAN, Agriculture, Multisector – Financial Governance, and Private Sector Development) where the Bank was active during the review period.

Evaluation Findings

Relevance

Relevance was evaluated by assessing the extent to which the Bank’s country strategies and operations were aligned to Malawi’s development needs, strategies and priorities as well as the needs of ultimate beneficiaries. The relevance of Bank strategy and assistance to Malawi was Satisfactory.

Bank strategies are aligned with the priorities of the Government of Malawi (GoM) and continued to adapt to changes in national development plans as they evolved over time. The Bank’s interventions at sector level are directly related to one or more of the Government’s strategic areas of focus.

However, it should be noted that the CSP did not clearly detail specific beneficiaries even though the

project documents contain references to beneficiaries, and the Bank’s projects are aligned with their needs.

Effectiveness

Effectiveness was evaluated by assessing the extent to which the Bank’s interventions in Malawi achieved their expected development results (outputs and outcomes) against targets set in project approval documents. Additionally, effectiveness was evaluated also by ascertaining the extent to which the Bank interventions benefited targeted beneficiaries. This performance dimension was Satisfactory.

Project Completion Reports (PCR) that were available for the portfolio under review reflect a satisfactory level of performance in terms of achievement in both outputs and outcomes. However, it should be noted that the PCRs took into consideration a post-Midterm Review (MTR) definition of expected results, which according to the majority of PCRs, had considerably reduced expected output and outcome targets.

In the agriculture sector, on the two projects that were closed during the review period, expected effects were clearly stated and monitored, and achievement levels were clearly stated. In the transport sector, two projects which were evaluated in-depth had delivered expected outcomes and physical outputs were well achieved in the sector.

In the transport sector, project progress reports and two PCRs were scrutinized. All Bank supported interventions for provision (or rehabilitation/upgrading) of road transport infrastructure delivered expected outputs (in some cases at additional time and cost; and in some cases at lower than estimated cost). Overall, the physical outputs of the interventions were well achieved in the sector.

2 Malawi: Evaluation of the Bank’s Country Strategy and Program 2005–2016 – Summary Report

For the WATSAN sector, the sector targets were set out in general terms in the CSPs and responded to Malawi Gov’t Development Strategy (MGDS) targets. All sector interventions specify expected results (outputs, outcomes and impacts) in response to higher-level strategies. Monitoring indicators are defined and logical linkage formulated, although the translation of outputs into higher level outcomes and impacts was not as clearly outlined.

Sustainability

Sustainability was evaluated by assessing the extent to which the benefits accrued and gained from Bank interventions continued to exist or were likely to continue to exist after project completion. This performance dimension was Unsatisfactory.

None of the completed projects under review had reached a post-closure period of 5 years, which made assessment of actual sustainability challenging. The evidence collected from the field shows that some benefits gained from past interventions (in particular the soft components such as training and community-led management) were already at risk of not being sustained in the immediate years after completion. Benefits from past interventions, most notably infrastructure, have continued to exist (for longer than 5 years), but longer-term sustainability (i.e. for the expected project life) could not be ascertained.

Better sustainability practices were found in the agriculture and transport sector, compared to the social sector. At the same time, sustainability was not guaranteed in any of these three sectors. In the agriculture sector, e.g. the Smallholder Crop Production and Marketing Project (SCPMP) was rated unsatisfactory on three parameters: i) financial sustainability; ii) ownership and sustainability of partnerships; and iii) environmental and social sustainability. Only institutional sustainability and strengthening of capacities was rated satisfactory, with a score of three. In contrast, the Climate Adaptation for Rural Livelihoods and Agriculture

(CARLA) project was rated satisfactory on all similar parameters, with a highly satisfactory rating of 4 for environmental and social sustainability. In the social sector, the evaluation concludes that the Local Economic Development (LED), Competitiveness, and Job Creation may not be sustainable under the present aid architecture (i.e. unless considerable investment continues to flow to those strategies over a long period).

Crosscutting issues

Crosscutting issues were strongly considered at the strategic level by the Bank and were evaluated by assessing the extent to which the Bank’s objectives on inclusiveness across gender, regions, and environmental sustainability been mainstreamed into Bank interventions. This performance dimension was Satisfactory.

In both the Malawi CSP and the Southern Africa Regional Integration Strategy Paper (RISP), gender issues were addressed (including disparities in access to services, decision-making structures, economic opportunities, cultural biases, budgetary allocation and monitoring of gender actions).1

The Bank carried out a country Gender Profile in 2005, which highlighted the main constraints to women empowerment. Therefore, project documents, mainly appraisal reports, have a section dedicated to gender issues, which are expected to be directly addressed by the project. Bank projects in Malawi during the period under review had a gender informed design, which specifically incorporated gender related elements.

For the social sector (education and health), it is noted that specific targets for gender are included for every intervention. In particular, for the education sector, the analysis shows that women tend to be less represented in the secondary cycle with negative consequences on employment. As for the transport sector, although Bank strategies do not identify transport sector-specific gender issues,

3Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

there is little doubt that the transport projects would support women, men, children and the elderly equally via improved social and economic mobility and improved access to markets and health centers.

Investments in the WATSAN sector usually generate great inherent impacts on women and girls at multiple levels – access to adequate quantities of potable water within a reasonable distance, water carrying (which is predominantly a task of women and girls), sanitation and hygiene with direct impacts on family health, irrigation for agriculture (an activity with high female participation).

There is evidence that women participated in the agriculture sector projects. For instance, the SCPMP completion report notes that the project was designed to promote gender mainstreaming and specifically targeted female-headed households to improve their livelihoods through membership in Farmers’ Organizations.

The Bank programs are expected to cover all regions, but there is limited rollout to the remote and poorest districts either by the GoM or other donors. All interventions cover more than one district and are implemented in rural areas. However, inclusive coverage of regions where poverty is most prevalent were the most problematic (as is quantification of Bank sector support contribution to poverty alleviation). Unless the poorest and most vulnerable are explicitly targeted, they are unlikely to benefit from the Bank support for big infrastructure (road corridors) or health and education projects.

In accordance with the 1996 Environmental Management Act, all sector projects in Malawi should undertake an Environmental Impact Assessment. Therefore, the Bank has made efforts to mainstream environmental safeguards into project preparation and implementation. Although in most cases good engineering practices should minimize adverse environmental and social impacts, there is only limited capacity in applying sound environmental and social mitigation measures.

At the CSP level, environmental and social issues in Malawi are concerned with describing the current situation rather than formulating explicit strategies to address these issues. However, it is clearly stated that environmental protection and mitigation of adverse social impacts are a component of Bank support. The RISP goes further, noting that the Bank will take leadership in promoting climate friendly infrastructure and reduced carbon emissions,2 making specific reference to Malawi’s (and some other southern African countries) problems regarding the effects of increasing climatic variability on infrastructure, food security and development.

Environmental and Social Impact Assessments (ESIAs) and Environmental and Social Managements Plans (ESMPs) are produced for all sector interventions, although a lack of reporting on the implementation of ESMPs is noted. These ESIAs and ESMPs are prepared in compliance with national legislation and AfDB environmental and social policies (which, in turn, are fully compliant with international best practices). All AfDB projects are categorized at an early stage in preparation and this categorization defines the expected nature of environmental (and social) impact and the mandatory mitigation treatment.3 Climate change is mentioned in such studies, but no proactive resilience measures are noted in the projects scrutinized; also there are no targets for carbon footprint reduction.

Efficiency

Efficiency was evaluated by assessing the extent to which the resources and inputs provided were converted to results at a rate that reflects the assumptions at appraisal. Timeliness in implementation was an integral part of this assessment. This performance dimension was Unsatisfactory.

The Bank has generally been responsive and flexible in ensuring satisfactory project performance by allowing a reallocation of resources across the Public Finance Management (PFM) of the project’s


main components, and using unused resources to fund follow-up activities within the ‘multisector.’ Completed agriculture projects (SCPMP, CARLA) had PCR results that gave a resource use efficiency rating of three, which is satisfactory.

However, the Bank’s interventions are associated with delays, largely due to noncompliance with the Bank’s operational standards (although slow Bank response to Government requests for ‘no objection’ has also been noted). Added to this are the country capacity constraints, causing implementation delays.

For non-budget support projects, disbursement delays were caused mainly by the administrative procedures of GoM - this is a general problem for almost all Bank operations in Malawi. The non-Bank managers find the Bank’s finance-related processes cumbersome. Finally, it may be noted that the disbursements predicted for private sector development through commercial banks did not materialize.

Efforts were made by the Bank to use national systems, without much success. For a country at Malawi’s level of development, difficulties in the use of national systems are predictable. Project Implementation Units (PIU) are very common and the Bank’s systems were used such that there were no considerable delays in first disbursements, which took place largely as planned.

Knowledge and Policy Advice

This performance dimension was evaluated by assessing the extent to which the Bank was engaged in and influenced Policy Dialogue (PD) through provision of relevant advices which were then incorporated into policy decisions. The appropriateness and adequacy of the analytical work in support of its interventions, positioning and policy advice were assessed. Knowledge and policy advice dimension was Satisfactory.

The Bank engaged regularly in dialogue with the government. However, such dialogues had no clear

predefined objectives or targets. It was also mostly collective in nature and executed through various donor groups. Interviews with donors stressed the fact that the PD undertaken by the Bank was very useful and was always done in a spirit of collaboration, cooperation and support. Examples in General Budget Support (GBS), health, education, transport and agriculture were noted, with many expressing a desire for the Bank to take a leadership role, given its ‘preferred relationship’ with the GoM, which considers the AfDB as ‘its Bank.’

The Bank was consulted during the preparation of all Government MGDSs and the Government took on board some of the advices given by the Bank. The GoM’s decision to change its policy on farm subsidies and a subsequent move to a more private sector model was largely the result of the Bank’s PD with the Ministries responsible for agriculture, trade, irrigation and related sectors. Other examples include health policy on maternal care and content of MGDSs. The Bank also fully participated in the Common Approach to Budget Support (CABS) process and structure. However, it did not significantly influence policy change. There is no major example to show that CABS was influential in getting any difficult reforms off the ground.

Partnership, Harmonization and Leverage

Partnership, Harmonization and Leverage were assessed by evaluating the extent to which the Bank’s interventions and processes were harmonized with those of other donors to avoid duplication, and the extent to which the Bank’s interventions and resources were influencing other stakeholders to become involved in the sectors that were the focus of country strategies. This performance dimension was Satisfactory.

The Bank and other donors have long followed a coordinated approach to their development assistance in Malawi. In key sectors, sector-wide approaches have allowed government and donors

5Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

to work towards shared strategic objectives. While donors have continued to follow their own programing cycle,4 this aid architecture has been conducive to a relatively high level of shared (rather than joint) strategic planning. In most sectors, donors broadly share the same analysis and objectives about key reforms.

The Bank has been an active participant in several donor platforms such as the Donor Committee on Agriculture and Food Security (DCAFS) for agriculture, and the Public Financial and Economic Management Reform Program (PFEMRP) for PFM. While some de facto form of division of labor among donors has been in place, donor ambitions to move towards joint undertaking, managed jointly, have only been met partly. There are limited examples of co-funded projects, such as the Shire Valley Irrigation Project (SVIP), which is implemented together with the World Bank (WB), and the National Water Development Project (NWDP) in the WATSAN sector, which is co-financed by AusAid. In the transport sector, the Joint Infrastructure Unit (JIU) was discontinued in 2012 after only two years of operations. The JIU was a local initiative by AfDB in Malawi for closer cooperation between AfDB and WB for joint management of the infrastructure support portfolios (WATSAN and transport) of the two Banks.

Budget support has limited shared undertaking with the multitude of projects in some sectors, like PFM - where many donors intervene, and has continued to weigh heavily on the government as donors stick rigidly to their own procedures and there is no common reporting format. Even in the case of CABS, the Bank and other donors occasionally came up with their own specific requirements.

Managing for Results

This performance dimension evaluated the extent to which the Bank was successful in focusing on results in its management of day-to-day operations, and the extent of its support to national systems that focus on results. Managing for results was Unsatisfactory.

The Bank’s strategy and project documents contain results that are well stated in near- Results Based Management (RBM) terms or expected effects, but there are no intermediate results, a chain of execution or implementation in the key documentation. Bank monitoring focused on outputs rather than outcomes.

Analysis of transport and WATSAN sectors shows that CSPs and individual projects have a Logical Framework Matrix which specifies results indicators at activity, output, outcome and impact levels, but not a detailed project path analysis or Theory of Change logic including assumptions and risks.

In the agriculture sector, the Bank has often successfully implemented management systems that focus on process-based results and facilitate learning from experiences. The designs of both the SCPMP and CARLA projects include a Result-Based Logical Framework, which demonstrates the linkage between inputs, outputs, outcomes and impacts, and risk mitigation measures and key assumptions.

In terms of GBS and PFM, recent changes to the Program Based Operation (PBO) guidelines are designed to improve the extent to which these types of interventions are results-based. PBOs had causal chains that were poorly defined and confused different levels of results (outputs and outcomes) with the corresponding indicators.

The experiences from the transport and WATSAN sectors show that the Bank has broadly supported development of national capacities and management systems, but that focus has been largely on process and program/project implementation rather than outcomes, or overall capability to design and implement regulatory/ policy frameworks.

Knowledge Management

Knowledge Management was evaluated by assessing the extent to which the Bank learned from its experience. This was done by examining whether the factors affecting performance and results of the


Bank interventions were identified5 and informed6 its actions and strategies. Moreover, the issue of incorporating lessons from experience to inform future strategies and programs is also looked at. This performance dimension was Satisfactory.

The Bank has mechanisms in place to picture and record positive and negative factors affecting project performance. Lessons from experience are usually recorded in CSPs, MTRs, PARs7, PCRs and BTORs. However, there is no knowledge management system in place that makes storing, sorting, searching and retrieving such lessons possible and accessible via data mining or thematic searches. The utilization of the lessons included in the documents mentioned above is dependent on the availability of such documents or lessons. Notwithstanding, the Task Managers appear to have strong knowledge and experience of available knowledge in their respective fields. Cross-sectoral and generic lessons (pertinent to Bank-wide processes such as procurement and quality-at-entry), are repeated in consecutive reports though the underlying issues persist.

Lessons Learned

The following lessons are identified from the Bank’s program in Malawi:

1. Clear definition of beneficiaries. When the intervention beneficiaries are described in overly generic terms, they tend to be seen as a numerical target to be reached, and therefore the supervision and management of the projects tend to become input- and process focused, rather than outcome focused. Beneficiaries need to be as astutely described as possible so that the effects of the targeted ‘benefits’ can be monitored. Specific baselines are to be established upfront and constantly updated with valid information.

2. Results-based monitoring system. There is no benefit accrued by the Bank in generating results chains that are only partially managed. If

outcomes are the core objectives of investments, then monitoring and supervision of processes and indicators must be undertaken at all levels, including outcomes. With commitment to RBM, it is necessary to have a monitoring system that generates decisions focused on outcomes, and a solid analysis of results chains before project approval.

3. Facilitative conditions for policy dialogue. The Bank is well regarded in terms of its non-lending activities and produced results. The policy and program dialogue that has taken place over the years has also been well appreciated as a driver of change. Given that the GoM requires a considerable amount of support in order to accelerate its development, an important lesson is that the Bank has been able to influence and advantage policy when it has put in place the conditions that facilitated and supported dialogue. Increasing the quality and strategic nature of the dialogue will likely improve the level of influence. These conditions include developing a knowledge trust, promoting evidence-based decision making, developing the capability of institutions and agencies to design change strategies and monitor progress.

4. Timely decisions on design change. Major changes in project design, if required, should be introduced by taking into account not only achievable outputs but also continued project viability and relevance; this should not be postponed to the MTR if identified early during implementation.

5. Fiduciary safeguards and budget support. In a highly aid-dependent country, periodic withdrawal of budget support constitutes an external shock that aggravates macroeconomic instability and can cause unintended damage to growth and to poverty alleviation efforts. It is essential to strike a balance between promoting country ownership and using additional fiduciary safeguards that can prevent the sudden withdrawal of budget support.

7Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Recommendations

Based on the foregoing findings and lessons learned, the evaluation proposes the following recommendations:

1. Enhance capacity of the Bank’s officers and managers in Results-based Logical Framework and the use of Results-based Management tools, including theory-based design, results-based contracting, outcome-focused supervision and management of interventions. Each intervention should have clear identification of its target beneficiary characteristics and design the subsequent content and required management systems. This includes a baseline for monitoring, the focus of the M&E8 system, reference points for the RBM system, indicators for outcome measurement, contexts for project implementation, basis for the Theory of Change analysis, and basis for assumptions and risks.

2. Enhance the proficiency of the Bank staff to effectively operate in the business ecosystems in which key sector actors operate, and design interventions that are holistic enough to stimulate the private sector’s investment and actions. This is essential since most GoM development strategies rely on the operation of its private sector. As most of the business ecosystems are international in scope, the Bank actions in Malawi need to be international to some extent as well, with country programs developing ‘regional at least’ markets in a number of countries.

3. Since the Bank and Development Partners (DP) are pressing for effective and efficient PFM systems, the Bank should establish a clear timeline within which all required performance standards are met and proven. The GoM has been implementing PFM reforms for over 20 years, and therefore the Bank should identify what it should do if the systems cease to function during a specific period. The Bank could consider generating contingency

supports to overcome critical weaknesses without compromising transparency and mutual accountability and achieve development effectiveness.

4. The Bank should redesign its Knowledge Management (KM) practices to address the need for access to information and develop explicit and implicit knowledge in its Operations Divisions to improve decision making capabilities of the Bank and its clients. The Bank should build effective learning organizations by making learning routine as well as stimulating cultural change and innovation. It is therefore important that the Bank invest in modern KM technologies, develop its capacity and capability for data mining and use KM as an integral part of any implementation strategy.

5. The Bank should ensure sustainability of its interventions by taking concerted actions throughout the project cycle to ensure that impediments to sustainability are addressed and managed. From design through implementation, critical assumptions should be analyzed, and steps taken to mitigate risks. In situations where sustainability cannot be reasonably ensured, it is advisable not to approve such interventions in the future and ongoing interventions should be adjusted so that sustainability can be assured.

6. The Bank should take adequate measures to ensure that its interventions and strategies provide equality and equity between genders and that the interventions provide the means to manage, track and report on these objectives. Gender as a crosscutting issue should be included in project designs and mainstreamed into implementation. Beyond simply including women as a fraction of the participants in activities, they should be part of the management, and their needs – as being different from that of men – fully acknowledged. Project designs need to be clearly structured so that results are defined in terms of outcomes that bear on women and girls.

9Management Response

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Management Response

Management acknowledges the Independent Development Evaluation (IDEV)’s report on the Bank’s development assistance to Malawi covering the period 2005 to 2016. The period under review in this evaluation, covers the Bank’s assistance through three strategic cycles, consisting of two CSPs (2005–2009 and 2013–2017) and an ICSP (2011–2012) while also being cognizant of the inclusion of Malawi in the Regional Integration Strategy Paper (RISP) for Southern Africa 2011–2015. Management is pleased to note that the Evaluation finds as satisfactory the relevance and alignment of Bank’s past strategies to the development needs of Malawi. Management further notes with appreciation that Bank interventions have largely been effective in delivering satisfactory results in knowledge management, policy advice and integrating cross-cutting issues, despite challenging conditions in some instances, which have limited efficiency achieved in other areas. Management also broadly concurs with the other main findings of the evaluation, such as the limited sustainability and weaknesses in managing for results. At the same time, Management wishes to underscore the progressive improvements programme implementation achieved in most of the evaluated areas over the period. These achievements would have been impossible without the combined effort of the Bank and the Government as well as the positive support of the other development partners. Finally, the new CSP for the period 2018–2022 has been designed with the various issues identified in the Evaluation, and recommendations therein, in mind. In this regard, the evaluation report is timely and of great relevance.

Introduction

The Evaluation report provides a timely assessment of nine critical programmatic areas in terms of the relevance, effectiveness, efficiency, and sustainability, the management of crosscutting issues, knowledge management (KM) and policy advisory, partnerships harmonization and leverage, and managing for results of the past Bank strategies and associated programs in Malawi, and the lessons derived therefrom.

The evaluation provides detailed reviews of Bank interventions implemented in the various CSPs over the period 2005–2016. The report also includes assessments of thematic issues such as environmental impact, regional integration, financial governance and gender that have been integrated into the Bank’s strategies. Management notes that the main objective of the evaluation is to inform the next country strategy paper (CSP) and

to contribute to lesson learning within similar Bank programs. The evaluation exercise focused on both the strategic and program level interventions in all sectors where the Bank was active during the review period.

Relevance

Management concurs with IDEV’s satisfactory assessment of the relevance of the three broad strategies over the evaluation period. The CSPs and ICSP projects were well aligned with the Malawi Growth and Development strategies I and II priority areas of i) agriculture and food security, and ii) the Greenbelt Initiative and water development by focusing on irrigation and market infrastructure, resilience building and promotion of commercially oriented crops. The strategies were further aligned to the Bank’s corporate strategies including the Ten Year Strategy.


Effectiveness

The “satisfactory” assessment of this performance dimension in the IDEV Evaluation is in agreement with the general sentiment of Management and achievement of targeted development results outlined in the two PCRs that were scrutinized.

Sustainability

Management notes the Sustainability criterion’s rating as unsatisfactory and appreciates the importance of government commitment and capacity in ensuring sustainability. Management notes the emphasis on cost over-runs and inadequacy of resources beyond the project cycle.

At a strategic level, the Bank will ensure that issues of sustainability are addressed at project level through the national budgeting process, and making sure that operations and maintenance related costs are factored in the national budgeting process. Furthermore, in partnership with the sector Ministry and the Ministry of Finance, Economic Development and Planning, the Bank will work to ensure that the Medium Term and Expenditure Framework includes project related budgets.

Further, the Bank in Malawi will scale up its dialogue with Government (local and central governments), development partners, and beneficiaries to ensure ownership, mutual accountability, and a stronger link between Bank’s investment and Government longer and medium term expenditure framework.

Agriculture Sector: The unsatisfactory rating on sustainability with regard to financial sustainability, ownership as well as environmental and social sustainability in the sector. On financial sustainability, the Bank is addressing this by providing more resources to building the capacity of farmer cooperatives and associations that include management units comprised of professionals as is the case in SIVAP, promotion of market led agriculture, where smallholder farmers are linked to larger private sector companies, and

promote value addition for all commodities. In addition, Management has included an operation to establish an agricultural financing mechanism to enhance access to finance by the agricultural groups/individuals. In terms of ownership, detailed feasibility studies, which include participatory community participation, communication on critical issues such as land tenure, crop development, and markets, are addressed before the approval of a project. On Environmental and safeguards sustainability, Management is utilising or working with Environmental Specialists from the Bank and Government more than was the case in the previous operations. On the issue of underutilization of the infrastructure developed, Management has included an evaluation study on the impact of the irrigation investments in the non-lending operations under the new CSP. The outcomes from the study will inform future operations to avoid similar occurrences. To address the issues of sequencing of activities, cost determination and other issues affecting sustainability, Management encourages the use of PPF in project design. Such an approach results into timely implementation of activities and sustainability of the benefits.

Transport Sector: Management agrees with IDEV’s findings that benefits from past interventions in infrastructure projects have continued to exist (for longer than 5 years). By nature of the sector, infrastructure projects have a longer lifespan, especially when quality works are executed. Further, the Government of Malawi established the Road Fund Administration with a legal mandate to mobilise resources and collect levies for road infrastructure development and maintenance. Road levies are collected by the Malawi Energy Regulatory Authority and remitted to the Fund on road construction, rehabilitation and maintenance. The National Roads Authority was also established to manage all roads construction of rehabilitation works is well noted. These policy and legal initiatives have also helped Management and project teams in ensuring that road related projects are sustainable.

Water and Sanitation Sector: Management appreciates that sustainability has a profound impact on the water sector. Particularly, sustainability in terms


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

of project designs, resource base and sustainable use of the facilities after implementation completion. For instance, under the ongoing Rural Water Supply and Sanitation Project, 12 water user associations were formed and trained in readiness for managing the rehabilitated Gravity Fed Pipe schemes. Aspects of user fees for raising funds for maintenance and environmental catchment protection sustainability perspective towards climate change with tree planting have been integrated into the program. Other factors such as strengthening of the capability to monitor the resource base (groundwater monitoring station/hydro-met stations) will go a long way to ensure sustainability. As IDEV’s report emphasised the need for increased correlation with climate and bio-diversity, Management will mobilise specialised funds for addressing these issues. Already, the water sector is in the process of securing funding from GEF to complement the investment in the Rural Water Supply and Sanitation with the aim of improving climate change resilience of communities.

Social Sector: The limited sustainability noted in social sector operations calls for continuous and significant investments in the sector to improve the sustainability of achieved development outcomes. Under the new CSP, Water sector operations will integrate measures focused on improving sustainability by instituting mechanisms that cover operational and maintenance costs in Government strategies and budgeting.

Cross-cutting Issues

Management notes that the evaluation found the Bank’s objectives on inclusiveness across gender, regions as well as environmental sustainability to have been mainstreamed in Bank interventions and rated them Satisfactory.

Management wishes to inform the Board that crosscutting issues are strongly considered at a strategic level by the Bank and will continue to be so in the future. In both the new Malawi CSP and the new RISP for Southern Africa, gender is considered (e.g. disparities in access to services, decision

making structures, economic opportunities, cultural biases, budgetary allocation and monitoring of gender actions). Equally, environmental and social issues in Malawi will continue to underpin Bank interventions. The new RISP also notes that the Bank will take leadership in promotion of climate friendly infrastructure and reduced carbon emissions9 making specific reference to Malawi’s (and some other Southern African countries) problems regarding the effects of increasing climatic variability on infrastructure food security and development.

Management appreciates that Environmental and Social Impact Assessments (ESIAs) and Environmental and Social Managements Plans (ESMPs) are produced for all sector interventions in compliance with national legislation and in conformity with AfDB environmental and social policies. It is critical that all AfDB projects are categorized at an early stage in preparation and this categorization defines the expected nature of environmental (and social) impact and the mandatory mitigation treatment. The new CSP has put in place proactive resilience measures and with targets on carbon footprint reduction.

Efficiency

Management applauds IDEV for assessing the extent to which the resources and inputs provided through Bank’s support are converted to results at a rate that reflects assumptions at appraisal. More importantly, whether Bank’s interventions were implemented in a timely manner is also assessed. Management notes with concern that this performance dimension is rated Unsatisfactory and points to the fact that more needs to be done in this area. Management believes in being flexible in the use national systems. This approach remains in line with Government policies and directives. However, Management, just like the IDEV evaluation, notes that implementation was very weak in those projects where there were no PIUs. Serious delays in areas of procurement, financial reporting, disbursements, etc., were experienced. As such, a hybrid approach has been adopted whereby existing Government Departments mandated to coordinate projects are being supported


by short to long term Technical Assistance (TAs). In some cases, consultants are recruited specifically in areas of need such as procurement and financial management. This has helped in improving the status and ‘healthiness’ of the portfolio. Management shall continue using a hybrid approach with a view to build capacity, protect Bank resources, and ensure smooth implementation of projects through continuous portfolio improvement plans.

Agriculture Sector: With respect to the sector, Management has noted with equal sense of disappointment the unsatisfactory rating on efficiency. This might have arisen from a number of factors, which included inadequate preparation of the projects, capacity constraints, little resources spread over large geographical areas (e.g. SCPMP) thereby increasing administrative costs, frequent changes to implementing staff and limited knowledge on Bank’s procedures. Management in the new CSP, emphasizes having detailed feasibility studies with designs and tender documents ready prior to approval, developing a clear capacity development program, focusing on larger operations as is the case with SVTP to be implemented within particular geographical areas and organizing more regular training for implementing staff on fiduciary, project management and monitoring and evaluation.

Transport Sector: As acknowledged by Management above under effectiveness, cost overruns have not been serious challenges facing the sector in Malawi. This has largely been due to strong fiduciary requirements of the Bank including close supervision of the projects. Time overruns have been attributed to reasons stated under the same section (effectiveness). Management shall continue to support capacity-building interventions focussing on procurement, financial management and disbursement management with a view to ensure timely implementation of projects and that resources are put to good use.

Water and Sanitation Sector: Management wishes to note that the major challenge in terms of efficiency for the water sector emanates from the weak capacity in the public sector and decentralised nature of the

interventions demanding certain capacity at district level. The water sector has been mitigating this issue for rural/district interventions by keeping key project management functions at central level and designating regular staff to work on projects.

Social Sector: Management takes note of the various challenges that account for the “unsatisfactory” assessment of the performance in the social sector. In the implementation of the new CSP, it should be noted that the social sectors are no longer pillars of the new CSP, however lessons from here will be implemented under other sectors of the CSP.

Private Sector Development: Management agrees with IDEV’s assessment of the dismal performance of private sector development in the country during the period under review. It was due to: i) unstable macroeconomic conditions; ii) policy uncertainty; iii) shortages of power and general infrastructure bottleneck; and iv) weak business operating environment. Nevertheless, due to cyclicality and opportunistic nature of private sector projects, the Bank supported a number private sector projects directly and indirectly through regional operations and private equity (PE) funds. Management intends to compliment Government’s efforts of improving the enabling environment for private sector development through supporting the business reform agenda and infrastructure development, especially in the energy and transport sectors. Management will explore possibilities of working directly with the private sector operators especially in the energy sector through credit enhancement initiatives such as guarantees and direct lending to bankable private investment projects in the High-5 priority areas. Management intends to take full advantage of the African Investment Forum as a springboard for developing a pipeline of bankable private investment projects for possible financing.

Governance and PFM (multi-sector/PBOs): The Bank has been supporting implementation of PFM reforms together with other partners for some time. The niche for the Bank has been on procurement reforms and resource mobilisation through institutional support interventions and PBOs. Through PBOs, the Bank


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

also supported financial management, reporting and oversight reforms. The aim has been to reduce fiduciary risks associated with the various Bank supported interventions and protect Bank resources from abuse. Management shall continue intervening in these areas by mainstreaming PFM and governance interventions in its project designs.


Management is encouraged by IDEV’s findings that Bank knowledge and policy advice is satisfactory. This result was achieved because the Bank understood the need to engage regularly in policy dialogue with the government and donors on emerging issues. Such issues may affect the eco-system of development interventions in Malawi. Management is pleased that donors stressed the policy dialogue undertaken by the Bank was very useful and was always done in a spirit of collaboration, cooperation and support. Examples in GBS, health, education, transport and agriculture were provided. It is comforting to Management that many stakeholders had wished that the Bank would take on a leading role, however, given its ‘preferred relationship’ with the GoM, which considers the AfDB as ‘its Bank.’

Management also appreciates that fact that the Bank was consulted strongly during the preparation of all Government MDGS and the Government took some of the advice given by the Bank into consideration. These, point to the relevance of Bank and its role as a trusted partner in Malawi’s transformation agenda. It is therefore reassuring to note that the Government of Malawi’s decision to change its policy on farm subsidies and a subsequent move to a more private sector model was largely the result of the Bank’s policy dialogue with the Ministries responsible for agriculture, trade, irrigation and related sectors. Other examples include health policy on maternal care and content of MDGs. Because not all efforts at dialogue and advice have significantly influenced policy change, the new CSP will adopt innovative approaches to knowledge generation, management and the delivery of advice with greater focus on organizational learning.

Partnership, Harmonization and Leverage

Management is encouraged by IDEV’s findings that Bank interventions and resources are influencing other stakeholders to become involved in the sectors that are the focus of country strategies. It is therefore pleasing to Management that performance with respect to the partnership, harmonisation and leverage dimension is satisfactory.

Management concurs with IDEV’s findings that the Bank has been an active participant in several donor platforms such as the PFEMRP (for PFM) and the Common Approach to Budgetary Support (CABS) group. When the Bank and other partners were actively providing the budgetary support and before the emergence of the ‘cash gate’, the Bank co-chaired the CABS group and derived most of the targets (prior actions and triggers) from the Performance Assessment Framework (PAF). The PAF was contained a set of annual targets to be met by Government and were jointly agreed between Government and its partners. Due to diversity of sectors being supported by the partners and differing status levels of sector working groups (some were active, others inactive), Government and Partners could however agree, at times, to have some targets from outside the PAF. This was by mutual agreement and consent, taking into account the impact of such interventions. Management shall continue to work with other partners and Government with a view to promote partnerships and donor harmonisation. This is already happening through forums such as the PFM Donor Group, PFM Steering Committee, the High Level Forum, and Heads of Cooperation.


Management is glad that IDEV evaluated the extent to which the Bank has been successful in focusing on results in its management of the day-to-day operations as well as the extent of its support to national systems that focus on results. Management notes the Unsatisfactory rating and has put in place measures for improvement. Although Bank strategy and project documents contained outcomes and results that are


well stated in near-RBM terms (i.e. in terms of the effects expected), however there were no intermediate results or chain of execution or implementation in the key documentation. Bank monitoring and supervision do not focus on outcomes, but rather mostly on outputs. Management has mainstreamed and strengthened the Results Management Framework across both the new CSP and RISP. All projects in Malawi will also have portfolio performance improvement plans.

Agriculture Sector: Management acknowledges the IDEV rating as unsatisfactory under the managing for results, except for the cited examples in the agriculture sector. The monitoring system for the sector is based on the results based measurement framework which focussed on the results chain i.e. inputs, outputs, outcomes and impact. However due to delays affecting implemenation of the projects, the actual outcomes and impacts could not be effectively measured in some projects as projects tend to finish the infrastructure developmenst in the final years of the project. Management is, however, taking steps to address quality at entry in order to improve on the timeliness of project implementation. The process will support collection of adequate data for outcome measurements at an early stage.

Governance and PFM (multi-sector/PBOs): Managents also notes the finding that in terms of GBS and PFM, there has been weak causal chains relationships between different levels of results (output and outcomes) – and their corresponding indicators. This was partly due to weaknesses in PAF targets and lack of clearly defined targets, especially at output level. As such, Management undertakes to improve the design of such targets with a view to strengthen the linkages and ensure that the desired impacts are easily assessed or measured.

Social Sector: Management equally notes the results weaknesses highlighted in the review of the social sector as well. As such, more effort and resources shall be allocated to Results-Based Management (RBM) in social sector operations under the new CSP with a greater focus on results when conducting procurement,

contracting and supervision activities. Also, further RBM capacity building of social sector and PIU staff shall be conducted under new operations and the monitoring and results system of the Bank made more appropriate to RBM.

Lessons Learnt and Knowledge Management

Management appreciates IDEV’s observation that the Bank has developed a culture of learning in the various interventions and achieved tangible and satisfactory results. Management concurs with the findings that the Bank has mechanisms in place to record positive and negative factors that affect project performance. It is right and critical that lessons from previous experience are recorded in CSPs, MTRs, PARs, and PCRs as well as BTORs. The issue of incorporating lessons from Bank experience to inform future strategies and programs is thus wholly accepted as a recommendation.

The new CSP therefore adopts an innovative KM system to speed up the storing, sorting, searching and retrieving of such lessons via data mining, appropriate ESWs and/or thematic searches. The utilization of the lessons included in Bank documents is of course dependent upon the existence of requisite documents or lessons, and thus the need for relevant KM tools and strategies. Task Managers of the Bank are encouraged to continue having strong knowledge and experience of available knowledge in their respective fields. Management will keep the tradition whereby cross-sectoral and generic lessons (pertinent to Bank-wide processes such as procurement and Quality at Entry) are prominent not only in programme management reporting but in also eliminating the persistence of past underlying issues.

Conclusion

The new Malawi CSP 2018–2022 has been finalized and will be presented to the Board for consideration and approval in September 2018. The key findings and recommendations of the IDEV Evaluation have informed


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

the design of the new CSP. The new CSP operational priorities, being aligned to the High Fives and the third Malawi Growth and Development Strategy (MGDS III), will contribute to improving the various dimensions of Bank interventions in Malawi, particularly in fostering the effectiveness, relevance, efficiency, organizational learning, knowledge management, managing for results and sustainability of Bank interventions. The Bank intends to support the foundations for private sector led growth by investing in public infrastructure to unlock private sector investment, promote diversification and build economic resilience to reduce poverty and inequality. For example, the new strategy is designed to address issues

of sustainability by prioritizing economic, social and climate resilience in all future interventions. To reduce fragility and strengthen resilience to exogenous shocks, the interventions will address infrastructural bottlenecks to improve business creation and private sector led development, and drive agricultural transformation and small industry development to support diversification and (formal) job creation. The two pillars of the new CSP will support Malawi in expanding and closing infrastructure gaps to strengthen regional connectivity, increase market access and regional integration, and attract private investment; and strengthen economic diversification to build and improve resilience.

MANAGEMENT ACTION RECORDThrough the new CSP, Bank operations will have a transformative impact on agriculture in terms of increased productivity and production, value addition, and food security. Investments in the two pillars will also improve quality of people’s life, including women and the youth, by increasing access to quality and sustainable energy, water, transport and basic social services, employment, and food security. Altogether, focusing on High Fives, the new CSP ensures strong client ownership of Bank operations and their relevance to the country, thereby contributing to the effectiveness and sustainability of Bank operations. This Management Action Record indicates how the specific lessons and recommendation of the report have been addressed in the new CSP or at the corporate level.

Recommendation Management’s ResponseRecommendation 1: Enhance capacity of the Bank’s officers and managers in Results Based Logical Framework and the use of Results Based Management tools

This includes theory-based design, results-based contracting, and outcome-focused supervision and management of interventions. Each intervention should have clear identification of the characteristics of its target beneficiaries and design the content and required management systems. This includes the baseline for monitoring, the focus of the M&E system, the reference points of the RBM system, the indicators for outcome determination, the contexts for project implementation, the basis for the theory of change analysis, the basis for assumptions and risks.

Agreed. The 2018–2022 CSP and operations will strengthen the results agenda through the following innovations and measures:

❙ The CSP has a Results Based Logical Framework and Financing Facility, which provides indicators to be monitored to drive reforms in Malawi with an aim to enhance the portfolio's performance.

❙ Prioritise knowledge management and entrench a learning culture including those related Results Based Management.

❙ Build capacity of new staff such as those of is the recently established National Water Resources Authority and all project related implementation units as well as in the Bank.

❙ At mid-term, the Country Strategy will be reviewed and if needed, the strategic goals realigned with changes to Government priorities.

❙ At Strategy completion, a Completion Report will be prepared with a detailed performance analysis and the results achieved.

❙ At the portfolio level, institutionalized quarterly meetings with Executing Agencies and Ministry of Finance to track portfolio implementation and other portfolio related issues. In addition, regular follow-up will take place with Project Units.

In line with IDEV recommendations, management will implement the above actions through a coordinated effort of relevant Departments including PINS, SNFI (for Bank’s financial products), ECAD (for training of RMCs), SNOQ (for results, climate and result based management), PCER (disseminating impact). The new guidance note on Log-frame under development will also guide new Bank operations.


Recommendation Management’s Response

Management will also be launching a Task Manager Academy before the end of 2018 with a view to training all Task Managers, including on results based logical frameworks, with a compulsory certification. Additional details on this initiative will be provided in the forthcoming management response to the evaluation on quality assurance.

Recommendation 2: Since most GoM development strategies rely on the operation of its private sector, enhance the proficiency of the Bank staff to effectively operate in the business ecosystems in which key sector actors operate, and design interventions that are holistic enough to stimulate the private sector’s investment and actions.

Since most of the business ecosystems are international in scope to some extent, the Bank actions in Malawi need to be international, with country programs developing the “regional at least” markets in a number of countries.

Agreed. To enhance the proficiency of the Bank staff to effectively operate in the business ecosystem and design interventions to stimulate the private sector’s investment, management through PINS, COMW and RDGS will work with regional integration and other Bank Departments to enhance the knowledge of Bank’s officers in the Bank’s financial products, PPP, etc.

The Bank has also prioritised both regional integration and private sector development in Malawi. As examples, the medium to long-term development for the Transport sector is guided by the National Transport Master Plan 2017–37. The Plan identifies priority infrastructure to be developed and sector reforms to be implemented to make the sector more efficient and to increase impact.

The Bank’s support to the sector is focused on improving the Nacala Road Corridor and its related roads and infrastructure, with on-going projects being supported by the Bank and other development partners and a One-Stop-Border-Post. Transport sector interventions under in the Country Strategy will continue to support improving efficiency along the Nacala Road Corridor and expanding focus to the North South Corridor. The construction of One-Stop-Border-Posts will support regional integration. In addition to this, the Bank will support selected reforms in the sector as identified in the Master Plan. The CSP has also reflects the RISP, and is focused on the objective of regional transport and electricity connectivity/regional integration to neighboring countries. This focus will help Malawi strengthen trade as well as increase electricity supply that has been a huge challenge for private sector led development. The new CSP will further expand the support to the private sector development through value addition, market connection and enabling environment for private sector development through the following actions during the CSP period:

❙ Establishing eco-friendly renewable energy schemes linking SMEs to strategic industrial value chains, as well as scaling up agricultural commodity value chains;

❙ Addressing the challenge of low access to finance by combining public and private sector credit lines and guarantee schemes, to direct private sector lending for sustainable water and sanitation;

❙ Leveraging Bank’s knowledge by undertaking an ESW on the “Private Sector Profile” and Labour Force Survey and programs in support of enabling environment to promote competitiveness and broader private sector development, youth and women participation;

❙ Working with the GoM to redress issues related to legal and regulatory frameworks for promoting Public Private Partnerships.

❙ Contributing to promoting financial inclusion and improving financial sector efficiency through carrying out a financial sector diagnostic study


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


A roadshow on the Bank’s Private Sector financial products will also be organized in 2019. Responsible Departments: COMW and PINS.

Recommendation 3: Since the Bank and all development partners are pressing for PFM systems that are effective and efficient, the Bank should establish a clear timeline within which all required performance standards should to be met and proven.

The GoM has been implementing PFM reform for over 20 years, and therefore the Bank should identify what it should do if the systems cease to function during a specific period. The Bank could consider generating contingency supports to overcome critical weaknesses without compromising transparency and mutual accountability and achieve development effectiveness.

Agreed. Under the new CSP 2018–2022, the Bank has set out to strengthen Governance and the wider PFM accountability reform programme by mainstreaming it as a cross-cutting theme in all sectors under the CSP pillars that will be implemented under project implementation timelines. The Bank will continue to monitor the Government PFM Action Plan that focuses on the stringent management of financial economic governance.In line of the Division of Labour and building on the ongoing PFM support by AfDB, the EU and the World Bank are designing new

PFM interventions to support the Government of Malawi in undertaking reforms during the new CSP period. In line with this, the EU has taken over from the AfDB to Chair the PFM Working Group.

Specifically under the new CSP, the Bank will continue adopting innovative approaches by ensuring:

❙ Financial and economic governance, gender, skills and development, and environment and climate are mainstreamed as cross-cutting issues across all sectors.

❙ General Fiduciary Clinics are held on an annual basis. ❙ There are continuous improvements in the PFM legal and regulatory framework to ensure alignment with the constitutional provisions.

❙ The PFM Strategy implementation is well monitored including the PFM Action Plan that focuses on the stringent management of government bank accounts, cash management, commitment control systems, budgeting, internal audit, and arrears reduction and the enforcement of PFM operational guidelines.

❙ The Government during the course of the CSP promotes implementation of stronger Integrated Financial Management Information System (IFMIS) controls, undertakes payroll head count exercise, implements recommendation of payroll audit, steps up efforts towards procurement reforms as well enforcement of expenditure management; and increases staff numbers in the Public Financial and Economic Management Division for effective project monitoring and coordination.

As part of the CSP preparation, the Bank has also prepared the Country Fiduciary Risk Assessment (CFRA) to assess the inherent fiduciary risk associated with the use of national Public Financial Management (PFM) Systems as part of the Country Strategy Paper for Malawi (2018 – 2022).

Timeline: By 2022. Responsible Departments: COMW and ECGF.

Recommendation 4: The Bank should redesign its knowledge management practices to address the need for access to information, and develop explicit and implicit knowledge in its Operations Divisions to improve decision-making capabilities of the Bank and its clients.

The Bank should build effective learning organizations by making learning routine, and stimulate cultural change and innovation. It is therefore important that the Bank invest in modern KM technologies, develop its capacity and capability for data mining and use KM as an integral part of any implementation strategy.

Agreed. To address the need for access to information, and develop explicit and implicit knowledge in its Operations Divisions, management acknowledges the need to build effective learning organizations by making learning routines, stimulation of cultural change and innovation as well as effectively utilizing the modern KM technologies in the Bank. To this effect, the new CSP will enhance country dialogue through sharing findings of knowledge products, lessons learned from implementation of Bank’s program.



Furthermore, the CSP recognizes the need for effective learning strategies and tools at the sector and project levels. The new Malawi CSP incorporates this recommendation by:

❙ Supporting dialogue with Government to deepen knowledge sharing and learning;

❙ Generating new knowledge, prioritising knowledge management with relevant tools and entrenching a learning culture;

❙ Strengthening the labour market information system to improve policy decisions, and engaging in macro-fiscal and debt sustainability dialogue; and

❙ Prioritising capacity development for all programme and project staff.

Management has already committed to aligning its knowledge work with the client’s needs in the Management Response to BDEV’s 2016 Comprehensive Evaluation on Development Results. In particular, Management agreed to “1) Ensure that economic and sector work (ESW) is guided by a clear definition and that more attention is given to aligning ESWs with the Bank’s new operational priorities and client needs; 2) Revisit its knowledge products and organise them into three groups, namely: i) knowledge for external clients, ii) knowledge as a public good, and iii) knowledge for internal use; and 3) ensure that the Bank’s regional hubs play an important role in coordinating ESWs and disseminating them at the regional level”.

Recommendation 5: The Bank should ensure sustainability of its interventions by taking concerted efforts throughout the project cycle to ensure that impediments to sustainability are addressed and managed.

Right from the design through implementation, the critical assumptions should be analyzed and steps taken to mitigate the risks. When sustainability cannot be reasonably assured, it is advisable not to approve such interventions and the ongoing interventions should be adjusted so that the sustainability can be ensured.

Agreed. The CSP recognizes as a key lesson the need to mainstream sustainability issues at the sector and project levels. The Malawi government recognized the need to allocate more resources to maintenance of public investments and related development projects. The Bank is, working with other development partners to enhance donor coordination and development cooperation by managing for long-term results. Furthermore, the Bank is in continuous dialogue with government on increasing budgetary investment on the value chain and other transformative activities.

The new CSP incorporates this recommendation by:

❙ Making sure all infrastructure related investment projects are underpinned by rigorous sustainability analytics to inform the design of the infrastructure projects.

❙ The Bank will promote domestic resource mobilization for the sustainability of basic service delivery. Currently, the Bank is engaged with other DPs to strengthen the coordination of interventions and enhance crowd-funding.

❙ Reducing vulnerability to climate shock: Adverse weather events will continue to affect Malawi as long as the country is dependent on rain-fed agriculture. The Bank’s on-going support to Shire Valley Agriculture Transformation Project and other greenbelt irrigation projects aims to increase irrigated land by 2,000 ha in Malawi, and reduce dependency on rain-fed agriculture.

❙ Lowering dependence on external resources in the midst of global economic uncertainties. While the share of the budget financed from aid has reduced to 20 percent, it remains significant to broaden the revenue base to cushion against adverse aid shocks. Reinforcing private sector led growth remains the focus of the new Country Strategy as well as expansion of the tax base.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


❙ The Bank will maintain policy dialogue in the face of the 2019 elections on macro-fiscal issues, which could lead to fiscal slippages. Presidential elections may see a looser fiscal stance to attract voters, causing a reversal of the recent macro-fiscal stabilisation gains. The IMF’s three-year ECF program to Government will also foster macro-stability. Public financial management programs will also help monitor slippages.

❙ Reducing institutional risk: limited government funding affects service delivery, especially for the vulnerable. Protecting budgetary allocations to social services will ensure continued provision of social transfers, health, education and water and sanitation.

❙ Strengthening implementation capacity constraints that cause delays in reform implementation. The Bank has committed to the provision of TA through the CSP for capacity building, with support of other Development Partners in areas where weaknesses have been identified.

❙ Strengthening PFM as accountability mechanisms remain weak, but are slowly improving. Operations under the Country Strategy will incorporate fiduciary safeguards. Similarly, support to accountability and oversight institutions, public financial management reforms, and capacity-building projects will help mitigate risks and strengthen IFMIS expenditure controls. Focusing on High Fives will ensure strong client ownership; thereby contribute to the sustainability of Bank operations.

Timeline: Delivery of all the above actions is by 2022. Responsible Departments: COMW and ECGF.

Recommendation 6: The Bank should take steps to ensure that its interventions and strategies provide equality and equity between genders and that the interventions provide the means to manage, track and report on these objectives.

Gender as a cross cutting issue is always included in intervention design, but not often mainstreamed into implementation, other than as a fraction of the participants in activities. They are rarely part of the management nor are their needs fully described (i.e. as being different from men). Except in certain targeted projects, such as maternal health, project design is not clearly structured so that results are defined in terms of outcomes that bear on women and girls.

Agreed. To ensure Bank’s interventions and strategies provide equality and equity between genders, the Bank management will further strengthen the gender mainstreaming in all its operations during the design and these will be monitored at implementation and results reporting.

Going forward, the CSP recognizes as a key lesson: the need for continuous building of implementing agencies capacity to ensure program implementation provides equity and equality between genders. The recommendation is addressed in the new CSP this as follows:

❙ Through economic and sector work under the pillars, the Bank will focus on three mutually reinforcing areas as stated in the Bank’s Gender Strategy: legal status and property rights; economic empowerment; and knowledge management and capacity building. The Bank will set aside funds for studies, and capacity building to improve economic status of women.

❙ Continuous capacity building on gender mainstreaming will be conducted by the Bank for Executing Agencies and Implementation Units in the areas of Financial Management Reporting, Disbursement and Procurement on a case-by-case basis.

❙ Strengthen the Results Logical Framework with specific gender related indicators

Timeline: Delivery of all the above is by 2022. Responsible Departments: COMW and all Task Managers responsible for projects

Phot

o: ©

AfD

B

21Introduction

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Introduction

This report summarizes the findings, lessons and recommendations of the independent evaluation of the African Development Bank’s (AfDB or Bank) Country Strategy and Program in the Republic of Malawi during 2005–2016. The main objective of the evaluation is to inform the next Country Strategy Paper (CSP) and to contribute to both accountability and learning in the Bank and in countries where the Bank’s program is similar to that in Malawi.

The evaluation focused on both the strategic and individual interventions levels in key sectors (Social, Transport, WATSAN, Agriculture, and Multisector-Financial Governance and Private Sector Development) where the Bank is active. In addition, the evaluation looked at the achievement of development results by answering evaluation questions based on four of the five evaluation criteria suggested by the Development Assistance Committee of the Organization for Economic Cooperation and Development (OECD–DAC), those of relevance, effectiveness, efficiency, sustainability and impact, with impact not a required criterion for this evaluation.

Additional AfDB-specific criteria (viz. the management of crosscutting issues, knowledge management and policy improvement, partnerships and harmonization, management by results and lessons learned) were added in order to cover the design and management-related

elements and show the benefit and the strategic positioning of the Bank in Malawi.

The evaluation methodology is based on the use of an evaluation matrix detailing the evaluation questions, judgment criteria (JC) and indicators as the building blocks for the analysis leading to conclusions and recommendations. The methodology focusses on results achieved and demonstrated, a Theory of Change approach to intervention design and management, and a sector-by-sector analysis followed by a roll-up of sector level findings to the program level, evidence-based research, contribution analysis and iterative feedback with key stakeholders to strengthen the validation bonds. The evaluation process adopted a systematic approach that uses different building blocks to answer the evaluation questions and to formulate key conclusions and recommendations. The methodology applied for this evaluation is based on the evaluation criteria developed by OECD–DAC. The detailed evaluation methodology and limitations are presented in Annex A.

This report is laid out in six main sections: i) Malawi Government development strategies 2005–2016; ii) the development assistance context of Malawi; iii) the Bank’s strategic support to Malawi during 2005–2016; iv) evaluation findings; and vi) lessons learned with recommendations.


The Development Assistance Context of Malawi

The Republic of Malawi is a landlocked country in South-East Africa, sharing borders with Tanzania, Zambia and Mozambique. Divided into 28 districts and three regions (north, central and south), Malawi has a total surface area of 118,484 sq. km and an estimated population of 16.4 million in 2015,10 with 67.0% of the population being under 24 years of age, only 5.7% over 55 years and a population growth rate of 2.8%.11 The capital is Lilongwe (population 905,00012) and the other major urban areas are Blantyre, Mzuzu and Zomba. Although approximately 80% of the population lives in rural areas, the annual urban growth rate is about 3.8%, thus making urbanization a real challenge.

The tables below and on the right provide an overview of the Official Development Assistance (ODA) that the country has received during the 2007–2015 period and the sectors of destination for that support. It may be noted that half of the ODA was directed at education or health.

The African Development Fund (ADF) is the seventh largest ODA partner, a long way behind the World Bank, International Development Association, the United Kingdom and the Global Fund, among others.

Table 1: Destination of Bilateral Aid 2007–1512

Sector Percentage

Health and Population 36%

Other social infrastructure 10%

Economic Infrastructure 9%

Production 17%

Education 13%

Multisector 5%

Program Assistance 3%

Humanitarian Aid 7%

Action relating to Defense Negligible

Other and unallocated 1%

Table 3: Government of Malawi Receipts 2013–2016

2013 2014 2015

Net ODA (USD million) 1,132.7 931.4 1,049.4

Net ODA/GNI (%) 21.1 15.8 16.5

Gross ODA (USD million)

1,176.1 1,030.6 1,104.0

Bilateral Share (Gross ODA) (%)

56.1 54.4 53.7

Net Private Flows (USD million)

444.6 35.5 0.8

Total Net Receipts 1,577.8 966.0 1,049.3

Source: OECD.org/dac/financing-sustainable –development /development-finance-data/aid-at-a-glance.htm

Table 2: Main donors and their disbursements in US$ over the last three years (2014–2016)

Main Donors Actual Disbursements (US$ million)

World Bank 372.4

Department for International Development

216.0

Global Fund (HIV/AIDS & Malaria) 189.3

African Development Bank 183.5

Norway 171.4

European Union 155.2

Centers for Disease Control and Prevention

90.2

Source: Malawi Aid Management Platform

23The Development Assistance Context of Malawi

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The population of Malawi stands at 18.1 million in 2016, up from 13 million in 2005 and 11.4 million in 2000. The ratio of net ODA received as a percentage of Gross National Income (GNI)

is a key indicator of the level of dependence of the country on external grant financing. Table 4 provides an overview of ODA/GNI ratio over selected years.

Table 4: Evolution of net ODA received as a percentage of GNI-Malawi

Year 1960 1980 1990 1994 2000 2005 2008 2012 2015ODA/GNI Ratio 2.6 12.4 27.2 41.4 26.3 15.9 17.5 19.8 16.9

Source: http://data.worldbank.org/indicator/DT.ODA.ODAT.GN.ZS?locations=MW


Malawi Government Development Strategies

In its Vision 2020, the GoM is committed to making the country ‘secure, democratically mature, environmentally sustainable, self-reliant with equal opportunities for and active participation by all, having social services, vibrant cultural and religious values, being a technologically driven middle-income country.’ To tackle the development

challenges under the evaluationperiod, three national policy documents were in place. The documents defined the overarching strategies of the country in order to accelerate the attainment of its long-term development aspirations and the MDGs. The main objectives of the three strategies are presented in table 5.

Table 5: Malawi Government’s strategies and priority areas

Strategy Objectives and Priority Areas2002–2005 Malawi Poverty Reduction Strategy & Plan (MPRSP)

The objectives of the MPRSP 2002–2005 included: ❙ a united, secure and democratically mature nation with full participation of the population; ❙ access to quality health and education for all; ❙ sustainable growth and development with a per capita income of USD 1,000 by 2020; ❙ a technology-based, manufacturing-led economy generating 25% of Gross Domestic Product (GDP); ❙ a fair and equitable distribution of wealth; ❙ a sustainably managed environment; and

❙ food security for all.

This strategy had a three-pronged approach:

❙ emphasize smallholder agriculture to raise the productivity and income of the rural poor; ❙ promote private sector growth to expand off-farm employment through a number of measures including

reduction of transport costs; and ❙ expand social services.

2005–2011 Malawi Growth and Development Strategy I (MGDS I)

Overall MGDS I priorities:

The overriding philosophy of the MGDS I was “poverty reduction through sustainable economic growth and infrastructure development.” The MGDS I identified six key priority areas, which defined the direction the country intends to take in the next five years to achieve economic growth and wealth creation because, it is noted, these are “critical for immediate improvements in the economic wellbeing of Malawians.” These priority areas were:

❙ agriculture and food security; ❙ irrigation and water development; ❙ transport infrastructure development; ❙ energy generation and supply; ❙ integrated rural development; and ❙ prevention and management of nutrition disorders, HIV and AIDS.

2011–2016 MGDS II13 Overall MGDS II priorities

The MGDS II was structured somewhat differently compared to its predecessor with the top level of analysis being the “theme,” and only the second being “priority,” thus providing a more focused approach.

The MGDS II identified six thematic areas, namely; ❙ Sustainable Economic Growth;

25Malawi Government Development Strategies

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Strategy Objectives and Priority Areas

❙ Social Development; • Nutrition;

• Population; ❙ Social Support and Disaster Risk Management; ❙ Infrastructure Development; ❙ Governance; and

❙ Gender and Capacity Development.

The MGDS II pointed to nine key priority areas from the themes. Much as what was done in MGDS I, these were considered “necessary to achieve rapid economic growth and improvement in the well-being of Malawians” within the implementation period (quotation from MGDS II)

The key priority areas were:

❙ Agriculture and Food Security; ❙ Energy, Industrial Development, Mining, and Tourism; ❙ Transport Infrastructure and Nsanje World Inland Port; ❙ Education, Science and Technology; ❙ Public Health, Sanitation, Malaria and HIV and AIDS Management; ❙ Integrated Rural Development; ❙ Green Belt Irrigation and Water Development; ❙ Child Development, Youth Development and Empowerment; and ❙ Climate Change, Natural Resources and Environmental Management.


AfDB’S Strategic Response for Malawi (2005–2016)

Overall Strategy

The AfDB’s cooperation with Malawi dates back to 1969. The Bank’s Malawi Field Office was established in 2007 as a result of the Bank’s decentralization. According to government officials, PIU coordinators and the Bank staff, the establishment of the field office has contributed to the enhancement of the Bank’s portfolio management, country dialogue and aid coordination.

The AfDB country strategies largely corresponded to the two principal MGDSs. The CSP 2005–2009 and ICSP15 2011–2012 corresponded to MGDS I (2005–2011) with a planning construct that began to take the direction of the GoM in its MGDS II into account. The CSP (2013–2017) corresponded to MGDS II (2011–2016).

For the period under review in this evaluation, the Bank’s assistance encompasses three strategic cycles, consisting of two CSPs (2005–2009 and 2013–2017) and an ICSP (2011–2012). In addition, Malawi was included in the RISP for Southern Africa 2011–2015. The CSPs provide a multiannual framework and a guide for the Bank’s interventions in order to support the country’s development agenda and priorities and operate in synergy with them. The RISP aims at providing support to member countries and Regional Economic Communities (RECs) in designing and implementing measures to tackle the regional infrastructure gap and accelerate economic integration.

The 2005–2009 CSP had three core principles: i) support the implementation of the new MGDS to achieve its goals and help eventually lead to the attainment of Malawi’s Vision 2020 goals and MDGs; ii) provide a clear focus on results and outcomes to improve the rate of return on the Bank Group’s interventions; and iii) improve the coordination of Bank

Group activities with government and other donors. The Bank Group’s interventions were to continue by providing project– and policy–based support.

The AfDB strategy was thus structured around three pillars:

1. Expanding rural infrastructure (increased levels of irrigation infrastructure and power supply to support agriculture, rural development and Private Sector Development or, PSD);

2. Developing human capital and institutional capacity (enhanced skills through better education, improved health service delivery and strengthened institutional capacity); and

3. Improved policy and institutional framework for Governance (support of policy and institutional reforms for good governance).

The preparation of an Interim Country Strategy and Program was agreed with the GoM in order for the Bank to align with the timeframe for the preparation of MGDS II, which was expected in 2012. This allowed the Bank to effectively align its next strategy from2013 onwards to the country’s new poverty reduction strategy, i.e. MGDS II. The ICSP 2011–2012 focused on two pillars:

1. Improving infrastructure to help remove infrastructure bottlenecks hindering Malawi’s competitiveness and to open access to regional markets by reducing transport costs and improving trade competitiveness; and

2. Accelerating PSD to help create an enabling environment for business growth and innovation focusing on the ‘missing middle’ in the private

27AfDB’S Strategic Response for Malawi (2005–2016)

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

sector supporting value chain integration, improving business infrastructure and access to finance.

The CSP 2013–2017 responded to GoM priority focus areas identified in the MGDS II and the Economic Recovery Plan 2012, focusing on areas of sustainable economic growth, infrastructure development and improved governance under two pillars, namely:

1. Address infrastructure bottlenecks to competitiveness and growth in order to address infrastructure constraints to service delivery, private sector growth and competitiveness supporting interventions in transport, energy, water supply and irrigation (including addressing regulatory and market challenges in infrastructure services and Malawi’s regional integration agenda); and

Figure 1: Evolution of the pillars of the Bank’s CSPs for Malawi (2005–2016)

Pillar 1: Regional

infrastructure

• Regional transport/trade facilitation infrastructure

• Regional energy development

• Information and communications technology

Pillar 2: Capacity building

• Support to the CES tripartite agreement

• Institutional strengthening

• Trade and transport facilitation and aid for trade

Cross-cutting

• Regional public goods (gender, environment and climate change)

• Knowledge management and networking

• Macroeconomic convergence

Pillar 2: Support actions to

expand private sector investment and trade

• Skills development for entrepreneurship

• Addressing impediments to trade expansion and business

• Economic governance

Pillar 1: Address infrastructure

bottlenecks for competitiveness

and growth

• Transport

• Energy

• Water supply

• Irrigation development

Pillar 1: Expanding rural infrastructure

• Irrigation

• Water supply and sanitation

• Electricity supply and access

Pillar 3: Improve the policy and institutional framework

for good governance

• Improve macroeconomic stability

• Improve budget implementation

• PFM reforms

• Accounting/auditing

• Tackling corruption

Pillar 2: Developing human

capital and institutional capacity

• Education

• Health

Pillar 1: Improving

infrastructure

• Agriculture

• Water

• Transport

Pillar 2: Accelerating private sector development

• Higher education for skills development

• Business development

CSP 2005–2009 ICSP 2011–2012 CSP 2013–2017 RISP 2011–2015


2. Support actions to expand private sector investment and trade including policy reform and actions to improve the economic and business environment (such as improved PFM, trade and regional integration and improved financial services for Small and Medium Enterprise.

This third CSP is unique in that it specifically added a focus on regional strategies. It notes, for instance, that “...to ease challenges posed by Malawi’s landlocked position, the Bank will scale up support to regional infrastructure to deepen the country’s integration with its neighbors.”

The Southern Africa RISP 2011–2015 has a primary focus on enhancing and advancing development of key corridors with the objective of fostering and sustaining economic growth and social integration by means of increased intraregional trade, greater legal, institutional and regulatory harmonization and lower production cost structure. There are two strategic support pillars as follows:

1. Regional infrastructure focusing on environment and climate friendly infrastructure programs in

transport, energy as well as Information and Communication Technology; and

2. Capacity building in support of infrastructure focusing on supporting the CES Tripartite Arrangement as well as developing joint frameworks to improve regional coordination by means of strengthening institutional capacity of RECs and national implementation units for implementation of infrastructure programs and trade facilitation activities.

As shown in Figure 1 above in all three strategic cycles, there has been a focus on infrastructure: transport at the national and regional levels, energy, water supply, irrigation and communication. In the first one cycle, the other major area of intervention was human capital and institutional capacity development, with a shift to PSD, investment and trade from 2011 onwards. The importance given to regional integration increased in the same period with the approval of the RISP. In addition to the projects, the Bank’s strategy in Malawi includes a range of non-lending activities, mainly PD, knowledge and analytical work and donor coordination, aid harmonization and alignment.

0

10

20

30

40

50

60

70

80

20162015201420132012201120102009200820072006

Net loan (UAC) Disbursement (UAC)

Mill

ions

Figure 2: Amounts (UA million) committed and disbursed by year in Malawi (2006–2016)

Source: AfDB SAP data


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

AfDB Portfolio

Lending operations

The analysis covers all projects implemented Bank in Malawi during the period 2006–2016. Over the evaluation period, the committed funds amounted to UA 428 million for a total of 36 projects, (Annex C), and the corresponding disbursements amount to UA 275 million. The total contribution per year amounts to an average of UA 36 million; however, from 2010 the funds have sharply decreased reaching UA 3 million in 2011 (Figure 2). The disbursed amount covers 64% of the total committed amount, however, for the period 2006–2011, disbursement almost equaled

commitments while, from 2012, there is a normal decrease due to ongoing and approved projects. Notably in 2016, the disbursed amount is only UA 30,688 corresponding to one project in the Agriculture sector.

In terms of sectoral distribution, the AfDB contribution is distributed among all major sectors with a particular focus on the social sector which accounts for 24% of total funds with the exception of the power sector which received less than 1% of the total committed amount (Figure 3).

Table 6 below provides data on the status of the projects. At the time of the evaluation, there were 16 closed and completed projects, and 15 projects were ongoing and 5 at the approval level.

Social

24%

Transport

21%

Agriculture

22%

Multisector

17%

Water and sanitation

16%

Power

<1%

Figure 3: AfDB commitments by sector in Malawi (2006–2016)

Source: AfDB data

Table 6: AfDB Commitments by status and sector

Status Commitment (UA million)

Sectors Total

Agriculture Transport Power WATSAN Social Multisector

Approved 28.0 3 1 1 5

Ongoing 208.4 3 2 1 2 5 2 15

Completed 158.1 2 2 2 1 4 11

Closed 33.6 2 2 1 1 5

Total 428.2 10 5 1 5 8 7 36


Non-lending operations

In addition to lending activities, the Bank also conducted, within the framework of CSPs, non-lending activities- i.e. Economic and Sector Work (ESW), and other studies and initiatives supporting the implementation of CSPs. These facilitated high level PD with the GoM and provided analytical evidence for future programing cycles. These products are developed in collaboration with various operational and thematic departments of the Bank, often in partnership with other donors.

For analytical purposes, ESW and other non-lending activities can be grouped into three main categories.

1. In-depth studies and sectoral reviews specifically aimed at helping in building the information/knowledge base to underpin the Bank’s lending operations and to inform the PD process. It includes both country studies and regional studies with case studies on Malawi relevant to this evaluation;

2. Periodical publications and bulletins, e.g. the African Economic Outlook. These contributions typically compile secondary data that are analyzed in greater detail in other documents.

3. Other factsheets and brief overviews, such as summary brochures of the Bank’s interventions in a given sector, or brief chapters in general publications.

In addition to financing instruments and ESW, the Bank actively engaged in PD with various Ministries, Departments and Agencies, both bilaterally and in the framework of thematic working groups. Benefiting from country presence since 2007, the Bank’s visibility and quality of dialogue have improved. The main issues for PD highlighted in CSP 2013–2017 included:

❙ Portfolio Management: dialogue with GoM to address the portfolio management and performance constraints.

❙ Economic Governance and Business Climate: PD to improve the business environment. The main issues include improving the regulatory and institutional environment for key services, deepening financial services, improving institutional capacity for trade promotion and export diversification, and maintaining a good macroeconomic environment.

❙ Urban Development and Population growth: In view of the high population growth and

Box 1: List of ESW conducted during the evaluation period

2012 Private Sector Profile and the Joint Country Analysis

2012 Evaluation of Public Financial Management Reform

2012 African Economic Outlook

2011 Policy brief on Economic Management and evaluation

2011 Public Expenditure and Financial Accountability Assessment (PEFA)

2010 Education and Employment in Malawi

2009 Country Economic Memorandum (CEM)

2009 Skills for Private Sector Development

2009 Case study on Malawi for enhancing Linkages with vertical Health funds

2005 Multi-sector Country Gender Profile

2005 Agricultural Sector Review


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

urbanization rates, the Bank engaged with other partners, to articulate measures to meet these challenges.

❙ Joint Assistance Strategies: The ongoing Joint Country Analysis being undertaken by DPs in Malawi was an opportune platform to move beyond discussing co-financing arrangements and joint implementation units towards developing joint assistance strategies with other donors. The Bank explored this during the CSP period and discussed the strategy with various donors.

Regional – national program interfaces

Malawi is a landlocked country with high transport costs and international journey times — only very recently has Malawi become a credible transit country (from Zambia to ports in Mozambique). In acknowledgement of this situation, the CSP 2013–2017 (and SA RISP 2011–2015) emphasized a regional focus to strengthen regional operations and movements by way of hard and soft infrastructure (in transport and energy) to reduce transit times and costs, and enhance connectivity to facilitate increased regional and international trade. This support identified rehabilitation of sections of the

Nacala Road Development Corridor and provision of two one-stop-border-posts (OSBPs) connecting Malawi, Mozambique and Zambia; the development of the Mtwara Corridor connecting Malawi, Tanzania, Zambia and Mozambique; and supporting the Sena rail line connecting Malawi and Mozambique. At the time of this writing, implementation progress has been limited — the corridor road links are well advanced and a Bank feasibility study has been undertaken for restoration of the washed-out Sena rail line. However, construction of the OSBPs has not yet started and support to the Mtwara corridor has not advanced beyond the concept level. The Bank’s strategy assumes that investment in transport infrastructure under national programs will provide the building blocks for regional connectivity (with associated support to facilitation issues such as border crossing procedures) but such national programs respond to national rather than wider regional priorities such that efficient implementation of cross-border regional programs is at risk as neighboring countries do not share a common vision of priorities. For example, Mozambique does not share Malawi’s sense of urgency for the development of OSBPs, the Sena rail line or the Shire/Zambezi Waterway. The Bank’s regional strategy is not effective in providing complementarity coordination with the national strategies for Malawi.

Phot

o: ©

AfD

B

33Evaluation Findings

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Evaluation Findings

The performance of the Bank’s program in Malawi in each of the evaluation criteria assessed is rated using a four-point scale. The ratings awarded in this evaluation are at a program level (overall/synthesis) but rely on the information and analysis contained in the sector reports, and ultimately from individual interventions (see Annex D). The four-point rating scale is briefly defined in the table below:

Relevance

This performance dimension assessed the extent to which the Bank’s country strategies and operations were aligned with Malawi’s development needs, strategies and priorities as well as the needs of ultimate beneficiaries. The relevance of the Bank strategy and assistance to Malawi was rated Satisfactory.

The Bank strategic areas were well aligned with GoM priorities. Overall, key national priority sectors were covered by the Bank’s strategies even though over a time span different from the MGDS. For instance, in terms of priority areas, the transport sector was well covered by the Bank starting from the ICSP 2011–2012 under Pillar I “Improving Infrastructure” (Figure 4), while

it was already a priority for the government in the MGDS I (2005–2009). By contrast, the Bank strategy included within the priority areas PSD and good governance interventions while these were considered by the GoM as overall thematic areas for development but not included under priority areas for intervention. The Bank’s strategic decisions in good governance, which were addressed in all of the Bank strategies, were in line with existing diagnosis and analysis of the context, which showed that Malawi needed to promote good governance to achieve its poverty reduction goals. The Bank therefore filled in key strategic gaps and helped to further define, in practical terms, GoM strategies and priorities. This is particularly relevant to regional development in the transport sector, where the regional actions required for the development of the transport in Malawi were essentially defined through the interaction of the Bank and RECs such as Southern African Development Community.

In each Appraisal Report, the Bank has tied the justification and design of its interventions directly to one or more of these policies and strategies. This was done by specifying how the Bank’s policies and strategies related to GoM sector strategies. In many cases, however, the justifications were not tied to GoM’s expected outcomes but to overall policy and strategy statements (e.g. HEST and LED social projects). Some GoM officials also believe that while the Bank’s project designs are relevant, some thought should be given to bigger but fewer projects, making the bigger scale more relevant to the achievement of the outcomes and therefore to the needs of the beneficiaries.

Similarly, the Bank’s provision of GBS within the CABS framework has broadly reflected government’s own priorities, as laid down in the MGDS and other policy documents; however there were at times some deviations or inconsistencies in the choice

Table 7: Rating scale and description

Rating value Represents Brief description4 Highly

SatisfactoryExpected performance was exceeded

3 Satisfactory Expected performance levels were met

2 Unsatisfactory Most performance levels were met but a few were not

1 Highly Unsatisfactory

Most expected performance levels were not met


of performance indicators (in multisector projects), weakening the linkage between the Bank and the GoM.

The interventions at sector level were aligned with the country strategies. The analysis of project documents throughout the sectors indicates that the operations contain analyses that compare country strategies to the results framework of interventions; thus clearly showing that the Bank is aligned at the sector level with the strategies for Malawi. That is to say that, for instance, the Education V project was aligned to the education strategy and the Health SWAp project was aligned to the National Health Policy and a Program

of Works. The analysis also shows that at a higher level, the Bank interventions were also in line with the strategies laid down in the CSPs.

Nevertheless, the documentation that defined interventions do not compare the intervention result framework to that of the CSP. In part, this is because the CSPs are not specific enough in their operations descriptions to do that. However, a majority of the PARs refer to a coherence between the pillars-thrust of the CSPs and the objectives/targets of the project (establishing a high level alignment).

Figure 4: AfDB CSPs and Malawi MGDS priority areas

Pillar 2: Support actions

to expand private sector investment

and trade

• Skills development for entrepreneurship

• Addressing impediments to trade expansion and business

• Economic governance

Pillar 1: Address

infrastructure bottlenecks

for competitiveness and growth

• Transport

• Energy

• Water supply

• Irrigation development

Pillar 1: Expanding rural infrastructure

• Irrigation

• Water supply and sanitation

• Electricity supply and access

• Agriculture and food security

• Integrated rural development

• Irrigation and water development

• Transport and infrastructure development

• Energy generation and supply


• Energy, industrial development, mining and tourism

• Integrated rural development

• Climate change, natural resources and environmental management

• Transport, infrastructure and Nsanje World inland port

• Education, science and technology

• Public health, sanitation, malaria, HIV and AIDS

• Child development, youth development and empowerment


Pillar 3: Improve the policy and institutional

framework for good governance

• Improve macroeconomic stability

• Improve budget implementation

• PFM reforms

• Accounting/auditing

• Tackling corruption

Pillar 2: Developing

human capital and institutional capacity

• Education

• Health

Pillar 1: Improving

infrastructure

• Agriculture

• Water

• Transport

Pillar 2: Accelerating private sector development

• Higher education for skills development

• Business development

CSP 2005–2009 Pillars

MGDS 2005–2011 Priority Areas

ICSP 2011–2012 Pillars

MGDS 2011–2016 Priority Areas

CSP 2013–2016 Pillars


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Of particular interest are the considerations for the PSD pillar. Although the Bank is including PSD in its strategies, it has made little progress in developing a PSD portfolio. As a result, there has been a slight disconnect between the Bank’s past and present portfolio and the goals and objectives stated in its CSP.

Specific beneficiaries were rarely identified in CSPs and RISP though there were references in project documents. Analysis shows that by defining the “beneficiary” at a high level and by profiling beneficiaries into a very small number of types, the Bank’s operational designs contained targets for beneficiaries, even if these were not characterized or broken down further in terms of expected effects within sub-samples. The beneficiaries defined in the CSPs tend to be generic, as one would expect from a document that is designed to show strategic alignment and only point to intentions for interventions by sector.

The Bank’s particular concern in ensuring that its operations are aligned with the needs of the population can be demonstrated by the Bank’s decision to put particular emphasis on social sectors, while designing its PBOs in 2007, and then again in 2012 (with Restoration of Fiscal Stability and Social Protection (RFSSP)), 2015 (Protection of Basic Services - PBS), and in 2016 with Food Crisis Response Budget Support (FCRBS). On two occasions, the Bank used the Crisis Response Budget Support modality—first to protect the population from the adverse social impact of the Kwacha devaluation in May 2012 (RFSSP, 2012), then to contribute to the Government’s Emergency Response Plan to mitigate the food crisis (FCRBSP, 2016).

Effectiveness

Effectiveness is evaluated by assessing the extent to which Bank’s interventions in Malawi achieved their expected development results (outputs and outcomes) against targets set in project

approval documents. Additionally, effectiveness is also evaluated by ascertaining the extent to which Bank interventions benefited targeted beneficiaries. This performance dimension is rated Satisfactory.

The portfolio representing the Bank program in Malawi during the timeframe under review, was made up of 36 projects, for which a total of 11 PCRs were available. Bank PCRs reflect a satisfactory level of performance both for output and outcome level achievements. However, it may be noted that the PCRs take into consideration post- MTR definition of expected results. For the majority of PCRs, the MTRs have reduced the expected output and outcome targets considerably.

In the agriculture sector, of the two sector projects that were closed during the review period, expected effects were clearly stated and monitored, and achievement levels were stated. For the SCPMP, there was a 54% overall achievement for irrigation development, a 102% achievement for farmer support program and 100% achievement for project coordination and management. About 85% of the schemes and 74% of the Market Centers (MCs) and Community Storage Facilities (CSFs) were only completed after Project Year 4, leaving little time for farming activities. 24 out of 26 CSFs and 13 out of 14 MCs were developed. The remaining two CSFs and 1 MC were among the contracts, which were terminated. Only 59% (23 out of targeted 39) of Water Users’ Associations were formed and registered.

In the transport sector, project progress reports and two PCRs were scrutinized. All Bank support interventions for provision (or rehabilitation/upgrading) of road transport infrastructure have delivered expected outputs (in some cases at additional time and cost, and in some cases at lower than estimated cost). Overall, the physical outputs of the interventions were well achieved in the sector. The beneficiaries (private sector firms


needing transport, consumers of transported products, etc.) were not monitored to see if their needs were met. The information on “meeting needs” is limited to the opinions of government officials and the consultations with key stakeholders such as the Chambers of Commerce. At a higher level the Bank monitors CSP implementation (e.g. MTR, Supervision Missions) and measures progress of implementation of MGDSs (e.g. IMF Joint Staff Appraisals) but weak institutional capacity of line ministries and agencies involved in transport jeopardizes ongoing and real-time M&E, policy analysis and eventually decision making. In addition, the sector beneficiaries are mostly considered to be those who need to use roads to make a living (e.g. transport firms) while the other beneficiaries are not sufficiently taken into account.

For the WATSAN sector, sector targets for outcomes and impacts were set in general terms in CSPs, responding to the MGDSs targets. All sector interventions specified expected results (outputs, outcomes and impacts) in response to higher level strategies. Monitoring indicators were defined and logical linkage presented in the Logical Framework even though the translation of outputs into higher level outcomes and impacts was not clearly discussed. However, while there was some monitoring of direct project-related outcomes there was little or no monitoring of the achievement of target impacts, as no ex-post project evaluations have been carried out. It is thus difficult to accurately state to what extent the Bank interventions have achieved the expected outcomes and impacts or development objectives.

The Bank interventions have benefitted target group members (though not necessarily to the extent expected at project design). Monitoring systems were established for individual projects (although baseline information against which change could be measured was not always available or of good quality). Monitoring systems did not deal with ultimate beneficiaries in large part because most monitoring was restricted to activities and outputs.

The Bank’s interventions partly benefited the target group in the agriculture sector. The SCPMP had issues in identification of genuinely committed beneficiaries and therefore had a lower (72%) achievement rate of progress towards target. The issue of ‘commitment to participate in a project’ or ‘being shortlisted as a targeted beneficiary’ was identified in the field as among the extremely important first steps in project implementation, as, if the wrong people are ‘targeted’ or ‘recruited’ into a project, the whole project process starts off on a wrong trajectory and implementation becomes a constant struggle, resulting in low achievement rates. Analysis shows that during the project planning phase, there was insufficient clarity in defining the beneficiaries, with clear selection criteria. SCPMP had important weaknesses in knowing exactly who to support, and the evaluation team found this to be a serious weakness in design and implementation. For the CARLA project, the number of beneficiary farmers surpassed the original target at the end of the project period. There was positive response from the communities due to the felt needs arising from frequent disasters occurring among the targeted communities. Participation of women farmers in both the SCPMP and CARLA projects was very good. The target of 45% was achieved in the SCPMP while the CARLA achieved 53% against a 60% target.

At the higher level, and very much the same as for the transport sector, the Bank monitored CSP implementation (e.g. MTR, Supervision Missions) and measured progress of implementation of MGDS (e.g. IMF Joint Staff Appraisals). However, weak institutional capacity in WATSAN-related ministries jeopardized the analysis of the effects that interventions were having on beneficiaries.

Sustainability

Sustainability is evaluated by assessing the extent to which the benefits from Bank interventions continued to exist, or were likely to continue to exist, after project completion. This performance dimension is rated Unsatisfactory.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Few of the completed projects under review have reached a post-closure period of 5 years, which makes actual sustainability assessment challenging. Some benefits gained from past interventions (in particular the soft components) were already at risk of not being sustained in the immediate years after completion. Benefits from past interventions, most notably infrastructure, have continued to flow (longer than 5 years), but longer-term sustainability (i.e. for the life of the project envisaged at design) could not be ascertained.

Better sustainability practices were found in the agriculture and transport sectors, compared to the social sector. At the same time, sustainability is not guaranteed in any of these sectors. In the agriculture sector, for example, the SCPMP was rated unsatisfactory on three parameters including i) financial sustainability; ii) ownership and sustainability of partnerships; and iii) environmental and social sustainability. Only institutional sustainability and strengthening of capacities was rated satisfactory, with a score of 3. In contrast, the CARLA project was rated satisfactory on all similar parameters, with a highly satisfactory rating of 4 for environmental and social sustainability. In the social sector, the evaluation concludes that the LED, competitiveness, and job creation may not be sustainable under the present aid architecture (i.e. unless considerable investment continues to flow to those types of strategies over a long period).

It is a requirement that sustainability considerations be included at the design stage. The projects in agriculture and infrastructure in particular included sustainability plans, with a clear definition of roles and responsibilities.

❙ In the agriculture sector, plans included clear definition of accountability for sustainability. This included the GoM paying for recurrent costs, the Water Users’ Association paying for costs using user fees as well as management of schemes, all at the same time as the Dwangwa Cane Growers Trust in Dwangwa

and village management committees were to manage and maintain market and irrigation infrastructure in Dwangwa, Blantyre and Shire Valley Agriculture Development Divisions.

❙ In the transport sector, plans are in place for the maintenance of roads both at the national level and in the preparation of individual projects. There is also clear definition of responsibilities for sustainability (maintenance and operation) in the short term, with the contractor responsible for the ‘defects liability period’ immediately following completion of construction (usually one year), and thereafter being the responsibility of the Roads Authority (with funding raised mainly by fuel levies accruing to the Roads Fund Administration).

However, the management of sustainability has not been institutionalized, follow-up actions have been limited, and the execution of sustainability commitment, including financial, has remained weak. More specifically, the sustainability plans developed at design stage have been based on a set of intertwined, unproven, assumptions about government commitment, government capacity / internal sustainability, community participation, recurrent costs, and the external environment. These are discussed below.

Government commitment

Most projects implicitly equate sustainability with government commitment, which is inaccurate. For example, the Competitiveness and Job Creation project, Health SWAp, and Education V projects note that sustainability was to be ensured by the Ministry. Yet the Ministries involved had a very hard time in financing operations, e.g. this was a considerable concern that justified the PBS project. Interviewees in the health sector, for example, noted that the GoM had not budgeted sufficient resources for ongoing maintenance and operations of the maternal care facilities and programs. One of the sites visited during the field mission had no running water for the maternal care unit even though the water tower was


built, the result of a lack of budget for a simple water pump. There is hence a constant assumption that the government institutions involved will not only honor their commitment but also have access to sufficient resources to fulfil these commitments (which may include resources from the Government budgets and/or setting of tariffs/collection of revenues).16

Similarly, PBO’s full alignment of government priorities aims to ensure strong government ownership. In practice, however, progress with selected government actions has been at risk of reversal in Malawi, as the country went through three changes in government and commitment to PFM reforms has remained cyclical. There is also a perception amongst donors that government actions in PFM are mostly driven by donor requests. In addition, when GoM policy involves devolution of responsibilities to lower level administrative bodies in sectors such as WATSAN and agriculture, this rarely comes with the provision of commensurate resources.

Government capacity / internal sustainability

The Bank has assumed that using and supporting established government structures to implement the projects would be adequate to ensure sufficient follow-up in terms of both commitment and capacity. In practice, the soft component of the interventions has been insufficient and/or not adequately followed up with complementary measures to ensure that the required skills were institutionalized. For example, the SCMP project was implemented within the established structures in the Department of Irrigation and the District Councils. While field extension staff and district engineers were adequately trained, long lasting capacity building of mainstream civil service staff was not addressed at design stage and the hand-over between project specific staff and the mainstream civil service was in some cases not coordinated. Similarly, in the health sector, the Ministry of Health and Districts have not taken the training of people sufficiently into account, therefore not providing qualified employees as expected.

PBOs are also designed so that the government can fully use and, in so doing, strengthen, its PFM systems. According to the Bank, this inherent feature of PBOs ensures, in principle, strong institutional/internal sustainability. The Bank’s decision to move towards ring-fenced budget support operation has had an impact on sustainability; however, with PBS, while funding still goes to the Treasury, the Bank operation no longer relies on the country’s financial reporting and procurement systems. These additional safeguards mean that PBS bypasses the country’s procurement and financial reporting systems, with key informants criticizing the lack of knowledge sharing and/or skills transfer between the hired audit company and Ministries, Departments and Agencies’ procurement unit. However, PBS was never designed to be sustainable in the first place, and the project shows how precarious the situation is in Malawi for long term sustainability of any program.

The inadequate quality of training and equipment financed by the Bank in the social sector was also likely to have a negative impact on government capacity for follow-up actions. More specifically, interviews and studies show that some of the programs supported by the Bank have received just enough funding to remain active, but at a level of quality that is insufficient to meet the statement of outcomes. For example, teachers do not receive better training, and new texts are not forthcoming within the Ministry of Education Science and Technology budget allocation; maintenance of maternal health clinics is substandard and some facilities have no water or power; professors of universities and technical colleges do not get upgraded and equipment maintenance and replacement is substandard (i.e. poorly planned and funds not available and/or improvement not done as required).

In contrast, the Bank’s decision not to set up a PIU but select the PFM Division as the body responsible for coordinating and monitoring its Institutional Support Project in PFM scores well in terms of sustainability. The Bank’s Institutional Support Project is also likely to generate sustainable results when its focus is on


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

strengthening local training institutions. Elsewhere, the Bank’s approach to training remains overall poorly defined and is unlikely to provide results, let alone sustainable results.

Community participation

Where communities were given management and operational responsibilities, their level of ownership still remained weak. Reasons put forward included: i) underutilization of the projects’ infrastructure and services (some being barely functional at completion, for example with the SCPMP project, most of the schemes were not fully operational at completion); ii) the mindset of local communities, as evidenced through undue reliance on the government; and iii) the lack of adequately timed and resourced capacity building activities (in the agriculture sector, for example, SCPMP, farmer training on leadership, marketing and technical issues was conducted before the infrastructure was developed). From a financial point of view, this has led, among other factors, to lower than expected users’ fee contributions, putting sustainability at risk. From a physical point of view, this has led to a premature deterioration of the projects’ community managed infrastructure.

In the agriculture sector, community participation was compromised in the Agriculture Infrastructure Support Project (AISP). Inefficiencies and continued delays in implementing the infrastructure component17 meant that the time left for operationalization – and with it, capacity building – was likely to be reduced significantly. In the project sites where irrigation and solar infrastructure were installed, there was a lack of a coherent plan for the beneficiaries to fully utilize the irrigation scheme. The mission further observed that the farmer cooperative lacked the capability to initiate and manage a market-facing business strategy.

Recurrent costs

The cost of operating and maintaining the project’s infrastructure was consistently under-estimated. For example, at appraisal, beneficiaries of operation and

maintenance commitments in the SCPMP were estimated at USD 12,360 per year (about 0.14% of capital costs or about USD 3/ha), but actual costs were closer to USD 30-60/ha/year/plot (including fuel for schemes that were using motorized pumps). The HEST budgets for replacement of computers and software were linked to operating budgets that have traditionally been based on ‘replacement of components’ and do not cover repairs, broken-down equipment, technical advances, etc. In addition, sustainable benefits further hinged on the provision of good quality infrastructure built to appropriate specifications and hence requiring low, yet regular, maintenance work. This was not always the case, as was seen in the transport sector.

External environment

Depending on the intervention, environmental sustainability has only partly been taken into account. Projects in the WATSAN sector have only limited reference to climate and biodiversity despite wider resource management being an important issue in Malawi (as many water supply systems are gravity-fed piped systems which have fallen into disuse due to lack of maintenance and, in some cases, reduced surface water runoff). Continuing availability of sufficient water is assumed instead of being managed proactively under a sustainability thrust.

Other external factors, some of which are linked to the policy environment, have also been overlooked. For example, the limited market linkages partly explain the lack of sustainable results under the SCPMP; in the transport sector, axle load control has not been adequate, which contributed to prematurely deteriorating infrastructure. Overall, although the impact of external shocks (for example political instability, balance of payment issues, flood) on sustainability has generally been recognized, they have not been dealt with. In the transport sector, it was also assumed that there will be a rapid response to emergencies (e.g. flood damage), but this did not materialize.


The policy environment also remained problematic, as in the WATSAN sector, which remains characterized by inefficient systems, demand outstripping supply, ineffective community-based management systems, unregulated WATSAN services, slow and unbalanced decentralization of responsibilities, inadequate budgets, and institutional weakness.

Similarly, the long-term sustainability of PBOs’ key benefits, i.e. improved macroeconomic stability, strengthened PFM systems, and improved public service delivery, has been affected by economic and external shocks.

Finally, the issue of sustainability is not adequately monitored. An absence of ex-post evaluations or other follow up makes for limited assessment on the effectiveness of actions aimed at ensuring sustainability, the degree of fulfilment of commitments, and actual performance of the systems delivered by Bank support. As a result, there is usually no long-term follow-up of service levels and condition of Bank-supported infrastructure.

Crosscutting issues

This dimension is evaluated by assessing the extent to which the Bank’s objectives on inclusiveness across gender, regions, and environmental sustainability have been mainstreamed in Bank interventions. This performance dimension is rated Satisfactory.

Gender issues are strongly considered at the strategic level by the Bank. In both the Malawi CSP and the RISP, gender issues were taken into consideration (e.g. disparities in access to services, decision making structures, economic opportunities, cultural biases, budgetary allocation and monitoring of gender actions).18

The Bank carried out a country Gender Profile in 2005, which highlighted the main constraints to women empowerment. Therefore, project documents, mainly appraisal reports, have a section dedicated to gender issues, which are expected to be directly

impacted or addressed by the project. Bank projects in Malawi during the time-frame under review had a gender-informed design, which specifically incorporated: i) sector-specific gender analysis; ii) a gender-equality-related outcome statement; iii) a gender-equality-related baseline; iv) specific activities to address gender gaps; and v) adequate financial and human resources to implement the activities. Moreover, most of the monitoring indicators are gender disaggregated.

For the social sector (education and health), it is noted that specific gender targets are specified for every intervention. In particular, for the education sector, where the indicators show that women tend to be less represented in the secondary cycle with negative consequences on employment−although disaggregated data on unemployment and salary rate are not available. As for the transport sector, although Bank strategies do not identify transport sector-specific gender issues, the transport projects would support women, men, children and the elderly equally via improved social and economic mobility and improved access to markets and health centers.

Usually the WATSAN sector is one with the greatest inherent impacts upon women and girls at multiple levels – access to adequate quantities of potable water within a reasonable distance, water carrying (which is predominantly a task of women and girls), hygiene and sanitation with direct impacts upon family health, irrigation for agriculture (an activity with high female participation).

As for the agriculture sector, there is evidence that women participated in the projects. For instance, the SCPMP PCR further notes that the project was designed to promote gender mainstreaming and specifically targeted female headed households to improve their livelihoods through membership in Farmers’ Organizations. The envisaged participation of women farmers (45%) was achieved, but not so for Government women staff, where only 10% was achieved against a target of 33%. In the CARLA project, women participation was 53% against a target of 60%.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The Bank programs appear to cover all regions, but there is no evidence of a further rollout to other GoM districts, nor other donors once the intervention ends. All interventions cover more than one district and are widely implemented in rural areas. However, inclusive coverage of regions where poverty is most prevalent is highly problematic. The poorest and/or most vulnerable have not been explicitly targeted in any of the projects and the poorest people predominantly living in rural areas are unlikely to be direct beneficiaries unless the rural (and urban) poor can enjoy greater direct benefits of Bank interventions such as big infrastructure (road corridors) or health and education projects which targeted rural areas within the beneficiaries.

Environment. In accordance with the 1996 Environmental Management Act, all sector projects in Malawi should undertake an Environmental Impact Assessment. Therefore, the Bank has made efforts to mainstream environmental assessment into project preparation and implementation but, although in most cases good engineering practices should minimize adverse environmental and social impacts, there is a reported capacity deficit in the application of sound environmental and social mitigation measures.

At the CSP level, most reference to environmental and social issues in Malawi is concerned with describing the current situation rather than setting out explicit strategies addressing these issues. However, it is clearly stated that environmental protection and mitigation of adverse social impacts is a component of Bank support. The RISP goes further, noting that the Bank will take leadership in promotion of climate friendly infrastructure and reduced carbon emissions,19 making specific reference to Malawi’s (and some other Southern African countries’) problems regarding the effects of increasing climatic variability on infrastructure, food security and development.

ESIAs and ESMPs are produced for all sector interventions, but a lack of reporting in the implementation of ESMPs during the course of

a project is noted. These ESIAs and ESMPs are prepared in compliance with national legislation and in conformity with AfDB environmental and social policies (which, in turn, are fully compliant with international best practices). All AfDB projects are categorized at an early stage in preparation and this categorization defines the expected nature of environmental (and social) impact and the mandatory mitigation treatment.20 Climate change is mentioned in such studies, but no proactive resilience measures have been noted in scrutinized projects and there are no targets on carbon footprint reduction.

In the agriculture sector, the Bank’s interventions demonstrated a clear link with green growth strategies and plans. For instance, the two agriculture projects SCPMP and CARLA (completed projects) as well as ongoing projects (AISP and Smallholder Irrigation and Value Addition Project (SIVAP)), were designed with the intention of supporting resilience capabilities and managing natural assets. New projects in the agriculture sector had climate-informed designs that included satisfactory actions to mitigate against impacts of climate change, climate variability, extreme weather events and biodiversity loss.

Despite having very good environmental management plans, not all projects (e.g. SCPMP in the agriculture sector) were managed in a manner that would ensure environmental sustainability and/or support the transition to green growth. On the other hand, environmental management plans were included in all stages of project planning and contracting. Indeed, all examined projects considered such issues and sanitary conditions were manifestly due to such support. Community sensitization has included environmental (and hygiene) best practices and catchment protection.

As regards to social aspects, beyond compensation (for land, resettlement, etc.), Bank sector support has included good practices such as community consultation and collaboration in rural areas (9 districts) and informal urban settlements (in Blantyre). The support projects

Phot

o: ©

AfD

B


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

scrutinized were all responsive to expressed demand and such community support was expected to contribute to ownership and management of the facilities. However, longer-term continuity of such involvement and motivation is not proven.21

Efficiency

This performance dimension is evaluated by assessing the extent to which the resources and inputs provided through the Bank’s support are converted into results at a rate that reflects assumptions at project appraisal. Moreover, whether the Bank’s interventions were implemented in a timely manner is also assessed. This performance dimension was rated Unsatisfactory

The Bank was responsive and flexible to ensure satisfactory project performance, notably by allowing a reallocation of resources across the PFM project’s main components, and using unused resources to fund follow-up activities within the ‘multisector.’ Completed agriculture projects (SCPMP and CARLA) had PCR results that gave a resource use efficiency rating of 3, which is satisfactory. There was reference to ‘increasing the size of operations in order to reduce transaction costs’ (RBCSP22 2005–2009) and ‘the Bank needs to focus on a few sectors with an increased average size of operations by leveraging resources from other donor partners'. This was expected to contribute to the reduction of transaction costs for both the Government and the Bank (2005–2009 CSP Midterm Review).

However, the Bank’s operations have incurred delays, mainly due to noncompliance with its own operational standards (although slow Bank response to Government requests for ‘no objection’ has also been noted). The delays are mainly procedural and reflected in delays in decision making, disbursement, government commitment, and capacity constraints for implementation.

Long-term predictability of budget support disbursement has been problematic. The Bank did not provide budget support during 2011–2012, 2013–2014 and 2014–2015, and the amount fluctuated widely from year to year for the years when budget support was provided. The main constraint to ensuring the Bank’s predictable provision of budget support has been linked to the Malawi context, with the GoM first going off-track with IMF arrangements in 2009, then the ‘Cashgate’ scandal prompting a budget support withdrawal from all CABS DPs in 2013. As a result, the Bank’s budget support grant resources under ADF XI were reduced from UA 20.3 million to UA 11.5 million in 2010 to a slippage in the Country Policy and Institutional Assessment rating.23 This meant that the provisional budget support amounts announced to the government as part of CABS, could not be achieved.

For non-budget support projects, wherever there was a problem with disbursements, it was related to GoM administration of the disbursement process. This is a general problem across almost all Bank operations in Malawi, with non-Bank managers finding it difficult to manage the finance-related processes of the AfDB. Finally, the disbursements predicted for private sector development through the commercial banks did not materialize.

Efforts were made by the Bank to use national systems but with little success, meaning that PIUs are very common and the Bank’s systems are used. Overall, for a country with the level of development of Malawi, the tardiness of completion and the difficulty of implementation, the problems encountered in the use of national systems are predictable. In terms of time to first disbursement, most first disbursements took place more or less as planned.


This performance dimension was evaluated by assessing the extent to which the Bank was actively engaged in and influenced PD through the provision


of relevant advice that was incorporated into policy decisions, and the extent to which the Bank provided appropriate and adequate analytical work in support of its interventions, positioning and policy advice. This performance dimension was Satisfactory.

Even though detailed plans for engaging in PD was not available, the Bank has been substantially involved in PD in Malawi. For example, the PBS appraisal report describes the many strategic topics of the accompanying PD for the intervention. It is collective in nature and is largely executed through various donor groups. Although there is no mention of specific value-added by the Bank or of comparative advantage, and no targets or objectives set for the dialogue, the interviews have revealed that the Bank had a long history of PD and other non-lending activities in Malawi. These were strategic in that the Bank was consulted extensively for the preparation of all MGDSs and that some of its advice was taken on board, especially in irrigation and agriculture. The Bank was always part of the Heads of Cooperation discussions leading to taking strategic positions with the GoM (including those on farm subsidies in which the Bank played a leadership and determinant role in policy change). The Bank was instrumental in prioritizing, through dialogue, the maternal health component of the health strategic plan for its support, and in finding solutions to help Malawi in the immediate aftermath of ‘Cashgate.’ These examples point to a strategic perspective to policy change. Each Task Manager works on policy issues and the Country Manager was often mentioned by interviewees as having considerable legitimacy in the eyes of the GoM as regards to decisions on strategic options, including those in agriculture and power.

The Bank has adopted a strategy of engaging in PD through association with other donors. For instance, the Bank is part of various donor groups through which it had sector or strategic discussions with the GoM. While acknowledging the work done by the Bank, the country stakeholders, including other donors, thought that it could play a larger

role. It was often noted that other donors do much more preapproval analysis of interventions, so as to have a better idea of what has to be modified in terms of policy and to incorporate policy change requirements in the design. The Bank officials feel that they also need this opportunity to improve on PD and to participate in deeper level consultations and analysis.

The Malawi Country Office undertakes PD through a variety of means, including participation in technical and donor working groups, participation in high-level forums with the GoM, one-on-one meetings with GoM officials and ministers, donor-country meetings, and political dialogue meetings. In some of these meetings, the GoM reports on modifications it has made to regulatory and policy frameworks as part of the dialogue in those fora. In terms of the ‘perception of usefulness of PD by cooperation partners,’ various project reports describe processes and inputs and activities, not PD (with very few exceptions). Interviews with donors revealed that the PD undertaken by the Bank was very useful and was always done in a spirit of collaboration, cooperation and support. Examples in GBS, health, education, transport and agriculture were cited. Many had wished that the Bank would assume a larger leadership role, given its ‘preferred relationship’ with the GoM, which considers the AfDB as ‘their own Bank.’

Evidence from consultations shows that the GoM is particularly satisfied and even encouraged with the relationship that it has with the Bank, largely in the area of PD. The PD in question has had specific effects, for example, the GoM decision to change its policy on farm subsidies and a subsequent move to a private sector model was largely the result of the Bank’s PD with the Ministries responsible for agriculture, trade, irrigation and allied sectors. Other examples include health policy on maternal care and content of MDGSs. Interviews undertaken have shed light on the way the GoM uses PD strategically, indicating that, in essence, it could improve. It uses PD as a component of project management and as a way


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

of getting issues out into the open, but rarely uses it as a formative process, and seldom asks to start a dialogue on a particularly difficult topic.

In specific sectors the experience has been that PD is not documented well. For example, there is little explicit reference to WATSAN PD in Bank documentation – only RISP and 2013–2017 CSP MTR have reference to WATSAN sector PD, which may indicate increasing Bank engagement in PD but such reference is more concerned with activities (and coordination and harmonization, participation in Sector Working Groups, PFM donor groups, joint committee, Group on Financial and Economic Management, Donor Coordination Group and so on) rather than required policy change as such. On the other hand, the Bank’s pan-African WATSAN Initiative, Rural Water and Sanitation Supply Initiative (RWSSI), might be expected to provide a platform for such high-level PD but there is no reference to PD in recent RWSSI progress reports. This situation is also recorded in the 2015 IDEV report (WATSAN in Africa). It is almost as though the Bank separates ‘dialogue’ from ‘policy’ in this sector. GoM’s capability to use PD as a strategic tool is not established – there is little such references in MGDS apart from the GoM’s intention to “lead the dialogue with donors…and seek to ensure that aid flows are predictable”. However, institutional inertia suggests not only GoM’s limited capacity to harness such dialogue but also casts doubt on the effectiveness of PD as a whole.

In the agriculture sector, the ‘Troika’ has produced policy papers that have greatly facilitated PD. Some of the recent policies that the DCAFS has influenced include the shift of the focus of the Farm Input Subsidy Program to work through private sector, the drafting of the Seed Policy and the legislation of the National Agriculture Policy. Cooperating partners have a high opinion of the PD, but perceive the Bank’s attendance at DCAFS as being sporadic with only 60% attendance.24 Furthermore, the Bank turning down an opportunity to chair the DCAFS

in 2017 was perceived as a lost opportunity, which would have given it greater advantage and visibility, and could have influenced the GoM. The GoM has used PD to lobby for increased support from DCAFS members, and this has resulted in a noticeable shift in the support that it has received from donor partners. For instance, the support for the irrigation sector has come after many years of lobbying, with the GoM’s interest to invest in irrigation going back to the 1940s.25

In the multisector domain, the Bank fully participated in the CABS process and structure. However, it may be noted that while CABS provided an effective platform for dialogue, it did not significantly influence policy change. For example, there is no single example to show that CABS was influential in getting some difficult reforms off the ground. Some government and donor officials have questioned whether the scope of PD was not too broad and whether CABS was the right platform to discuss non-PFM related issues. At the same time, an independent review carried out in 2013 indicates that “line ministries felt marginalized in the CABS process (headed by the Ministry of Finance) and that many did not see the direct benefits of GBS support.” The same review concludes that CABS had been effective in strengthening donors’ relationship and dialogue with government, but that the dialogue had become diluted away from key policy issues.

As indicated by a donor representative during the country mission, several compromises were made along the way, as the government continued to meet the majority of its Performance Assessment Framework indicators and targets. The CABS mechanisms also allowed little opportunity for follow-up with the aide memoires not carrying a list of agreed recommendations and/or actions. While all parties committed to a mutual accountability principle, there was no effective mechanism for holding government and donors to account.

Overall, policy influence was most effective when the IMF arrangements went off-track, budget


support was withheld, and the government felt compelled to take remedial actions. PD, for example, intensified after the ‘Cashgate’ scandal with the government agreeing to address donor concerns through the PFM Action Plan and an e-Performance Assessment Framework. This dialogue was mostly carried out through the Troika (with AfDB and European Union as co-chairs) and the Group on Financial and Economic Management26, involving donors and Ministry of Finance, rather than during the CABS reviews.

A significant number of new economic and sector papers were produced by the Bank over the years. Even the other donors referred to some of the themes or topics covered in the papers as having been central to their programing. Unfortunately, the mechanisms to finance the generation of these documents are unwieldy, based on competition, or are embedded within projects and so are long and complex to be written contractually. Embedding financing as part of an intervention means that the funds cannot easily be used upfront for ex-ante analysis, and the timing will be delayed if the analysis is required. The Bank’s internal research facility is not used as it could be as it is not perceived as being client-oriented or responsive to urgent or even operational needs.

GoM and sector experts in donor offices appreciated the quality and depth of analysis and the way the advice was offered. They also appreciated the value of the Bank Task Managers in resolving project management issues. This is important in the Malawi context because interviewees noted that the various departments, agencies and ministries of the GoM had very limited capability to generate evidence-based analytical research in any domain, with the possible exception of phytosanitary or agricultural research.

There is limited reference to ESW in PARs and PCRs in the transport sector (although project preparation studies such as feasibility, detailed design and ESIAs could arguably be considered as

ESW). Interestingly ESW is classed with dialogue activities hinting that ESW is intended to be used as a facilitation for such dialogue although such linkage/leverage is not articulated as such. The CSP 2013–2017 is more explicit about the ESW to be conducted – mainly feasibility and other studies – while RISP indicates that ESWs will be grant-funded.

Perceptions of the value of ESW advice in that sector were limited (mainly to project identification, feasibility and design studies) with some reference to Quality at Entry issues, which were, overall, not considered serious. However, GoM transport sector capacity to take advantage of such ESW outputs in order to generate analysis for policy development is limited. In that sector, as in WATSAN, the Bank has provided reasonably adequate analytical work in support of its interventions (although these interventions continue to be subject to over-optimistic assumptions). The positioning of the Bank has been dictated by its higher-level policies and initiatives (e.g. RWSSI) and there is little evidence to show to what extent the Bank’s dialogue activities are informed by such ESW; the perceived value of such advice and analysis (to the Bank, GoM and other sector stakeholders) has been confirmed by the stakeholders.

The Agricultural sector has seen a significant number of analytical and policy options papers prepared through the Bank’s non-lending facility. These include a: i) Feasibility Study on the Establishment of an Agriculture Cooperative Bank; ii) Regional Comparative Fertilizer Subsidy Study; and iii) SVIP feasibility study. The Bank contributed to the following sector papers through the existing Bank staff: i) the Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; ii) the Background Note 2: Agriculture Growth and Diversification; iii) Background Note 3: Value Chain Analysis; and iv) Background Note 4: Agribusiness.

Field research shows that the Bank’s contribution to PD through these analytical works is appreciated, especially since the Bank also supports the implementation of the policies. The GoM is able


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

to generate evidence-based analysis for policies relevant to the Bank through the work of the National Statistical Office, which has the necessary capabilities, with additional external expertise brought in when necessary.27

In terms of analytical work related to PFM and GBS, the Bank has regularly conducted or contributed to these analyses. Some analytical work was used to inform interventions, positioning and policy advice. In addition to the regular Fiduciary Risk and Country Procurement Analysis, and the yearly African Economic Outlook, the Bank published several economic and sector work relevant to financial and economic governance.

Partnerships, Harmonization and Leverage

This performance dimension was assessed by evaluating the extent to which the Bank’s interventions and processes were harmonized with those of other donors to avoid duplication in addition to the extent to which the Bank’s interventions and resources influenced other stakeholders to get involved in sectors that are the focus of country strategies. This assessment dimension was Satisfactory.

The Bank and other donors have long followed a coordinated approach to their development assistance in Malawi. In key sectors, sector-wide approaches (SWAps) have allowed government and donors to work towards shared strategic objectives. While donors have continued to follow their own programing cycle,28 this aid architecture has been conducive to a relatively high level of shared (rather than joint) strategic planning. For example, in most sectors, donors broadly share the same analysis and objectives on key reforms.

The Health SWAp and agriculture sectors have entailed regular Joint Sector Reviews. There was also a high level of harmonization among budget support donors until 2013, with all CABS

donors broadly following common procedures for monitoring and reporting. In addition, the Bank has regularly participated in joint studies such as the SVIP feasibility study and sector papers such as the i) Malawi Country Economic Memorandum: Seizing Opportunities for Growth Through Trade; Background Note 2: Agriculture Growth and Diversification, ii) Background Note 3: Value Chain Analysis, and iii) Background Note 4: Agribusiness. As noted earlier, the Bank also conducted a joint fiduciary risk analysis with DfID29 in 2009. It may be noted that while Joint Sector Reviews have been commonly used, the Bank (like many other DP) has continued to use its own monitoring and reporting procedures in parallel.

The Bank has been an active participant in several donor platforms such as the DCAFS (agriculture) and the PFEMRP (for PFM). While some de facto division of labor amongst donors has been in place, donor ambitions to move towards joint undertaking as managed jointly have been only partly met. There are limited examples of co-funded interventions, such as the SVIP, which is implemented together with the WB, and the NWDP in the WATSAN sector, which is co-financed by AusAid. In the transport sector, the JIU was discontinued in 2012 after only 2 years’ operations. The JIU was a local initiative by AfDB in Malawi for closer cooperation between AfDB and WB for joint management of the infrastructure support portfolios of the two Banks (i.e. WATSAN and transport). However, this initiative was short-lived owing to the lack of formal agreement ‘signed off’ by WB and AfDB, the absence of a mutual acceptance of ‘proprietary’ administrative, fiduciary, procurement or safeguard/governance systems, and limited support from the two Bank’s headquarters. In PFM, attempts to launch a Multi-Donor Trust Fund were only partly successful, as only a few donors joined in, owing to legal reasons (like the AfDB, which did consider it), or, because some projects were already ongoing.

As a result of the above, the level of complementarity between donors has not been optimal. In principle, the Bank takes into account other existing donor interventions during appraisal


to minimize the risk of overlap. For example, the complementarity between the SCPMP and existing donor interventions in the sector was noted to be high during appraisal,30 yet in practice, there is no description of how these various projects are made to be complementary. Complementarity is neither monitored nor measured during project implementation, and PCRs do not refer to the interfaces between the interventions. Evidence collected during the field visit nonetheless tends to indicate a higher level of complementarity in some sectors, in particular agriculture.

Because of the limited success in shared undertakings, the multitude of projects in some sectors, like vPFM, where many donors intervene, has continued to weigh heavily on the government, as donors stick rigidly to their own procedures and there is no common reporting format. Even in the case of CABS, the Bank together with other donors were occasionally seen to come up with their own specific requirements.

Going forward, recent developments have shaken the donor landscape, bringing new challenges for future harmonization. The multi-sector and transport sectors illustrate these challenges well:

❙ In the last 5–6 years, donors’ response to the context and their readiness to accept (fiduciary) risk has diverged, with bilateral donors no longer providing budget support.31 This has in effect ended the CABS arrangements. Understandably, some bilateral donors are uncomfortable with the AfDB (and now WB)’s decision to resume budget support, which they say is sending wrong signals to government that the prerequisites for resuming budget support have been met. It has hence become crucial for all donors to share the same messages to the government, whatever their choice of aid modality.

❙ In the transport sector, the number of traditional donors has reduced with the withdrawal of the European Union and some EU

Member States from the sector. In this sector, new donors follow different approaches and do not normally take part in sector donor fora.

The Bank’s influence on traditional donors also appears to have been relatively limited, although this varies across sectors. While the Bank’s decision to resume budget support with PBS may have helped the WB’s subsequent decision, this was not directly confirmed by key stakeholders interviewed during the field visit. In the governance sector, the Bank has also been relatively passive in getting other stakeholders involved in its PFM work. More specifically, the main visit highlighted the lack of a sufficiently inclusive and open consultation process over the amendment of the Public Procurement Bill, which the Bank initially supported by a standalone Technical Assistance. The use of a regional envelope32 may explain this relatively ad hoc approach to Technical Assistance.

Agriculture provides a more positive example of the Bank’s influence. In this sector, the Bank was able to leverage the SCPMP to get financing for the CARLA project from the GEF. As a reliable and consistent partner in supporting irrigation (including the use of solar power), the Bank’s interventions are also inspiring other donors to support similar interventions,33 thus serving as pilot projects that other donors can learn from.


This performance dimension evaluated the extent to which the Bank succeeded in focusing on results in its management of day-to-day operations as well as the extent of its support to national systems that focus on results. This performance dimension was Unsatisfactory.

The Bank’s strategy and project documents contain outcomes and results that are well stated in near-RBM terms (i.e. in terms of the effects expected) but there are no intermediate results,


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

nor chain of execution or implementation in the key documentation. The verification of whether Bank interventions were managed by results requires an analysis of intervention management practices, including procurement, contracting, and supervision. The documentation does not support the premise that these were results-based, and interviews indicate that the Task Managers do not believe they have RBM-ready systems (this applies to most interventions with the partial exception of CARLA and SCPMP).

The Bank’s Monitoring systems do not focus on results, the evidence points to instances where lower-level results (outputs) were monitored, but not the entire results chain to outcome level. No mention of the extent to which RBM must be used has been found in the M&E sections, neither of Appraisal Reports, nor in the profiles and descriptions of the PIUs. Analysis and interviews on the ground indicate that monitoring systems are clearly not geared towards RBM. The reporting systems of the Bank are not RBM-based with most being process or input management-based and some output-based. Where there was a reference to results, it generally quantified achieved targets, rather than described the effects in terms of the original expected outcomes.

In terms of the sector level findings, the Transport and WATSAN sector analysis showed that all sector support documents (CSPs and individual support projects) have a Logical Framework Matrix which specifies results indicators at output (and activity), outcome and impact levels, but not a detailed project path analysis or Theory of Change logic including assumptions and risks. To that extent, Bank support can be said to be only partially results-based (as already mentioned above). However, in practice monitoring of such support is largely confined to activities and outputs, with limited coverage of outcomes and virtually no coverage of impacts. Thus, although action is taken during the course of implementation based upon mainly activity monitoring (e.g. MTR of CSP, progress reports of transport and WATSAN

projects) the benefits of such RBM is limited by not supervising for results, nor in proactively managing assumptions and timing, nor extending management to higher level results (e.g. not putting in place any form of baselines on benefit analysis over the long term).

In the Agriculture sector, evidence points to the finding that the Bank has often successfully implemented management systems that focus on process-based results and facilitate learning from experiences. The designs of both the SCPMP and CARLA projects include a Result-Based Logical Framework, which demonstrates the linkage between inputs, outputs, outcomes and impacts, and includes a risk mitigation assessment and key assumptions. Monitoring systems and mechanisms were based on mid-level results, which enabled stakeholders to keep track on implementation progress in order to achieve the intended objectives. Reporting mechanisms were component-based and structured according to evaluation criteria, such that the supervision missions have ToRs, and the reports (Back-to-Office report, aide memoire) are based on implementation progress, but lack focus on what the ultimate outcomes should be. These reports include a ‘Status of Project Implementation/Key Indicators of Progress’, which keeps track of the progress made against project results/targets.34

In terms of GBS and PFM, recent changes to the PBO guidelines are designed to improve the extent to which these types of interventions are results-based. PBOs evaluated in this mandate displayed the following characteristics:

❙ The causal chains were poorly defined and confused different levels of results (output and outcomes) and their corresponding indicators.

❙ The choice of indicators was largely driven from PAF. The Bank chose its own indicators in some occasions (often to ensure timely payment), which created some misunderstanding and delays due to additional reporting.


❙ PAF was too broad, placed a heavy emphasis on process and input-based targets, and was not consistent with or directly linked to other government planning and M&E systems, which in turn remained weak (with often contradictory baseline values). When results indicators were used (e.g. pupil/teacher ratio), they did not sufficiently take into account the time it took the government to achieve the agreed targets.

In terms of the experience of the Transport and WATSAN sectors, evidence points to the finding that the Bank broadly supported the development of national capacities and management systems, but that focus has been largely on process and program/project implementation rather than outcomes, or overall capability to design and implement regulatory or policy frameworks; results have been assumed to be developed from such enhanced capacities. Transport and WATSAN sector projects include capacity strengthening activities as a component of implementation (e.g. pump maintenance, district capacity for water supply operation and maintenance, road maintenance) but such capacity is usually related to the operation of project infrastructure and systems after the end of the project period. However, the evidence clearly points to important capacity constraints that will continue with multiple examples of both Transport and WATSAN sector infrastructures rapidly falling into disuse due to lack of sustainability. Most donors in the sector have RBM systems, although limited information is available for Non-Governmental Organizations and Civil Society Organizations active in the WATSAN sector. Most public bodies involved in WATSAN management are seriously under-resourced such that effectiveness of results management due to capability weaknesses or gaps is constrained.

In the Agriculture sector, the Bank has supported the development of national capacities and management systems, but a great deal remains to be done, at all levels of the capability chain. Results-based capacity development efforts for GoM line ministries and agencies in the sector

are related to input and output organization and combination, not strategic outcome improvement. The evidence shows that most of the actions are not based on knowledge of what will, or won’t, work but on ideas for structuring and processing economic elements (from agricultural inputs to cooperatives and market councils, etc.). Very little analysis of cause and effects is drawn up and much of the strategic direction is based on isomorphic analysis (i.e. if it works there it will work here). The Bank’s supervisory and midterm analyses determined that it had been successful in its capacity building of GoM staff, such that both the SCPMP and CARLA projects had a satisfactory rating of 3 on institutional sustainability and strengthening of capacities. The evaluation findings do not support this level of appreciation, believing that it is a reflection of the level of execution of activities and not one of sustainable capability.

In the Multisector domain, evidence shows that the Bank’s specific objectives on procurement and domestic resource mobilization were to improve capacity in revenue administration and customs service delivery, leading to increased revenues, enhanced competition, efficiency and controls in procurement. In practice, insufficient attention has been given to managing and monitoring any of the results in those areas and the entire revenue and procurement systems are still unable to perform satisfactorily.

The PFM Division reports quarterly to the Bank on progress-to-date. These reports, however, focus mostly on activities. The choice of performance indicators at output level does not seek to capture the positive effect that training should have on procurement performance; while indicators at outcome level are too broad and/or largely depend on policy actions that are outside the scope of its interventions.35 The quality of the deliverables has not sufficiently received attention either and serious issues remain; consistency is lacking and the systems are dependent on the skills and knowledge of people in a context of rapid turnover.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Lessons Learned and Knowledge Management

This performance dimension is evaluated via assessing the extent to which the Bank learns from its experience. This is done by looking at whether the factors affecting performance and results of Bank interventions are identified36 and inform37 Bank actions and strategies. Moreover, the issue of incorporating lessons from Bank experience to inform future strategies and programs is also looked at. This performance dimension is rated Satisfactory.

The Bank has means in place to capture and record positive and negative factors that influence project performance. Lessons from previous interventions have been included in the CSPs under review, MTRs, PARs, PCRs as well as Back-to-Office reports of supervision missions. The narrative in these reports included description of achievement or failure to achieve results. Some reports also make recommendations for mitigation of negative effects and accentuation of positive aspects at sector level and/or for the portfolio as a whole.

An assessment on the extent to which such findings and lessons were incorporated into the design of new CSPs and interventions shows that use of KM depends upon the perceptions of potential users of the KM system, its availability and accessibility, and the definition and scope of the said KM system.38 Experienced Task Managers have a depth of knowledge about the sectors they managed although their access to wider available information is limited – the Bank KM system is rudimentary, relying on access to whole documents, which is dependent on knowledge of the existence of these documents, rather than to data-mining or thematic search in a KM database. Beyond publication of some multi-national case studies, there is little active dissemination of lessons learned or experiences by the Bank to

ensure better application of such experiences in subsequent support. Current use of the existing Bank KM system does not frame communities of practice nor facilitate system functionality or collaborative knowledge broking.

In terms of completed operations with a timely PCR, all available PCRs, except 3 reports, were prepared within 12 months following closure date.39 Government generated PCRs for Bank projects are not available and GoM contacts advised the evaluation that these are not normally prepared. Notwithstanding, PCRs contain elaborate sections on lessons learned, although lessons are stored in raw format (generally text) in the documents in which they were originally produced.40 Therefore, accessing information, findings, conclusions and recommendations from that data is tedious and depends upon knowledge of the existence of the specific intervention. At the strategy level, analysis of lessons learned is not always indepth or integrated into future support strategies and interventions.

Cross-sectoral, or generic lessons (e.g. procurement and decision-making delays, quality at entry) that are continuously repeated in consecutive support programs, show that the lessons identified were not always learned and that the underlying issues are not resolved. This takes into account that some issues are beyond the Bank’s capacity to resolve (or avoid), remediation or mitigation being dependent upon other actors (e.g. delivery of GoM or community commitments). The Bank’s support has taken such political economy contexts (multi-variable cause and effect analysis) into account by identifying issues of government commitment (and capacity) to undertake policy and institutional reforms; these were treated as a factor which would influence results and effectiveness of the Bank support (positively and negatively) thus prompting the Bank to belatedly launch PFM ISPs.


Lessons Learned and Recommendations

Lessons Learned

The following lessons are identified from the Bank’s program in Malawi:

1. Clear definition of beneficiaries. When the beneficiaries of the interventions are described in overly generic terms, they tend to be seen as a numerical target to be reached, and therefore the supervision and management of the projects tend to become input- and process focused, rather than outcome focused. Beneficiaries need to be as explicitly described as possible so that the effects of the targeted ‘benefits’ can be monitored. Specific baselines are to be established upfront and constantly updated with valid information.

2. Results-based monitoring system. There is no benefit that will accrue to the Bank in generating results chains that would only partially be managed. If outcomes are the core objectives of investments, then monitoring and supervision of processes and indicators must be undertaken at all levels, including outcomes. With commitment to results-based management, it is necessary to have a monitoring system that generates decisions focused on outcomes, and a solid analysis of results chains before project approval.

3. Facilitative conditions for PD. The Bank is well regarded in terms of its non-lending activities and the results they have produced. The policy and program dialogue that has taken place over the years has been well appreciated as a driver of change. Given that the GoM requires a considerable amount of support in order to

accelerate its development, an important lesson is that the Bank has been able to influence and advantage policy when it puts in place conditions that facilitate and support dialogue. Increasing the quality and strategic nature of the dialogue will likely improve the level of influence. These conditions include developing a knowledge trust, promoting evidence-based decision making, developing the capability of institutions and agencies to design change strategies and monitor progress.

4. Timely decisions on design change. Major changes in project design, if required, should be introduced by taking into account not only achievable outputs but also continued project viability and relevance, and this should not be postponed to the MTR if identified early during implementation.

5. Fiduciary safeguards and budget support. In a highly aid-dependent country, periodic withdrawal of budget support constitutes an external shock that aggravates macroeconomic instability and can cause unintended damage to growth and to poverty alleviation efforts. It is essential to strike a balance between promoting country ownership and using additional fiduciary safeguards that can prevent the sudden withdrawal of budget support.

Recommendations

Based on the foregoing findings and lessons learned, the evaluation proposes the following recommendations:

53Lessons Learned and Recommendations

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

1. Enhance capacity of Bank’s officers and managers in Results-based Logical Framework and the use of Results-based Management tools, including theory-based design, results-based contracting, and outcome-focused supervision and management of interventions. Each intervention should have clear identification of the characteristics of its target beneficiaries and design the content and required management systems. This includes the baseline for monitoring, the focus of the M&E system, reference points of the RBM system, indicators for outcome measurement, contexts for project implementation, basis for the Theory of Change analysis, and the basis for assumptions and risks.

2. Enhance Bank staff proficiency to effectively operate in the business ecosystems in which key sector actors operate, and design interventions that are holistic enough to stimulate the private sector’s investment and actions. This is essential since most GoM development strategies rely on the operation of its private sector. As most of the business ecosystems are to some extent international in scope, the Bank's actions in Malawi need to be international, with country programs developing ‘regional at least’ markets in a number of countries.

3. Since the Bank and DPs are pressing for effective and efficient PFM systems, the Bank should establish a clear timeline within which all required performance standards are met and proven. The GoM has been implementing PFM reforms for over 20 years, and therefore the Bank should identify what it should do if the systems cease to function during a specific period. The Bank could consider generating contingency supports to overcome critical weaknesses without compromising transparency and mutual accountability, and achieve development effectiveness.

4. The Bank should redesign its KM practices to address the need for access to information, and develop explicit and implicit knowledge in its Operations Divisions to improve decision making capabilities of the Bank and its clients. The Bank should build effective learning organizations by making learning routine and stimulate cultural change and innovation. It is therefore important that the Bank invest in modern KM technologies, develop its capacity and capability for data mining, and use KM as an integral part of any implementation strategy.

5. The Bank should ensure sustainability of its interventions by taking concerted actions throughout the project cycle to ensure that impediments to sustainability are addressed and managed. Beginning at design and continuing through implementation, the critical assumptions should be analyzed and steps taken to mitigate risks. In situations where sustainability cannot be reasonably ensured, it is advisable not to approve such interventions in the future and ongoing interventions should be adjusted such that sustainability can be assured.

6. The Bank should take adequate measures to ensure that its interventions and strategies provide equality and equity between genders and that the interventions provide a means to manage, track and report on these objectives. Gender as a crosscutting issue should be included in project designs and mainstreamed into implementation, going beyond simply including women as a fraction of the participants in activities. They should be part of the management and their needs, as being different from those of men, fully acknowledged. Project designs need to be clearly structured so that results are defined in terms of outcomes that bear on women and girls.

Phot

o: ©

AfD

B

Annexes


Annex A: Evaluation Methodology

The evaluation process adopted a systematic approach that uses different building blocks to construct an answer to the Evaluation Question (EQ) and to formulate key conclusions and forward-looking recommendations. The various phases and subsequent stages coincide with different methodological steps undertaken within the framework of the evaluation:

❙ During the inception phase the Evaluation Team collected the relevant documents and data, gained a clear understanding and overview of the object of the evaluation, mapped AfDB support interventions to Malawi and developed a full methodology.

❙ During the data collection and validation phase, the team completed the data collection at sector and overall levels.

❙ The reporting phase was devoted to constructing answers to the EQs at sector and overall levels and formulating key conclusions and forward-looking recommendations on the basis of the data collected throughout the process.

The methodology applied for this evaluation is based on the evaluation criteria developed by OECD-DAC, but not on impact (not a requirement for this evaluation).

In summary, the evaluation methodology is based on the use of an evaluation matrix that uses evaluation questions, judgment criteria (JC) and indicators as the building blocks for the analysis leading to conclusions and recommendations. In terms of epistemological boundaries, the methodology strongly focusses on results achieved and demonstrated, a Theory of Change approach to intervention design and management, a sector by sector analysis followed by a roll-up of sector-level findings to the program level, evidence-based research, contribution analysis and iterative feedback with key stakeholders to strengthen the validation bonds.

Five main categories of information were collected and analyzed for the evaluation technical report:

❙ Background and context information on the country, the region and the cooperation policy of the Bank. This information is necessary first for an understanding of the main issues addressed, or that should have been addressed, by the Bank support; and second, for an overview of the country’s situation in relation to these issues, so as to form an initial idea of the possible contribution of Bank support to the observed trends. This work was started during the inception phase and then complemented during the field and desk phase with information for each specific sector.

❙ AfDB cooperation with Malawi in general and at the sector level including a systematic review of strategic and programing documents, alignment with the Government of Malawi (GoM) policies, and analysis of the sector intervention logic, a portfolio of the AfDB funding, etc. This started during the inception phase and has continued throughout the evaluation process.

❙ Portfolio. The Bank portfolio has been reviewed on the basis of additional information collected during the field phase. An overview of each intervention is presented in each sector report. Such analysis supported the identification of lessons and provided elements of the answers relating to each of the JC. The available documentation on these interventions has been screened at the level of the relevant JC.

57Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

❙ Case studies: The case studies provide an in-depth analysis of a representative intervention for each sector providing detailed review of the intervention itself, including a Theory of Change and its accompanying assumptions and risks, and an analysis of its evolution and contribution to the sector goals. The case studies were generated through a more in-depth documentation analysis and a thorough interview set in the field. In the documentation, a special effort was made to formally tie the context and management issues of the interventions into the performance actually achieved. In addition, each case study is analyzed to see what contribution it has made to the Malawi program as a whole, including lessons learned. There is no presumption of a typology being used in the selection and use of these case studies as the performance variables are not defined by the Bank except for the overriding variable of supporting the GoM. This handicap does not diminish the validity of the use of the cases; they represent in-depth analyses from which common lessons and findings have been extracted, and from which common lessons learned can be aggregated. It is clear that the overriding factor that distinguishes the case studies from the analysis that was done on other interventions is the extensive use made of the case studies in the analysis of the Country-level Program of the Bank (i.e. at the CSP level).

❙ Interviews: Interviews with key stakeholders (Bank staff, other relevant donors, civil society, Non-Governmental Organizations, beneficiaries, implementing partners) have been carried out for each sector and information has been included in the analysis for each EQ.

The process of data gathering and analysis has been structured in two steps:

❙ Identification, collection and processing of information relevant to all JC and Indicators, all information collected is presented in Volume 2 of the sector reports.

❙ Analysis and synthesis of the information at the level of JC and Indicators. For each JC the team has provided information at sector level on the basis of the data collected and presented in Volume 2 of the sector reports. The team has then analyzed the data across sectors in order to synthesize the information and provide an overall answer for each EQ.

In addition to this Final Technical Report, five sector reports of two volumes each, were prepared during the course of this evaluation. The first volume contained sector context analysis and the responses to the EQ as they applied to the relevant sector. The same Evaluation Matrix used for this report was also used for the sector reports with the exception that sector specific indicators were included in the latter. The evaluation team synthesized and critiqued the analysis of the EQ in each of the sector reports and adapted them to the Evaluation Matrix at the program level. They also actively participated in the development of conclusions and recommendations to the Final Technical Report.

Limitations

There were a number of important constraints/limitations in the application of the evaluation methodology:

❙ It was difficult to meet with and interview some of the key GoM officials, Project Implementation Unit managers and Bank Task Managers who had firsthand knowledge of the interventions. Accordingly, many questions that


were directed at ministries and agencies41 remained unanswered. When possible, the evaluation team mitigated the former constraint by directing questions to other individuals who might have the required knowledge. In many cases this was successful (e.g. vocational training strategies), but in others it was not (e.g. health). The Bank task officers were also asked to fill in the information gaps and, where the question was of a content nature, they were invaluable. They could not, of course, respond to policy or organizational capacity development issues.

❙ There has been some difficulty in obtaining project documents due to retirement of key personnel (i.e. the Bank’s Country Manager and the Country Economist). This caused the response to requests for information to come very late most of the time. However, the evaluation team obtained a good part of information through interviews with the Bank team in the field and through other documents that were available.

❙ The monitoring systems used by the Bank and the GoM do not monitor for outcomes. However this constraint was mitigated via the use of proxy indicators.

❙ For assessing the achievement of objectives, the evaluation considered the objectives included in Project Appraisal Reports PARs. In that context, it should be noted that many interventions42 had their levels of expected results significantly reduced during the midterm reviews (MTR), so the ratings provided by this evaluation are by necessity often lower than those provided in PCRs.

❙ Regarding the impact of weather shocks on the economy, the evaluation team took this ‘climate and periodical’ effect into account in assessing the effectiveness of projects, especially, if the project included weather shocks as a critical risk factor to be carefully monitored.

59Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Annex B: Evaluation Matrix

Evaluation question Judgment criteria Indicators/notes43

EQ 1. To what extent have the Bank’s country strategies and operations been aligned to Malawi’s development needs and its own strategies and priorities?

JC 1.1. The Bank’s country strategies and key operational mechanisms have been aligned with Malawi’s development needs and development strategies and priorities.

❙ Strategy and target correlation between GoM poverty-reduction strategies and development plans and AfDB CSP for Malawi

❙ AfDB operational policies correlate with defined strategies and plans of GoM

❙ AfDB operational policies support development realities of GoM

JC 1.2. AfDB country strategies and key operational mechanisms are aligned with the needs of main (key) ultimate beneficiaries.

❙ Beneficiary impact analyses (or equivalent) inform CSP and intervention design

❙ Beneficiaries are well defined in planning documents ❙ M and E systems of AfDB contain means to measure

progress on beneficiary needs

JC 1.3. Bank interventions, when viewed at a sector performance level in the country, are aligned with its country strategy and priorities.

❙ Intervention design contains analyses that compare country strategies (CSP) to result framework of intervention

❙ New operations rated satisfactory

EQ 2: To what extent has the design and internal management of the Bank’s strategic direction in Malawi reflected a clear, viable and coherent path for achieving development outcomes in a changing national and international context?

JC 2.1. The CSP meets all quality standards of both the Bank and GoM.

❙ AfDB guidelines on CSP quality standards exist ❙ Mechanisms for monitoring and judging quality ❙ Correlation between CSPs and Bank guidelines ❙ Correlation between CSPs and policies of the Bank not

covered in Guidelines

Note: Quality standards were defined in a way that referred to the requirements set out in Bank policies and guidelines. If these are unclear or not comprehensive, these were not in the appropriate documents.

JC 2.2. The Bank has applied selectivity in designing its country portfolio including a focus on areas where it brings added value?

❙ Bank’s own definition of what it considers to be its domains where it has added value

❙ Average CSP rating (from Bank Readiness review)

JC 2.3. The Bank has adapted its approach to the country’s context and development challenges/needs?

❙ Examples of where the Bank has specifically adapted its way of working to take Malawi’s specific context into account.

❙ Examples where GoM believes that the Bank has not been flexible and adaptable.in its approaches

❙ Existence of Country Portfolio Performance Reviews ❙ Follow up of CPPR

JC 2.4. Taken as a whole, the Bank’s interventions have been coherent (between interventions) and were well-coordinated internally.

❙ Interventions are sufficient to contribute to the realization of GoM objectives

❙ Internal strategies and plans are documented for sector and inter-sector leverage (e.g. rural roads having effects on agriculture, social services, trade etc.?)

❙ Internal strategies are documented for using one intervention to leverage another in the same sector



EQ 3: To what extent has the Bank contributed to the achievement of development objectives and results?

JC 3.1. The Bank’s interventions achieved their expected results (intermediate outcomes and outcomes)

❙ Expected effects from each intervention stated, monitored and achieved

❙ Specific sector targets for Bank defined 44: Transport, Power, Agriculture, Education, Governance, Regional integration.

❙ Note: sector-specific indicators will be added to this list. For example, in education the following might be used:

• % students that graduate from Technical, Entrepreneurial and Vocational Education and Training (TEVET) institutions find jobs in their field • % women that graduate from TEVET institutions• TVET training meets expectations of future employers.

JC 3.2. The Bank’s interventions benefited target group members45 ?46

❙ Monitoring reports from the interventions

JC 3.3. The Bank’s interventions contributed to the achievement of higher-level (outcomes and impacts) development objectives, both intended and unintended.

❙ AfDB “Completed operations rated as satisfactory” index ❙ Ex post evaluations of interventions

EQ 4: Is it likely that the benefits realized as a result of Bank interventions will continue to exist once its support is terminated?

JC 4.1 Benefits gained from past interventions (i.e. completed projects have continued to exist in the long-term (five years or more) even though the Bank’s interventions are completed

❙ Existence of intervention and sector sustainability plans ❙ Clear definition as to accountability for sustainability ❙ Follow-up (M and E) actions and execution of sustainability

commitments, including financial, ❙ Evidence of internal sustainability support ❙ Sustainability of climate and biodiversity based on solid

science at individual and sector levels

JC 4.2 Benefits gained from ongoing interventions will likely continue to exist once the Bank’s interventions are completed.

❙ Existence of intervention and sector susinability plans ❙ Clear definition as to accountability for sustainability ❙ Follow-up (M and E) and execution of sustainability

commitments, including financial. ❙ Existence of sustainability actions in place at least 48

months before end of intervention ❙ Completed Operations with Sustainable Outcomes Rating

(COSOR) ❙ Analysis and follow-up of COSOR

EQ 5: To what extent are key cross-cutting policy objectives of the Bank (in terms of inclusiveness47 and a sustainable environment) mainstreamed within the Bank’s interventions?

JC 5.1. The Bank’s interventions have been inclusive (i.e., bringing prosperity by expanding the economic base across the barriers of age and gender.

Because the evaluation dealt with a number of sectors, each with its own set of sector performance indicators, a sampling of the type of indicators that were used for this JC is included below. We assumed that all indicators dealt with the AfDB’s CCO, and not those of the GoM.

❙ Life expectancy ❙ Enrolment in education (gross and net enrolment ratios)48

Graduation ratios. ❙ Enrolment and graduation rates in TVET institutions, as a

share of total students enrolled in primary or secondary institutions

❙ Women’s and youth participation in the labor market. ❙ Average salaries by gender and age ❙ Gender-sensitive Country Institutions Index49 and its

component parts, ❙ Access to finance for business Gender-informed design in

results framework

61Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


❙ New projects with gender informed design (with sub indicators)

JC 5.2. The Bank’s interventions have been inclusive (i.e., expanding the economic base across all regions where poverty is most prevalent.

❙ Access to basic services including water sources; improved sanitation facilities; education facilities and services;

❙ Improvements in population in GDP per capita; Population living below poverty line; Gini index calculations or equivalent;

❙ Access to power; Access to markets and services through road networks (road density as a proxy)

❙ Share of services in rural versus urban contexts ❙ Percentage of people below poverty line ❙ Country Policy and Institutional Assessments ratings ❙ Access to finance (adults that have accounts in financial

institutions);50

JC 5.3. The Bank’s interventions are managed in a manner that will ensure that they are environmentally sustainable and/or support the transition to green growth.

Depending on the sector or the specific cross-cutting issue, the following indicators were retained

❙ Food insecurity: Percentage of population at risk of caloric inadequacy51

❙ Resilience to water shocks52

❙ Environmental management plans included in all stages of project planning and contracting

❙ Reporting on environmental risk mitigation plans ❙ Environmental monitoring by intervention steering

committee (refer to agendas and minutes) ❙ Independent audits or monitoring of environmental

plans within interventions ❙ Capability assessments for environmental

stewardship (policy) and operational management (interventions)

❙ Documented tie-in between interventions and green growth strategies and plans

❙ Agricultural value-added per agricultural worker as a proxy for green growth

❙ New projects with climate-informed design (with sub-indicators for share of projects that included satisfactory actions to mitigate against impacts of climate change, climate variability, extreme weather events and biodiversity loss

❙ Premiums for Interventions that actually promote green growth

EQ 6: To what extent have the Bank’s interventions in Malawi been successful in economically converting resources into expected results in a timely manner?

JC 6.1 Resources and inputs provided to GoM through the Bank’s support are converted to results at a rate that reflects the assumptions in planning documents

❙ Development resources recorded on budget53 ❙ Operations no longer at-risk ratio ❙ Operations at risk ❙ Cost of transforming inputs into outputs.

JC 6.2 The Bank’s interventions are implemented in a timely manner (from the perspective of the beneficiary or implementing agency) and in compliance with the Bank’s operational standards

❙ Predictable disbursements ❙ Use of country systems (PFM and procurement) ❙ Time to first disbursement ❙ Disbursement ratio of ongoing portfolio ❙ Time for procurement of goods and works ❙ Time for approving operations



❙ Time from identification to first disbursement ❙ Extent of projects managed entirely from field ❙ Administrative cost per UA 1 million planned ❙ Cost of preparing lending project or grant project ❙ Cost of supporting project planning ❙ Cost of supporting project implementation

JC 6.3 The GoM and intervention partners and stakeholders meet their commitments as planned (ex. delivery of national commitments/contributions and short comings in performance by national partner(s)

❙ Mention of non-compliance in steering and management meetings

❙ Intervention reports and evaluations are positive concerning non-Bank contributions and compliance.

EQ 7: To what extent has the Bank been successful in influencing policy through dialogue supported by the provision of expert advice and analysis?

JC 7.1. The Bank has actively engaged in and influenced PD through the provision of relevant advice that has been incorporated into policy decisions

❙ Strategic nature of PD ❙ Strategy documents showing the Bank’s PD plan ❙ Diversity of PD effort, geared to objectives ❙ Perception of usefulness of PD by cooperation

partners ❙ Perception of PD by GoM officials ❙ Examples of effects of PD ❙ GoM capability to use PD as a strategic tool

JC 7.2 The Bank provided appropriate and adequate analytical work in support of its interventions, positioning and policy advice?

❙ New economic and sector papers produced using intervention modality

❙ New economic and sector papers produced using existing Bank personnel modality

❙ Perceived value of advice ❙ Capability of GoM to generate evidence-based

analysis for policies relevant to Bank

EQ 8: To what extent are the Bank’s interventions coordinated with other donors and stakeholders in a way that improves efficiency and enables complementarity?

JC 8.1. The Bank’s processes and interventions are harmonized with those of other donors (avoiding duplication, simplifying procedures etc.).

❙ Level of complementarity between donors ❙ Joint evaluations and reviews ❙ Joint strategic planning ❙ Joint undertakings managed jointly ❙ Use of non-Bank processes

JC 8.2. The Bank’s interventions and resources are influencing other stakeholders to become involved in the sectors that are the focus of country strategies

❙ Cases where new (non-traditional) sources of funding have been forthcoming based on opportunities with the Bank

❙ Recent changes in membership in sector management fora

JC 8.3 The Bank’s participation in Malawi is leveraged by other stakeholders with a view to maximizing development effectiveness at both country and regional levels?

❙ Subsidiarity ❙ Joint financing under Bank management

63Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


EQ 9: To what extent has the Bank managed its interventions and programs, as well as its own internal processes, in a way that focusses on results54?

JC 9.1. The Bank has successfully implemented management systems that focus on results and allow learning from past experience?

❙ Intervention design is RBM ❙ Monitoring systems focus on results ❙ Reporting mechanisms are results based ❙ Oversight is done at result levels

JC 9.2 The Bank has supported the development of national capacities and management systems that focus on results

❙ Results-based capacity development efforts for GoM that are related to outcome improvement

❙ % of GoM partners that have RBM-based internal systems

EQ 10: Does the Bank’s ongoing analyses of its achievement of development results inform its planning and operations?

JC 10.1. Factors that have positively or negatively influenced the achievement of the development results supported by Bank Interventions are identified55 and inform56 Bank actions and strategies

❙ Means in place to capture positive and negative factors and store in a KM system

❙ Evidence of use of KM in the design of CSP and interventions

JC 10.2. Lessons learned from the Bank’s experience in Malawi are identified and inform its future strategies and programs.

❙ Completed operations with a timely PCR (measure of the percentage of PCR that were submitted within the 12 months following closure date)

❙ Systems allowing research quality access to Lessons Learned

❙ Systems allowing for capture of knowledge ❙ Justifications for CSP that refer to Lessons Learned


Annex C: Project List

Sector Dept.

N. Long name Status Sector Name Year of commitment Duration Net loan (UA) Disbursement Type of service provided

AHAI 1 FOOD CRISIS RESPONSE BUDGET SUPPORT PROGRAM approved Agriculture MINISTRY OF FINANCE 2016 1 12,000,000 0 grant

AHAI 2 SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN) ongoing Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2013 5 29,306,198 15,718,991 grant

AHAI 2 SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN

ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY 2013 5 253,000 194,964 study

PIFD 3 FEASIBILITY STUDY ON THE ESTABLISHMENT OF AN AGRICULTURE COO

ongoing AgricultureMINISTRY OF FINANCE 2016 1 365,000 30,689 study

AHAI 4 HUMANITARIAN EMERGENCY ASSISTANCE TO MITIGATE EFFECTS OF 2015 complete Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2016 0,5 740,056 0 grant

AHAI 5 AGRICULTURE DEVELOPMENT PROGRAM - ISP ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY 2009 7 15,000,000 14,532,277 loan

AHFR 6 (SCPMP) SMALLHOLDER CROP PRODUCTION AND MARKETING PROJECT (co-financing loan with CARLA project below) complete Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2006 8,9 14,882,276 14,882,276 loan

AHFR 7 SHIRE VALLEY IRRIGATION PROJECT FEASIBILITY STUDY ongoing Agriculture Ministry of Irrigation and Water Develop 2014 4 1,414,629 1,100,586 study

AHAI 8 AGRICULTURAL INFRASTRUCTURE AND YOUTH AGRIBUSINESS PROJECT approved Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2016 5 16,000,000 0 loan

AHAI 9 HUMANITARIAN EMERGENCY ASSISTANCE TO THE 2005 DROUGHT VICTIM closed Agriculture WORLD FOOD PROGRAM - WFP-

MALAWI 2006 1,6 370,028 0 grant

PECG 10GEF CARLA CLIMATE ADAPTATION FOR RURAL LIVELIHOODS AND AGRICULTURE (co-financing grant with SCPMP project above)

closed Agriculture MINISTRY OF IRRIGATION AND WATER DEVELOP 2012 4,5 2,220,167 2,219,716 loan

PICU 11 MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA complete Transport NATIONAL ROAD AUTHORITY 2009 5,2 19,034,450 19,034,450 loan

PICU 11 MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 159,222 159,222 loan

PICU 12 MALAWI: MZUZU-NKHATABAY ROAD REHABILITATION PROJECT ongoing Transport NATIONAL ROAD AUTHORITY 2013 5 21,890,000 4,991,151 loan

PITD 13 MALAWI NACALA RAIL AND PORT VALUE ADDITION PROJECT complete Transport Malawi Investment and Centre (MITC) (Private sector financing) 2017 1,0 0 0 loan

PICU 14 NACALA ROAD CORRIDOR (MALAWI) - phase I complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 8,925,658 8,925,658 loan

PICU 15 NACALA ROAD CORRIDOR PROJECT PHASE IV (LIWONDE-MANGOCHI) MA ongoing Transport NATIONAL ROAD AUTHORITY 2014 4 42,360,000 3,313,755 loan

PESD 16 KOLOMBIDZO HYDRO POWER PROJECT FEASIBILITY STUDY ongoing Power Ministry of Energy Department of Energy 2013 4 2,000,000 1,496,749 study

AHWS 17 MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 15,200,000 14,746,195 loan


AHWS 17 MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 2,774,708 2,774,708 grant

AHWS 18 SUSTAINABLE RURAL WATER AND SANITATION INFRASTRUCTURE FOR IMPROVED HEALTH AND LIVELIHOODS ongoing WATSAN MINISTRY OF IRRIGATION AND

WATER DEVELOP 2014 5 15,000,000 2,011,450 loan


WATER DEVELOP 2014 5 2,761,407 720,664 grant

65Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Annex C: Project List

Sector Dept.


AHAI 1 FOOD CRISIS RESPONSE BUDGET SUPPORT PROGRAM approved Agriculture MINISTRY OF FINANCE 2016 1 12,000,000 0 grant

AHAI 2 SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN) ongoing Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2013 5 29,306,198 15,718,991 grant

AHAI 2 SMALLHOLDER IRRIGATION AND VALUE ADDITION PROJECT (SIVAP/FUN

ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY 2013 5 253,000 194,964 study

PIFD 3 FEASIBILITY STUDY ON THE ESTABLISHMENT OF AN AGRICULTURE COO

ongoing AgricultureMINISTRY OF FINANCE 2016 1 365,000 30,689 study

AHAI 4 HUMANITARIAN EMERGENCY ASSISTANCE TO MITIGATE EFFECTS OF 2015 complete Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2016 0,5 740,056 0 grant

AHAI 5 AGRICULTURE DEVELOPMENT PROGRAM - ISP ongoing Agriculture MINISTRY OF AGRICULTURE & FOOD SECURITY 2009 7 15,000,000 14,532,277 loan

AHFR 6 (SCPMP) SMALLHOLDER CROP PRODUCTION AND MARKETING PROJECT (co-financing loan with CARLA project below) complete Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2006 8,9 14,882,276 14,882,276 loan

AHFR 7 SHIRE VALLEY IRRIGATION PROJECT FEASIBILITY STUDY ongoing Agriculture Ministry of Irrigation and Water Develop 2014 4 1,414,629 1,100,586 study

AHAI 8 AGRICULTURAL INFRASTRUCTURE AND YOUTH AGRIBUSINESS PROJECT approved Agriculture MINISTRY OF AGRICULTURE &

FOOD SECURITY 2016 5 16,000,000 0 loan

AHAI 9 HUMANITARIAN EMERGENCY ASSISTANCE TO THE 2005 DROUGHT VICTIM closed Agriculture WORLD FOOD PROGRAM - WFP-

MALAWI 2006 1,6 370,028 0 grant

PECG 10GEF CARLA CLIMATE ADAPTATION FOR RURAL LIVELIHOODS AND AGRICULTURE (co-financing grant with SCPMP project above)

closed Agriculture MINISTRY OF IRRIGATION AND WATER DEVELOP 2012 4,5 2,220,167 2,219,716 loan

PICU 11 MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA complete Transport NATIONAL ROAD AUTHORITY 2009 5,2 19,034,450 19,034,450 loan

PICU 11 MALAWI TRUNK ROAD REHABILITATION: BLANTYRE-ZOMBA complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 159,222 159,222 loan

PICU 12 MALAWI: MZUZU-NKHATABAY ROAD REHABILITATION PROJECT ongoing Transport NATIONAL ROAD AUTHORITY 2013 5 21,890,000 4,991,151 loan

PITD 13 MALAWI NACALA RAIL AND PORT VALUE ADDITION PROJECT complete Transport Malawi Investment and Centre (MITC) (Private sector financing) 2017 1,0 0 0 loan

PICU 14 NACALA ROAD CORRIDOR (MALAWI) - phase I complete Transport NATIONAL ROAD AUTHORITY 2009 6,0 8,925,658 8,925,658 loan

PICU 15 NACALA ROAD CORRIDOR PROJECT PHASE IV (LIWONDE-MANGOCHI) MA ongoing Transport NATIONAL ROAD AUTHORITY 2014 4 42,360,000 3,313,755 loan

PESD 16 KOLOMBIDZO HYDRO POWER PROJECT FEASIBILITY STUDY ongoing Power Ministry of Energy Department of Energy 2013 4 2,000,000 1,496,749 study



AHWS 17 MALAWI RWSSI - RURAL WATER SUPPLY AND SANITATION INITIATIVE complete WATSAN GOVERNMENT OF MALAWI 2008 5,0 2,774,708 2,774,708 grant




WATER DEVELOP 2014 5 2,761,407 720,664 grant


Sector Dept.




AHWS 19 MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT ongoing WATSAN -- 2015 5 10,989,824 0 loan

AHWS 19 MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT ongoing WATSAN -- 2015 5 3,600,000 122,649 loan

AHWS 20 STRENGTHENING WATER SECTOR MONITORING AND EVALUATION closed WATSAN MINISTRY OF IRRIGATION AND

WATER DEVELOP 2010 3,4 1,494,315 1,494,315 grant

AHWS 21 IMPROVING ACCESS TO WATER & SANITATION FOR URBAN POOR IN BLANTYRE complete WATSAN CCODE CENTER FOR COMMUNITY

ORGANIZATION 2010 5,0 481,897 481,897 grant

AHHD 22 JOBS FOR YOUTH MALAWI approved social Ministry of Labor 2017 4 0 0 n.a.


AHHD 23 SUPPORT TO SECONDARY EDUCATION (EDUCATION V) PROJECT closed social IDA/ADF EDUCATION PROJECTS 2006 6,6 14,621,981 14,621,981 grant

AHHD 24 Support to Higher Education, Science and Technology (HEST) ongoing social GOVERNMENT OF MALAWI 2012 7 6,500,000 2,477,458 loan

AHHD 24 Support to Higher Education, Science and Technology (HEST) ongoing social GOVERNMENT OF MALAWI 2012 7 10,950,000 5,165,758 grant


AHHD 25 SUPPORT TO THE HEALTH SECTOR PROGRAM complete social MINISTRY OF HEALTH & POPULATION 2006 8,0 14,668,070 14,668,070 grant

AHHD 26 SUPPORT TO LOCAL ECONOMIC DEVELOPMENT ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE 2008 8 14,000,000 13,968,722 loan

AHHD 27 SUPPLEMENTARY LOAN LOCAL ECONOMIC DEVLOP ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE 2011 6 3,162,000 3,136,780 loan

AHHD 28 COMPETITIVENESS AND JOB CREATION SUPPORT PROJECT ongoing social Ministry of Industry and Trade 2012 6 10,000,000 7,318,945 grant

AHHD 29 PROTECTION OF BASIC SERVICES (Health) budget support (ring-fenced)57 ongoing social MINISTRY OF FINANCE 2015 3 19,000,000 19,000,000 grant

ECGF 30 RESTORATION OF FISCAL STABILITY AND SOCIAL PROTECTION complete multisector MINISTRY OF FINANCE 2012 1,4 26,000,000 26,000,000 grant

ECGF 31 SUPPLEMENTARY BUDGET SUPPORT TO RFSSP complete multisector MINISTRY OF FINANCE 2013 0,5 4,000,000 4,000,000 grant

ECGF 32 PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT ongoing multisector MINISTRY OF FINANCE 2013 5 2,980,000 2,080,181 grant

ECGF 33 PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT-PHA ongoing multisector MINISTRY OF FINANCE 2015 3 1,860,000 358,278 grant

ECGF 34 POVERTY REDUCTION SUPPORT LOAN closed multisector MINISTRY OF FINANCE 2007 1,2 14,890,000 14,890,000 loan

ECGF 35 POVERTY REDUCTION SUPPORT GRANT I complete multisector MINISTRY OF FINANCE 2009 0,7 10,000,000 10,000,000 grant

ECGF 36 GOVERNANCE AND POVERTY REDUCTION SUPPORT GRANT II (GPRSG II) complete multisector MINISTRY OF FINANCE 2010 1,0 11,546,000 11,546,000 grant

67Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sector Dept.




AHWS 19 MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT ongoing WATSAN -- 2015 5 10,989,824 0 loan

AHWS 19 MZIMBA INTEGRATED URBAN WATER AND SANITATION PROJECT ongoing WATSAN -- 2015 5 3,600,000 122,649 loan

AHWS 20 STRENGTHENING WATER SECTOR MONITORING AND EVALUATION closed WATSAN MINISTRY OF IRRIGATION AND

WATER DEVELOP 2010 3,4 1,494,315 1,494,315 grant

AHWS 21 IMPROVING ACCESS TO WATER & SANITATION FOR URBAN POOR IN BLANTYRE complete WATSAN CCODE CENTER FOR COMMUNITY

ORGANIZATION 2010 5,0 481,897 481,897 grant



AHHD 23 SUPPORT TO SECONDARY EDUCATION (EDUCATION V) PROJECT closed social IDA/ADF EDUCATION PROJECTS 2006 6,6 14,621,981 14,621,981 grant


AHHD 24 Support to Higher Education, Science and Technology (HEST) ongoing social GOVERNMENT OF MALAWI 2012 7 10,950,000 5,165,758 grant


AHHD 25 SUPPORT TO THE HEALTH SECTOR PROGRAM complete social MINISTRY OF HEALTH & POPULATION 2006 8,0 14,668,070 14,668,070 grant

AHHD 26 SUPPORT TO LOCAL ECONOMIC DEVELOPMENT ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE 2008 8 14,000,000 13,968,722 loan

AHHD 27 SUPPLEMENTARY LOAN LOCAL ECONOMIC DEVLOP ongoing social MINISTRY OF LOCAL GOVERNMENT & RURAL DE 2011 6 3,162,000 3,136,780 loan

AHHD 28 COMPETITIVENESS AND JOB CREATION SUPPORT PROJECT ongoing social Ministry of Industry and Trade 2012 6 10,000,000 7,318,945 grant

AHHD 29 PROTECTION OF BASIC SERVICES (Health) budget support (ring-fenced)57 ongoing social MINISTRY OF FINANCE 2015 3 19,000,000 19,000,000 grant

ECGF 30 RESTORATION OF FISCAL STABILITY AND SOCIAL PROTECTION complete multisector MINISTRY OF FINANCE 2012 1,4 26,000,000 26,000,000 grant

ECGF 31 SUPPLEMENTARY BUDGET SUPPORT TO RFSSP complete multisector MINISTRY OF FINANCE 2013 0,5 4,000,000 4,000,000 grant

ECGF 32 PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT ongoing multisector MINISTRY OF FINANCE 2013 5 2,980,000 2,080,181 grant

ECGF 33 PUBLIC FINANCE MANAGEMENT INSTITUTIONAL SUPPORT PROJECT-PHA ongoing multisector MINISTRY OF FINANCE 2015 3 1,860,000 358,278 grant

ECGF 34 POVERTY REDUCTION SUPPORT LOAN closed multisector MINISTRY OF FINANCE 2007 1,2 14,890,000 14,890,000 loan

ECGF 35 POVERTY REDUCTION SUPPORT GRANT I complete multisector MINISTRY OF FINANCE 2009 0,7 10,000,000 10,000,000 grant

ECGF 36 GOVERNANCE AND POVERTY REDUCTION SUPPORT GRANT II (GPRSG II) complete multisector MINISTRY OF FINANCE 2010 1,0 11,546,000 11,546,000 grant


Annex D: Sector Level and Overall Performance

Evaluation question Judgment criteria

Soci

al

Tran

spor

t

Agric

ultu

re

WAT

SAN

Mul

tisec

tor

Over

all

asse

ssm

ent

EQ 1. To what extent have the Bank’s country strategies and operations been aligned to Malawi’s development needs and its own strategies and priorities?

1.1 The Bank’s country strategies and key operational mechanisms have been aligned with Malawi’s development needs and development strategies and priorities.

3.1 3.3 3.5 3.0 3.5 Satisfactory

1.2 AfDB country strategies and key operational mechanisms are aligned with the needs of main (key) ultimate beneficiaries.

3.8 2.9 3.6 2.7 3.5 Satisfactory

1.3 Bank interventions, when viewed at a sector performance level in the country, are aligned with its country strategy and priorities.

3.9 3.0 3.7 3.0 2.0 Satisfactory

EQ 2. To what extent has the design and internal management of the Bank’s strategic direction in Malawi reflected a clear, viable and coherent path for achieving development outcomes in a changing national and international context?

2.1 The CSP meets all quality standards of both the Bank and GoM.

3.0 3.0 3.0 2.5 4.0 Satisfactory

2.2 The Bank has applied selectivity in designing its country portfolio including a focus on areas where it brings added value.

3.2 3.0 3.2 3.5 2.0 Satisfactory

2.3 The Bank has adapted its approach to the country’s context and development challenges/needs.

2.5 2.5 3.0 2.5 3.0 Unsatisfactory

2.4 Taken as a whole, the Bank’s interventions have been coherent (between interventions) and were well-coordinated internally.

3.0 3.3 3.0 3.3 3.0 Satisfactory

EQ 3. To what extent has the Bank contributed to the achievement of development objectives and results?

3.1 The Bank’s interventions achieved their expected results (intermediate outcomes and outcomes).

2.8 3.1 2.5 3.5 2.0 Satisfactory

3.2 The Bank’s interventions benefited target group members.

3.3 3.5 2.5 3.0 NA Satisfactory

3.3 The Bank’s interventions contributed to the achievement of higher level (outcomes and impacts) development objectives, both intended and unintended.


EQ 4. Is it likely that the benefits realized as a result of Bank interventions will continue to exist once its support is terminated?

4.1 Benefits gained from past interventions (i.e. completed projects have continued to exist in the long-term (five years or more) even though the Bank’s interventions are completed.


4.2 Benefits gained from ongoing interventions will likely continue to exist once the Bank’s interventions are completed.


69Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n


Soci

al

Tran

spor

t

Agric

ultu

re

WAT

SAN

Mul

tisec

tor

Over

all

asse

ssm

ent

EQ 5. To what extent are key crosscutting policy objectives of the Bank(in terms of inclusiveness and a sustainable environment) mainstreamed within the Bank’s interventions?

5.1 The Bank’s interventions have been inclusive (i.e., bringing prosperity by expanding the economic base across the barriers of age and gender.

3.6 3.0 3.8 4.0 3.0 Satisfactory

5.2 The Bank’s interventions have been inclusive (i.e., expanding the economic base across all regions where poverty is most prevalent.

3.8 3.0 3.0 3.0 NA Satisfactory

5.3 The Bank’s interventions are managed in a manner that will ensure that they are environmentally sustainable and/or support the transition to green growth.

3.0 4.0 3.0 3.0 3.0 Satisfactory

EQ 6. To what extent have the Bank’s interventions in Malawi been successful in economically converting resources into expected results in a timely manner?

6.1 Resources and inputs provided to GoM through the Bank’s support are converted to results at a rate that reflects the assumptions in planning documents.

3.2 2.5 3.0 3.0 2.5 Satisfactory

6.2 The Bank’s interventions are implemented in a timely manner (from the perspective of the beneficiary or implementing agency) and in compliance with the Bank’s operational standards.


6.3 The GoM and intervention partners and stakeholders meet their commitments as planned (ex. delivery of national commitments/contributions and short comings in performance by national partner(s).


EQ 7. To what extent has the Bank been successful in influencing policy through dialogue supported by the provision of expert advice and analysis?

7.1 The Bank has actively engaged in and influenced PD through the provision of relevant advice that has been incorporated into policy decisions.

3.7 3.0 3.4 3.0 3.0 Satisfactory

7.2 The Bank provided appropriate and adequate analytical work in support of its interventions, positioning and policy advice.

3.6 3.3 3.5 3.0 3.0 Satisfactory

EQ 8. To what extent are the Bank’s interventions coordinated with other donors and stakeholders in a way that improves efficiency and enables complementarity?

8.1 The Bank’s processes and interventions are harmonized with those of other donors (avoiding duplication, simplifying procedures etc.).

3.2 3.0 3.0 2.7 3.5 Satisfactory

8.2 The Bank’s interventions and resources are influencing other stakeholders to become involved in the sectors that are the focus of country strategies.

3.0 3.0 3.0 3.0 3.0 Satisfactory

8.3 The Bank’s participation in Malawi is leveraged by other stakeholders with a view to maximizing development effectiveness at both country and regional levels.

3.3 3.0 3.5 3.0 2.0 Satisfactory



Soci

al

Tran

spor

t

Agric

ultu

re

WAT

SAN

Mul

tisec

tor

Over

all

asse

ssm

ent

EQ 9. To what extent has the Bank managed its interventions as well as its own internal processes, in a way that focusses on results?

9.1 The Bank has successfully implemented management systems that focus on results and allow learning from past experience.


9.2 The Bank has supported the development of national capacities and management systems that focus on results.


EQ 10. Does the Bank’s ongoing analyses of its achievement of development results inform its planning and operations?

10.1 Factors that have positively or negatively influenced the achievement of the development results supported by Bank Interventions are identified and inform Bank actions and strategies.

3.2 2.6 3.0 2.5 3.0 Satisfactory

10.2 Lessons learned from the Bank’s experience in Malawi are identified and inform its future strategies and programs.

3.7 2.8 3.7 3.0 3.0 Satisfactory

71Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Year Malawi Africa Developing countries

Developed countries

Basic Indicators

Area ( '000 Km²) 2017 118 30,067 80,386 53,939

Total Population (millions) 2017 18.3 1,184.5 5,945.0 1,401.5

Urban Population (% of Total) 2017 16.7 39.7 47.0 80.7

Population Density (per Km²) 2017 194.1 40.3 78.5 25.4

GNI per Capita (US $) 2016 320 2 045 4 226 38 317

Labor Force Participation *- Total (%) 2017 81.1 66.3 67.7 72.0

Labor Force Participation **- Female (%) 2017 81.5 56.5 53.0 64.5

Sex Ratio (per 100 female) 2017 99.8 0.801 0.506 0.792

Human Develop. Index (Rank among 187 countries) 2015 170 ... ... ...

Popul. Living Below $ 1.90 a Day (% of Population) 2010 70.9 39.6 17.0 ...

Demographic Indicators

Population Growth Rate - Total (%) 2017 3.1 2.6 1.3 0.6

Population Growth Rate - Urban (%) 2017 4.1 3.6 2.6 0.8

Population < 15 years (%) 2017 44.7 41.0 28.3 17.3

Population 15–24 years (%) 2017 20.4 3.5 6.2 16.0

Population >= 65 years (%) 2017 3.4 80.1 54.6 50.5

Dependency Ratio (%) 2017 92.9 100.1 102.8 97.4

Female Population 15-49 years (% of total population) 2017 23.3 24.0 25.8 23.0

Life Expectancy at Birth - Total (years) 2017 65.5 61.2 68.9 79.1

Life Expectancy at Birth - Female (years) 2017 66.4 62.6 70.8 82.1

Crude Birth Rate (per 1,000) 2017 37.7 34.8 21.0 11.6

Crude Death Rate (per 1,000) 2017 6.9 9.3 7.7 8.8

Infant Mortality Rate (per 1,000) 2016 38.9 52.2 35.2 5.8

Child Mortality Rate (per 1,000) 2016 55.1 75.5 47.3 6.8

Total Fertility Rate (per woman) 2017 4.9 4.6 2.6 1.7

Maternal Mortality Rate (per 100,000) 2015 634.0 411.3 230.0 22.0

Women Using Contraception (%) 2017 59.8 35.3 62.1 ...

Annex E: Malawi – Comparative Socioeconomic Indicators


Year Malawi Africa Developing countries

Developed countries

Health & Nutrition Indicators

Physicians (per 100,000 people) 2009 1.8 46.9 118.1 308.0

Nurses and midwives (per 100,000 people) 2009 33.6 133.4 202.9 857.4

Births attended by Trained Health Personnel (%) 2016 89.8 50.6 67.7 ...

Access to Safe Water (% of Population) 2015 90.2 71.6 89.1 99.0

Access to Sanitation (% of Population) 2015 41.0 51.3 57 69

Percent. of Adults (aged 15-49) Living with HIV/AIDS 2016 9.2 39.4 60.8 96.3

Incidence of Tuberculosis (per 100,000) 2016 159.0 3.8 1.2 ...

Child Immunization Against Tuberculosis (%) 2016 86.0 245.9 149.0 22.0

Child Immunization Against Measles (%) 2016 81.0 84.1 90.0 ...

Underweight Children (% of children under 5 years) 2014 16.7 76.0 82.7 93.9

Prevalence of stunding 2014 42.4 20.8 17.0 0.9

Prevalence of undernourishment (% of pop.) 2015 25.9 2 621 2 335 3 416

Public Expenditure on Health (as % of GDP) 2014 6.0 2.7 3.1 7.3

Education Indicators

Gross Enrolment Ratio (%)

Primary School - Total 2016 139.3 106.4 109.4 101.3

Primary School - Female 2016 141.3 102.6 107.6 101.1

Secondary School - Total 2016 37.4 54.6 69.0 100.2

Secondary School - Female 2016 35.3 51.4 67.7 99.9

Primary School Female Teaching Staff (% of Total) 2015 42.0 45.1 58.1 81.6

Adult literacy Rate - Total (%) 2015 62.1 61.8 80.4 99.2

Adult literacy Rate - Male (%) 2015 69.8 70.7 85.9 99.3

Adult literacy Rate - Female (%) 2015 55.2 53.4 75.2 99.0

Percentage of GDP Spent on Education 2016 4.7 5.3 4.3 5.5

Environmental Indicators

Land Use (Arable Land as % of Total Land Area) 2015 40.3 8.6 11.9 9.4

Agricultural Land (as % of land area) 2015 61.4 43.2 43.4 30.0

Forest (As % of Land Area) 2015 33.4 23.3 28.0 34.5

Per Capita CO2 Emissions (metric tons) 2014 0.1 1.1 3.0 11.6

* Labor force participation rate, total (% of total population ages 15+)

** Labor force participation rate, female (% of female population ages 15+)

Note: n.a.: Not Applicable; ...:Data Not Available.

Sources: AfDB Statistics Department Databases; World Bank: World Development Indicators; UNAIDS; UNSD; WHO, UNICEF, UNDP; Country Reports.

last update: May 2018

73Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

GNI Per Capita (US $)

0

500

1000

1500

2000

2500

201620152014201320122011201020052000

Malawi Africa

Life expectancy at birth (years)

30

40

50

60

70

201720162015201420132012201020052000

Malawi Africa

Population growth rate (%)

2.0

2.5

3.0

3.5

201720162015201420132012201020052000

Malawi Africa

Infant mortality rate (Per 1000)

0

20

40

60

80

100

120

201620152014201320122011201020052000

Malawi Africa


Bibliography

African Development Bank Strategic Documents

African Development Bank, African Development Fund, Malawi 2005–2009 Country Strategy, September 2005

African Development Bank, African Development Fund, Malawi Results Based Country Strategy Paper: 2005-2009, Mid-Term Review Report, April 2008

African Development Bank Group, African Development Bank Group and Malawi. Thirty years of Fruitful Cooperation, November 2008

African Development Bank Group, Malawi Interim Country Strategy Paper (ICSP) 2011–2012, March 2011

African Development Bank, African Development Fund, Southern Africa Regional Integration Strategy Paper 2011–2015

African Development Bank Group, Combined Mid-Term Review and Regional Portfolio Performance Review of The Regional Integration Strategy Paper for

West Africa 2011 – 2015, September 2014

African Development Bank Group, Malawi Country Strategy Paper 2013–2017, ORSB DEPARTMENT

2015–2016 Budget Statement, Ministry of Finance

2016–2017, Budget Speech

Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume I: Summary of Main Findings and Recommendations

Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume II: Background Papers

❙ Constraints on Growth in Malawi ❙ Malawi’s Growth and Diversification ❙ Quantitative Value Claim Analysis ❙ Challenges in Malawi’s Agribusiness ❙ Connecting Malawi to Markets to Compete ❙ Infrastructure Investment in Malawi: Insights form a Spatial Analysis ❙ The Energy Constraint ❙ Key Messages

Main Assessment Report on Country National Competitive Bidding Procedures, OPRF, Aug. 2011

Critical Assessment Table for use of Borrower’s Procurement System in Bank Financial Operations: Cover Note: AfDB, May 2016

Bank Corporate DocumentsAfrican Development Bank, African Development Fund, Evaluation of Paris Declaration Implementation at the African Development Bank, May 2011

African Development Bank Group, The One Bank Results Measurement Framework 2013-2016, December 2013

African Development Bank Group, At the Center of Africa’s Transformation. Strategy for 2013–2022, Executive summary, 2013

African Development Bank Group, At the Center of Africa’s Transformation. Strategy for 2013–2022, 2013

African Development Bank Group, Annual Development Effectiveness Review 2015. Driving development through innovation, May 2015

African Development Bank Group, IDEV Annual Report 2015. Strengthening Impact, April 2016

African Development Bank Group, Comprehensive Evaluation of the Development Results of the African Development Bank Group 2004-2013. Synthesis Report, October 2016

Amendments to the Approved Development and Business Delivery Model (DBDM) October 2016

Operations Evaluation department: “Fostering Regional Integration in Africa: An Evaluation of the Bank’s Multinational Operations 2000-2010” Bank Publication, 2012

Operations Evaluation department: “Country Assistance Evaluation, Gabon, 1996-2008” Bank Publication, 2011

Operations Evaluation department: “Morocco: Evaluation of Bank Assistance to the Education Sector”” Bank Publication, 2005

Government of Malawi Strategic DocumentsGovernment of Malawi, Malawi Growth and Development Strategy. From Poverty to Prosperity 2006–2011

Government of Malawi, Malawi Growth and Development Strategy II 2011–2016

Public Finance Management Assessment for Malawi based on PEFA, June 2008

Public Finance Management Assessment for Malawi based on PEFA, March 2011

75Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Government of Malawi, Budget Speech 2015/16

Government of Malawi, Draft Training Needs Assessment Report for the Procurement Cluster, March 2011, EU / Government of Malawi

Capacity assessment and development report: Capacity in Public Financial Management and Accountability, 15 August 2008, UNDP/Government of Malawi

Malawi Development Assistance Strategy 2006–2011, Government of Malawi, September 2007

PFEM Situation Analysis Report, June 2010, PFEM Unit, Ministry of Finance

Revised Priority Public Financial and Economic Management Action Plan, September 2010–July 2011, PFEM Secretariat/Ministry of Finance, September 2010

Public Financial and Economic Management Reform Program, PFEM RP, July 2011–June 2014, Ministry of Finance, May 2011

Public Financial Management Strategy and Action Plan 2014–17, Ministry of Finance, June 2014

THE PUBLIC FINANCE MANAGEMENT REFORM PROGRAM, “Restoring Financial Control and Accountability” - 2015, Lilongwe, February 2015, Government of Malawi

Draft: A Synopsis of the Revised Public Finance Management Reform Program and a Revised Engagement Framework with Budget Support DP, Government of Malawi, October 2014

Draft report: Malawi Public Financial Management Strategy and Action Plan, June 2014

Statement by His Excellency, Prof. Arthur Peter Mutharika, President of the Republic Of Malawi at the Launch of the Public Finance Management Reform Program, April, 2015.

Progress Report: Extraordinary Performance Assessment Framework, March 2014.

Progress Report: PFM Action Plan, March 2016

Zero draft, PFM Rolling Plan 2017–22, Ministry of Finance, 2017

African Development Bank Sector Documents

Social sectorSocial Sector - Appraisal ReportsAfrican Development Fund, Support to the Health Sector Program. Appraisal Report, ONSD, September 2005

African Development Bank Group, Support to Secondary Education (Education V) Project. Appraisal Report, ONSD 1, February 2006

African Development Bank Group, Support to Local Economic Development Project. Project Appraisal Report, July 2008

African Development Fund, Supplementary Loan for Support to Local Economic Development Project. Project Appraisal Report, September 2010

African Development Bank Group, Support to Higher Education, Science and Technology (HEST). Project Appraisal Report, December 2011

African Development Fund, Malawi Protection of Basic Services. Appraisal Report, OSHD/SARC, April 2015

Social Sector-PCRAfrican Development Bank Group, Project Completion Report for Malawi: Support to Community Day Secondary Schools (Education IV) Project

African Development Bank Group, Project Completion Report for Public Sector Operations (PCR). Support to the Health Sector Program

Social Sector-Other Intervention Documents Implementation Progress and Results Reports: various, including PBS and HEST

Mid-term Review of Education V,

Chimombo, L.J. “Quantity versus Quality: Case Studies in Malawi”, International Review of Education, 2005:51

Kayuni, H., and Tambulasi, R., “Teacher Turnover in Malawi’s Ministry of Education: Realities and Challenges”, International Education Journal, April 2007

Green LW, Kreuter MW. Health Promotion Planning: An Educational and Environmental Approach. 2nd ed. Mountain View: Mayfield; 1991.

Freudenberg N, Eng E, Flay B, Parcel G, Rogers T, Wallerstein N. Strengthening individual and community capacity to prevent disease and promote health: In search of relevant theories and principles. Health Education Quarterly. 1995; 22:290–306. [PubMed]

Lewis SJK. An overview of health education activities and proposals for strengthening of health education services in Malawi. Lilongwe: Ministry of Health, Malawi; 1981.

Bomba WG. Strengthening of I.E.C & Social Mobilization Unit of Ministry of Health. Lilongwe: Ministry of Health; 1990.

Jiyani E. Paper presented at the peer education national development strategy workshop in Zomba, Malawi from 31st January to 1st February 2000. Lilongwe: Ministry of Gender Youth and Community Services; 2000. Peer education concept paper.

Southern Africa Regional LED Conference paper: “Creating Opportunities for Developmental Local Government through Regional Engagement on LED” March 2014, Lilongwe

AfDB Sector Studies

African Development Bank Fund, Multi-Sector Country Gender Profile, October 2005


African Development Bank Group, Enhancing Linkages with Vertical Health Funds Country Case Study: Malawi, July 2009

ESMPMalawi: Support to Higher Education, Science and Technology (HEST) Project. Environmental and Social Management Plan (ESMP)

Transport sectorImplementation Progress and Results Report (IPR), Mzuzu-Nkhata Bat Road Rehabilitation Project, 10/10/2016

An Evaluation of Environmental Mainstreaming in AfDB Support to the Road Transport Sub-Sector, OPEV, June 2010

The Mchenga Urban Poor Fund, CCODE

Inception Report: Data Collection on Nacala Corridor Integrated Development in Southern Africa, JICA, March 2017

Project Completion Report (PCR): Multinational Nacala Road Corridor Development Project: Phase I, 31/10/2016

Project Completion Report (PCR): Malawi Trunk Roads Rehabilitation Project (Blantyre-Zomba), 03/10/2016

Main Assessment Report on Country National Competitive Bidding Procedures, OPRF, Aug. 2011

Combined Mid-Term Review (MTR) of CSP 2013 – 2017 and Country Portfolio Performance Review 2015, 18/10/2016

Critical Assessment Table for use of Borrower’s Procurement System in Bank Financial Operations: Cover Note: AfDB, May 2016

Completion Report: RBCSP 2005-2010, June 2010

Project Appraisal Report (PAR): Nacala Corridor Project Phase III, Sept. 2012

Multinational Nacala Road Corridor Development Project: Phases I, II, III and IV – Progress on roadworks.

Guidelines for Preparation of a PCR to be submitted by the recipient to the AWF, AfDB: AWF Manual 8.1

Water Supply and Sanitation in Africa: Findings, Lessons and Good Practices to Improve Delivery, AfDB, June 2015

Preparation Mission Aide Memoire: Mzuzu-Nkhata Bay Road Rehabilitation Project: May/June 2012

Project Concept Note: Mzuzu-Nkhata Bay Road Rehabilitation Project: June 2012

Counterpart Funding Waiver: Mzuzu-Nkhata Bay Road Rehabilitation Project: July 2012

Project Appraisal Report (PAR): Mzuzu-Nkhata Bay Road Rehabilitation Project: Oct 2012

Aide Memoire: AfDB Transport Sector Supervision Mission, Feb/March 2017

Five Year Strategic and Business Plan, Vol. 1, Roads Authority, 2011-2016

Aide Memoire: AfDB Transport Sector Supervision Mission, Aug 2016

Aide Memoire: AfDB Transport Sector Supervision Mission, Nov 2015

Aide Memoire: AfDB Transport Sector Supervision Mission, Jan 2014

Aide Memoire: AfDB Transport Sector Supervision Mission, Aug 2014

Aide Memoire: Supervision Mission of Malawi Trunk Roads Rehabilitation Project and Launch: Mzuzu-Nkhata Bay Road Rehabilitation Project, July 2014

Aide Memoire: AfDB Transport Sector Supervision Mission, April 2012

Aide Memoire: Appraisal Mission: Mzuzu-Nkhata Bay Road Rehabilitation Project, Aug. 2012

Aide Memoire: AfDB Transport Sector Supervision Mission, September 2011

Back-to-Office Report (BTOR): Preparation Missions, Nacala Corridor Development Project, Feb/March 2013

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Sept. 2011

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Feb 2013

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, April 2012

Back-to-Office Report (BTOR): Multinational Road Corridor Development Project Phase IV: SOBP Launch Malawi/Zambia, April 2013

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, July 2013

Back-to-Office Report (BTOR): Transport Sector Supervision Mission, Sept. 2011

Back-to-Office Report (BTOR): Nacala Development Corridor OSBP Program, Project Steering Committee, June 2014

Internal Memorandum: Appraisal of Mzuzu-Nkhata Bay Road Rehabilitation Project, Aug 2012

Project Completion Report (PCR): Strengthening Water Sector M&E Project, MWDI

Progress Report No. 7, Oct – Dec 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, MAIWD

Progress Report Nr. 6, July – Sept 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, MAIWD

Unlocking Human Development Outcomes through WATSAN Interventions in Africa: An Impact Evaluation from Geospacial Data in Zambia and Malawi, AfDB

National Transport Policy, MTPW

Road AuthorityRoads Authority, Government of Malawi, Resettlement Action Plan (Rap) Draft for Repugrading of Thensipe- Liwonde- Mangochi Road, vol. II, July 2013

77Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

African Development Bank Group, Environmental and Social Impact Assessment & Resettlement Action Plan Summary. Nacala Road Corridor Project Phase IV Project (Nsipe-Liwonde-Mangochi Road Section), August 2013

Roads Authority, Annual Report 2011

Roads Authority, Annual Report 2012

The Roads Fund, Summarized Financial Statements for the year ended 30 June 2013

The Roads Fund Administration, Strategic Plan July 2009– June 2014

WATSAN sectorProject Completion Report (PCR): Improving Access to WATSAN for the Urban Poor in the City of Blantyre

Water Supply and Sanitation in Africa: Findings, Lessons and Good Practices to Improve Delivery, AfDB, June 2015

Project Completion Report (PCR): Strengthening Water Sector Monitoring and Evaluation Project, MWDI

Progress Report No. 7, Oct – Dec 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, MAIWD

Progress Report No. 6, July – Sept 2016, Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, MAIWD

Project Completion Report (PCR): Strengthening Water Sector M&E Project, MWDI

Unlocking Human Development Outcomes through WATSAN Interventions in Africa: An Impact Evaluation from Geospacial Data in Zambia and Malawi, AfDB

African Development Bank Group, National Water Development Program. Project Appraisal Report, March 2008

African Water Facility, African Development Bank Group, MALAWI Strengthening Water Sector Monitoring and Evaluation. Project Appraisal Report, August 2009

African Water Facility, African Development Bank, Malawi: Improving Access to Water and Sanitation for the Urban Poor in the City of Blantyre. Appraisal Report, November 2009

African Development Bank, Progress Report as at 30th June 2011 Rural Water Supply And Sanitation Initiative (RWSSI) Water And Sanitation Department (OWAS), September 2011

Rural Water Supply and Sanitation Initiative (RWSSI). 2013 Annual Progress Report RWSSI and RWSSI-TF, OWAS, March 2014

African Development Bank Group, Sustainable Rural Water and Sanitation Infrastructure Project for Improved Health and Livelihoods. Project Appraisal Report, April 2014

African Development Fund, Mzimba Integrated Urban Water and Sanitation. Appraisal Report, OWAS/GECL, October 2015

Implementation Progress and Results Report (IPR), Mzuzu-Nkhata Bat Road Rehabilitation Project, 10/10/2016

Appraisal Report: Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, March 2014

Technical Annexes: Sustainable Rural Water and Sanitation Infrastructure for Improved Health and Livelihoods, April 2014

Appraisal Report: Mzimba Integrated Urban Water and Sanitation, September 2015

Technical Annexes: Mzimba Integrated Urban Water and Sanitation, June 2015

Project Completion Report (PCR): National Water Development Program (NWDP), 15/04/2015

Comments and Responses to PCR, National Water Development Program (NWDP), 01/02/2016

Case Study: Strengthening Water Sector Monitoring and Evaluation Project, March 2015

Quarterly Progress Report: Mzimba Integrated Urban Water and Sanitation, Oct–Dec 2016

The Mchenga Urban Poor Fund, CCODE

Completion Report: RBCSP 2005–2010, June 2010

Agriculture sectorAfrican Development Fund, Smallholder Crop Production and Marketing Project (SCPMP). Appraisal Report, May 2006

African Development Bank Group, Agriculture Infrastructure Support Project (AISP). Appraisal Report, June 2009

African Development Bank Group, Climate Adaptation for Rural Livelihood and Agriculture (CARLA). Appraisal Report, 2010

African Development Bank Group, Shire Valley Irrigation Project (SVIP). Appraisal Report, 2013

African Development Bank Group, Smallholder Irrigation and Value Addition Project (SIVAP). Appraisal Report, 2013

African Development Bank, Feasibility Study on the Establishment of an Agriculture Cooperative Bank; Project Preparation Financing mechanism (PPF), 2015

African Development Bank, Proposal for a Grant of US$1 million for Emergency Humanitarian Assistance to Mitigate the Effects of the 2015 Drought, 2016

African Development Bank Group, Agriculture Infrastructure and Youth in Agribusiness Project (AIYAP). Appraisal Report, 2016

AfDB Sector Studies


African Development Bank Group, Agriculture Sector Strategy 2010–2014, 2010

African Development Bank Group, Feed Africa: Strategy for Agricultural Transformation in Africa 2016–2025, 2016

Mid-Term Review

African Development Bank Group, Smallholder Crop Production and Marketing Project (SCPMP). Mid-Term Review Report, 2010

African Development Bank Group, Agriculture Infrastructure Support Project (AISP). Mid-Term Review Report, 2013

African Development Bank Group, Smallholder Crop Production and Marketing Project (SCPMP). Project Completion Report, September 2013

African Development Bank Group, Climate Adaptation for Rural Livelihood and Agriculture (CARLA). Project Completion Report, December 2016

Multisector Appraisal Report: Poverty Reduction Support Loan, February 2007

Project Completion Report: Poverty Reduction Support Loan, January 2009

Appraisal Report: Poverty Reduction Support Grant I (PRSG I), January 2009

Aide Memoire: Appraisal Mission for the Poverty Reduction Support Grant (PRSG I), December 2008

Project Completion Report: Poverty Reduction Support Grant I (PRSG I), undated

Project Concept Note: Governance and Poverty Reduction Support Grant II (GPRSG II), October 2009

Aide Memoire: Governance Poverty Reduction Support Grant II: Appraisal Mission to Malawi, November 2-13, 2009

Appraisal Report: Governance and Poverty Reduction Support Grant II (GPRSG II), March 2010

Aide Memoire: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, August 2010

Back-to-Office Report: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, August 2010

Back-to-Office Report: Governance Poverty Reduction Support Grant II: Supervision Mission to Malawi, March 2011

Aide Memoire: Governance Poverty Reduction Support Grant II: Completion Report, February–March 2011

Project Completion Report: Governance and Poverty Reduction Support Grant II (GPRSG II), December 2013

Appraisal Report: Restoration of Fiscal Stability and Social Protection, June 2012

Appraisal Report: Supplementary Budget Support to RFSSP, April 2013

Back-to-Office Report, RFSSP Supervision, Project Completion and Budget Support Review Mission, Nov 2013

Implementation Progress and Results Report (IPR), Restoration of Fiscal Stability and Social Protection, December 2013

Aide Memoire: PCR Mission for Restoration of Fiscal Stability and Social Protection and Budget support Review, December 2013

Project Completion Report: Restoration of Fiscal Stability and Social Protection and Supplementary Budget Support to RFSSP, December 2013

Appraisal Report: Public Finance Management Institutional Support Project – Phase I, September 2013

Appraisal Report: Public Finance Management Institutional Support Project-Phase 2, June 2015

Aide Memoire: Public Finance Management Institutional Support Project, Supervision and Preparation Mission, November 2014

Implementation Progress and Results Report: Public Finance Management Institutional Support Project, Supervision, August 2016

Implementation Progress and Results Report: Public Finance Management Institutional Support Project, Phase 2, Launching, August 2016

Implementation Progress and Results Report: Public Finance Management Institutional Support Project, Phase 2, Launching, December 2015

Quarterly Project Progress Reports: Public Finance Management Institutional Support Project, 2014-present

MIPS, Strategic Plan 2016–20.

Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume I: Summary of Main Findings and Recommendations

Malawi Country Economic Memorandum: Seizing Opportunities for Growth through Trade; Volume II: Background Papers

Policy Brief: Analysis of Malawi’s Exchange Rate and Case Studies on Exchange Rate Adjustment, July 2011, Confidential

Case study report: Domestic Resource Mobilization for financing development, May 2014

Research Papers: Exchange Rate Misalignment in Malawi: A Monetary Approach (undated). Measuring Potential Output for Malawi (undated). A Small Macro-Model for Malawi (undated), Reserve Bank of Malawi

Study Report: Skills for Private Sector Development, Malawi, December 2009

Inter-office Memorandum: Bank’s legal opinion on the proposed amendments to the Joint Framework Agreement CABS, December 2007

Procurement and fiduciary risk assessment reports

79Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Other Donor DocumentsADB, ADF, DANIDA, SIDA, Joint Evaluation. Evaluation of Public Financial Management Reform in Malawi 2001–2010. Final Country Case Study Report, June 2012

AFD, AUSAid, CIDA et al, Joint Evaluation of General Budget Support 1994–2004, Malawi Country Report, May 2006

AICD, Country Report. Malawi’s Infrastructure: A Continental Perspective, March 2010

CIDA, Malawi Country Program Evaluation 1998-2008. Synthesis Report, July 2010

CMI Brief, the ‘Joyce Banda Effect’. Public Opinion and Voting Behavior in Malawi, Volume 15 No. 6, April 2016

CMI Brief, The Gender Machinery Women in Malawi’s Central Government Administration, Volume 15 No. 11, June 2016

DFID, Evaluation of DfID Country Program. Country Study: Malawi 2000 - 2005, Evaluation Report Ev661, April 2006

EU, Country Level Evaluation Republic of Malawi. Final Report. Volume 1: Main Report, November 2011

EU, Country Level Evaluation Republic of Malawi. Final Report. Volume 2: Annexes, November 2011

EU DEVCO “Methodological Approach to Blending” Final Report, December 2016, ADE publication for DEVCO

FAO (Food and Agriculture Organization), Socio-economic content and the Role of Agriculture; Country Fact Sheet, March 2015

Global Partnership for Effective Development Co-operation (GPEDC), Monitoring Guide 2015–16, 2015

IFC and World Bank Group, Impact of Efficient Irrigation Technologies on Small Farmers, 2014

JICA, Inception Report: Data Collection on Nacala Corridor Integrated Development in Southern Africa, JICA, March 2017

UN Women, Planet 50:50 by 2030: Step It Up for Gender Equality

World Bank Paper: “Emerging Stronger- Malawi Economic Monitor”, October 2016

World Bank Paper: “Absorbing Shocks, building resilience- Malawi Economic Monitor,” May 2016

World Bank Working Paper 182: “The Education System in Malawi,” 2010

World Bank, Malawi Poverty and Vulnerability Assessment, 2007

World Bank, Evaluation Brief: “Overlooked Links in the Results Chain” IEG-World Bank Brief 142011

www.nepad-caadp.org: Introducing CAADP. Website accessed 29th May 2017

Other DocumentsAide Memoire: Common Approach to Budget Support (CABS) Reviews, October 2010, March 2012, October 2012, March 2013, November 2013, March

2014, August 2014

Minutes: Group on Financial and Economic Management (GFEM) Meeting (incl. Technical Committees), May 2011, December 2011, September 2012, April 2013, June 2013, October 2013

Minutes: PFEM Donor Coordination Group, November 2016, December 2016, January 2017, March 2017

Minutes: PFM Joint Technical Committee Meeting, January 2016, September 2016, February 2017

Letter: GFEM Comments on PFM Reform Program, February 2015, EU.

Draft Final Report, Support for the Development of the Public Finance and Economic Management Reform Program for Malawi, September 2011, ACE / European Commission

Brief: Public Financial Management Malawi Donor Coordination Group, Briefing for Heads of Cooperation Meeting February 2017

PowerPoint presentation: PFM priorities from DP’s perspective, January 2017

Program Document: Public Finance and Economic Management Reform Program (PFEMRP), February 2012, World Bank

Final Report, Review of CABS Mechanism and Its Processes, November 2013 Sarah Holloway, DFID / Mokoro.

Draft Mission Report, Independent Monitoring of Progress against the GoM Action Plan to improve PFM, August 2014

Technical report: Malawi Restoring Financial Control and Accountability November 2014, Abdul Khan, Suzanne Flynn, Guy Anderson, and Amitabh Tripathi, IMF Fiscal Affairs Department

Donor Matrix, Development Partner support to assist the Malawi’s PFM, May 2016

Project Appraisal Document: Financial Reporting and Oversight Improvement Project, February, 2013, World Bank

Performance and Learning Review of the Country Assistance Strategy for the Republic Of Malawi for the Period Fy13-Fy16, World Bank

Implementation Completion and Results Report: Financial Management, Transparency, and Accountability Project (FIMTAP) April 30, 2010, World Bank

AfDB/ DANIDA/SIDA, Evaluation of Public Financial Management Reform in Malawi 2001–2010 Final Country Case Study Report, June 2012

A joint Evaluation of General Budget Support 1994-2004, Evaluation of GBS – Malawi Country Report, May 2006

Initial thoughts on the Procurement and Disposal of Assets Bill, March 2017

IMF Reports.

Malawi Confederation of Chambers of Commerce and Industry, “Malawi Business Climate Survey Report” November 2016

Hausmann, R. et al., “The Atlas of Economic Complexity-Mapping Paths to Prosperity” Harvard Kennedy School of Government Press 2016.


Endnotes1. The same issues are included in the Bank’s results framework.

2. No targets have been set for reduced or avoided emissions or reduction in carbon footprint (at project or program levels) exept in general terms (e.g. by reference to MDG 7: CO

2 emissions)

3. It should be noted that some social mitigation measures (e.g. compensation for land, houses, crops) might be frustrated due to late disbursement of funds for Government (payment of such costs is responsibility of Government and a component of GoM commitment to the project).

4. In the area of PFM,the Bank and WB went as far as conducting a joint appraisal mission in 2013 to define their respective contribution to strengthening PFM institutions. This example, however, remains isolated..

5. The Evaluation considers only documented evidence as being "identified"

6. Interpreted as being documented to show that there has been an influence.

7. Public Administration Reform

8. Monitoring and Evaluation

9. No targets have been set for reduced or avoided emissions or reduction in carbon footprint (at project of program levels) except in general terms (e.g. by reference to MDG 7: CO

2 emissions).

10. African Development Bank, https://www.afdb.org/en/countries/southern-africa/malawi/, accessed January 18th 2017

11. Ibid.

12. CIA World Factbook https://www.cia.gov/library/publications/the-world-factbook/geos/mi.html, accessed January 20th 2017

13. Source: OECD.org/dac/financing-sustainable-development/development-finance-data/aid-at-a-glance.htm

14. It may be noted that the priorities in MGDS II that are the same as those in MGDS I are italicized.

15. Interim Country Strategy Paper

16. It may be noted that the CARLA project learned a lot from the SCPMP project, and therefore had better internal sustainability support systems. For the CARLA project, government had incorporated activities on CARLA in its national budget, with possibility of inclusion into subsequent budgets.

17. Most schemes had to be redesigned due to changes in technologies and sites due to land disputes and an underestimation of the costs of certain items at appraisal stage.

18. The same issues are included in the Bank results framework.

19. No targets have been set for reduced or avoided emissions or reduction in carbon footprint (at project or program levels) except in general terms (e.g. by reference to MDG 7: CO

2 emissions).

20. It should be noted that some social mitigation measures (e.g. compensation for land, houses, crops) might be frustrated due to late disbursement of funds by Government (payment of such costs is responsibility of Government and a component of GoM commitment to the project).

21. There are examples (in Malawi and elsewhere) of reducing motivation (and resources) and technical back up over time resulting in reduced contributions to maintenance and spares.

22. Results Based Country Strategy Paper

23. Malawi’s PBA allocation was further reduced from UA 103 million under ADF XII to UA 68.39 million under ADF XIII.

24. Discussions with DCAFS members during Field Mission in April 2017

25. Discussions with Ministry of Agriculture staff during Field Mission in April 2017

26. GFEM

27. Discussions with the staff of the Ministry of Agriculture during Field Mission in April 2017

28. In the area of PFM, the AfDB and WB went as far as conducting a joint appraisal mission in 2013 to define their

29. United Kingdom Department for International Development

30. One notable exception concerned the LED project, where various donors did not coordinate, and no sense of complementarity was achieved.

31. Some donors like DfID have also stopped using country systems altogether. This has contributed to weakening the SWAps previously developed in key sectors.

32. The Bank first supported the proposed amendments since the late 2000s, when it first provided TA, with direct involvement of the Bank procurement officer – as part of Common Market for Eastern and Southern Africa (COMESA) Enhancing Procurement Reforms and Capacity Project grant.

33. Discussions with Ministry of Agriculture staff during field visit in April 2017.

34. SCPMP and CARLA Supervision Missions and Back-to-Office Reports.

35. This includes taxes on international trade as a percentage of GDP and number of contracts awarded on basis of open competition as a percentage of all contracts awarded in a given year.

36. The Evaluation considers only documented evidence as being “identified.”

37. Interpreted as being documented to show that there has been an influence.

81Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

38. ‘Knowledge management’ suggests use and application of such lessons learned and findings, not only a record of this information. Sources define a KM system as any kind of IT (or hard copy library) system that stores and permits location, retrieval and application of knowledge.

39. Exceptions are: Education V – 15 months after closure date; NCRP (Nacala Corridor Road Development Program) I – 19 months; NWDP – 16 months

40. Recently there has been improvement in this respect, by IDEV, via the creation and population of the Evaluation Results Database (EVRD)

41. E.g. Ministry of Education Science and Technology, Ministry of Health, and various agencies and ministries in agriculture and irrigation.

42. This includes Edu V (Support to Secondary Education V), PBS, LED in the social sector alone; SCPMP in agriculture.

43. To facilitate reading, it is understood that each JC that requires a sector analysis will contain specific indicators to that effect.

44. Refer to p. 12-17 of Results Measurement Framework-Indicators, 2013

45. It is understood that the Team interpreted “target group” and “beneficiaries” in a way that does not include the infrastructures, officials, systems and processes that generate service delivery. Teachers, for example, are not targets; students are. Farmers are targets, but the Agriculture ministry is not. The only exception to this interpretation is where the intervention results framework specifically stated otherwise.

46. The difference between beneficiaries and targets in existing documents was not always clear. For the purpose of this evaluation, “Beneficiaries” are any individuals, groups or organisations that will reap some form of benefit from Bank activities in Malawi or in the region. “Targets” are those individuals, groups or organisations whose problems the intervention were specifically designed to resolve. Targets are subsets of beneficiaries and are specifically identified from the outset in the reports generated by the evaluation.

47. Inclusiveness” is defined as in the Bank’s “One Bank Results Measurement Framework 2013-2016”

48. Part of UN Human Development Index

49. Of the OECD

50. Global Findex Database for example

51. Millennium Dev Goal Indicator for goal 1, target c (FAO)

52. FAO Aquastat

53. Global Partnership Monitoring Framework Indicator 6

54. The team focused on result levels that are beyond the mere production of outputs, as the objective of the Bank is to contribute to outcomes, not outputs.

55. The team considered only documented evidence as being “identified”

56. This was interpreted as being documented to show that there has been an influence.

57. Malawi had serious balance of payments problems so the Bank’s FOREX helped. The funds are for direct support of operations of sectors.

Des

ign

& la

yout

: CR

ÉON

• w

ww

.cre

onde

sign

.net

About this Evaluation

This report summarizes the findings, lessons and recommendations of the independent evaluation of the African Development Bank’s Country Strategy and Program in the Republic of Malawi during 2005–2016. The evaluation is mainly intended to inform the next Country Strategy Paper and to contribute to both accountability and learning in the Bank as well as in countries where the Bank’s program is similar to that of Malawi. The evaluation exercise focused on both strategic and individual interventions in all sectors (Social, Transport, Water and Sanitation, Agriculture, Multisector – Financial Governance, and Private Sector Development) where the Bank was active during the period under review. In addition, the evaluation looked at the achievement of development results by answering evaluation questions based on four of the five evaluation criteria of the Development Assistance Committee of the Organization for Economic Cooperation and Development (OECD–DAC), those of relevance, effectiveness, efficiency, sustainability and impact, with impact not a required criterion for this evaluation. The methodology focuses on results achieved and demonstrated, a Theory of Change approach to intervention design and management, and a sector-by-sector analysis followed by a roll-up of sector level findings to the program level, evidence-based research, contribution analysis and iterative feedback with key stakeholders.

An IDEV Country Strategy Evaluation

African Development Bank GroupAvenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d’IvoirePhone: +225 20 26 20 41E-mail: [email protected]

idev.afdb.org