Monitoring and evaluation

frameworks and the performance

and governance of international

funds

August 2011

This report is commissioned by the AfDB and prepared by Vivideconomics

2 Monitoring and evaluation frameworks and the performance and governance of international funds

Contents

1 Introduction .......................................................................................... 4

2 Key findings and themes ...................................................................... 6

3 Adaptation Fund ................................................................................. 13

4 African Development Bank ............................................................... 19

5 Asian Development Bank ................................................................... 26

6 Climate Investment Funds ................................................................. 32

7 GAVI.................................................................................................... 37

8 Global Environment Facility ............................................................. 42

9 Global Fund ........................................................................................ 48

10 Inter-American Development Bank .................................................. 53

11 International Finance Corporation ................................................... 59

12 International Fund for Agricultural Development .......................... 65

13 MDTFs ................................................................................................. 70

14 World Bank Group ............................................................................ 75

15 References ........................................................................................... 82

3 Monitoring and evaluation frameworks and the performance and governance of international funds

List of tables

Table 1. Summary of findings ............................................................................ 9

Table 2. The seven outcomes targeted by the Adaptation Funds Strategic

Results Framework ............................................................................. 15

Table 3. The AfDBs OPEV produces five categories of evaluations .............. 19

Table 4. There are ten different types of reports regularly produced by the

ADBs IED ......................................................................................... 26

Table 5. Strengths and weaknesses of the ADBs Accountability Mechanism 31

Table 6. The IDBs OVE is planning to conduct 4 different types of evaluation

in 2011-2012 ....................................................................................... 54

Table 7. The IFC uses four different tools to evaluate performance at the IFC

and the World Bank Group ................................................................. 60

Table 8. The IFC uses four performance areas to assess development outcomes61

Table 9. The IFADs IOE conducts a variety of different types of evaluations 66

Table 10. The WBGs IEG uses four different types of evaluations .................. 76

Table 11. Monitorable actions from the IDAs results framework used to track

agency effectiveness ........................................................................... 77

4 Monitoring and evaluation frameworks and the performance and governance of international funds

1 Introduction

This paper presents desk-based research examining the key institutional and design features of a range of

multilateral funds and institutions. Although the key focus of the paper is on the monitoring and evaluation

frameworks used in each of these funds/institutions, it also covers engagement of civil society organisations,

perceptions of developing country ownership, environmental and social safeguards, financial management

and conflict management.

The funds/institutions considered are the following:

the Adaptation Fund (AF)

the African Development Bank (AfDB)

the Asian Development Bank (ADB)

the Climate Investment Funds (CIFs)

the Global Alliance for Vaccines and Immunisation (GAVI)

the Global Fund to Fight Aids, TB and Malaria (Global Fund)

the Global Environment Facility (GEF)

the Inter-American Development Bank (IADB)

the International Finance Corporation (IFC)

the International Fund for Agricultural Development (IFAD)

Multi-Donor Trust Funds (MDTFs)

the World Bank (WB)1

The specific questions that have been addressed are:

What is the framework for undertaking monitoring and evaluation2? Where evidence exists, how effective

is this framework? To what extent are CSOs involved in monitoring and evaluation?

Does the fund/institution make use of results-based management techniques?

What are the structures for engaging civil society? How effective are these?

How does the fund promote developing country ownership? Are these structures effective?

What environmental and social safeguards does the fund have in place? How effective are these?

What are the processes in place to promote good financial management and ensure fiduciary

responsibilities are realised?

What policies does the fund/institution have in place to resolve conflicts? Are these effective?

1 We considered the World Bank and the IFC separately. The World Bank discussion focuses on the generic procedures used by the

World Bank Group and also draws on particular experiences from the International Development Association (IDA), International Bank of Reconstruction and Development (IBRD) and the Multilateral Investment Guarantee Agency (MIGA). There is a degree of overlap between the World Bank and IFC assessment.

2 Following GEF (2010), monitoring can be defined as: a continuous or periodic function that uses systematic collection of data,

qualitative and quantitative, for the purposes of keeping activities on track while evaluation can be defined as systematic and impartial assessment of an activity, project, program, strategy, policy, sector, focal area, or other topic. Typically monitoring takes place while a project/program is in operation while evaluation takes place when it is completed.

5 Monitoring and evaluation frameworks and the performance and governance of international funds

The paper is based primarily on an examination of the secondary literature on these topics. A further

phase of work could seek to corroborate these findings through interviews/discussions with key

representatives from each of these funds/institutions.

Section 2 provides a summary table of the key findings in relation to each fund/institution and outlines some

of the key themes that emerge from a comparative assessment of these topics. Sections 3-14 provide more

detail on each of the individual institutions/funds.

6 Monitoring and evaluation frameworks and the performance and governance of international funds

2 Key findings and themes

Table 1 on the following pages summarises the findings across each of the funds/institutions investigated.

Drawing generalisations across such a wide range of institutions and issues is challenging; especially as each

organisation (and fund) has specific mandates and objectives which might reasonably be expected to

necessitate different institutional structures. Nonetheless, some key findings and themes emerge.

Independent evaluation offices are recognised as international best practice. However, such offices are

not yet universally in place, in particular in organisations which rely on other implementing agencies.

The benefits provided from establishing an independent evaluation office are increasingly recognised. For

instance, the Asian Development Banks monitoring and evaluation is perceived to have improved

significantly since its evaluation office was made independent. There is also a broad consensus on how to

operationalise the principle of independence. M&E offices should have no part in line management structure

of the wider institution, report directly to the Board of Directors (or equivalent) and have unrestricted access

to an institutions staff and records. However, not all of the organisations studied have independent

evaluation offices, especially those organisations consciously seeking to have a lean organisational

structure and some of those that rely on others for implementing the day-to-day delivery and execution of

any funds disbursed i.e. GAVI, the Adaptation Fund. These organisations typically rely on external

consultants and/or partners/implementing agencies to assist with M&E.

Although there are advantages in having organisations which rely on separate implementing agencies,

this can complicate M&E and, unless managed carefully, can make M&E less effective. The use of

Implementing Agencies has many advantages not least the cost savings from avoiding institutional

duplication. At the same time, it can make M&E more challenging. In the case of GAVI, concerns have been

raised that it is not clear whether the GAVI Secretariat or the Implementing Agencies are responsible for

M&E. GEF, which also uses a similar organisational structure, appears to have overcome these challenges

but it recognises that its M&E arrangements create a co