ADB's Support to Gender and Development - IED Evaluations of 2009 & 2010

8/6/2019 ADB's Support to Gender and Development - IED Evaluations of 2009 & 2010

1/24

1

ADBs Support to Gender andDevelopmentIED Evaluations of 2009 & 2010

Briefing for 11thSession of ExternalForum on Gender and Development

Monday, June 27 2011

Rajesh Vasudevan

Monday, 27 June 2011 IED

The views expressed in this paper are the views of the author and do not necessarily reflect the views or policies of the AsianDevelopment Bank (ADB), or its Board of Governors, or the governments they represent. ADB does not guarantee theaccuracy of the data included in this paper and accepts no responsibility for any consequence of their use. The countrieslisted in this paper do not imply any view on ADB's part as to sovereignty or independent status or necessarily conform toADB's terminology.


2/24

2

IED Evaluation of ADBsSupport to GAD in Two Phases

Phase I: Meta-Evaluation

- Evaluation completed in December 2009

Phase II: Main Report and 6 Country Studies

- Evaluation completed in December 2010



3/24

3

Phase I Meta-Evaluation



4/24

4

Phase I Evaluation Criteria

(i) Relevance: Relevance at time of approval and in

context of strategic priorities

(ii) Responsiveness: Covered institutional, operational,

external responsiveness

(iii) Results: Review of available published informationon intended outputs and outcomes of loan and TAactivities



5/24

5

Phase I Methodology

(i) Desk review of about 500 related documents

including 31 project completion reports, 17 higherevaluations, and 20 RGAs.

(ii) Interviews with gender focal points in ADBheadquarters.

Note: 212 Category I and Category II projects between1998 and October 2009.



6/24

6

Phase I Findings

(i) Determinants indentified that influenced gender

achievements in departments: Realignments; unevenlevels of influence of country gender assessments;lending sources and modalities; technical assistance

distribution; and availability of gender specialists.

(ii) Shift in ADBs portfolio from 2004, to sectors notamenable to gender mainstreaming.

(iii) Gender action plans, a pragmatic and innovativeapproach.



7/24

7

Phase I Recommendations(i) Improve outcome orientation of GAD goals andtheir monitoring and evaluation;

(ii) Provide clarity to operationalize GAD goals in ADBoperations in the context of Strategy 2020;

(iii) Provide adequate financial and human resourcesto support policy implementation.

Monday, 27 June 2011 7IED


8/24

8

Phase II Country Studies



9/24

9

Phase II Evaluation Criteria

(i) Study brings a country and project perspective to

bear on the implementation and outcomes of GAD policybetween 1998-2009.

(ii) Study has assessed and rated gender-relatedcomponents of 55 projects using the 4 standard IEDcriteria - relevance, effectiveness in achieving outputsand outcomes, efficiency in implementation, and

sustainability.(iii) PNG, Lao PDR, INDO, NEP, BAN, PAK



10/24

10

Phase II Methodology

(i) Desk-review of available country and project-specific

documentation;

(ii) Interviews with headquarters-based GAD specialists,

and resident mission GAD staff;

(iii) Purposive selection of 55 projects for in-depthassessment;

(iv) In-country consultations; and(v) In-depth assessment of 55 sample projects



11/24

11

Phase II Findings

Sample: 25 (Cat I), 20 (Cat II), 9 (Cat III), 1 (Cat IV)(i) Overall: About 51% the projects were ratedsuccessful or higher in gender-related performance.

(ii) Relevance: 82% of the sample projects were rated

GAD relevant or higher.

(iii) Effectiveness: 44% effective or higher in achievingthe intended GAD outcomes.

(iv) Efficiency: 31% efficient or higher; and(v) Sustainability: 45% likely to be sustainable in termsof continuity of GAD outcomes.



12/24

12

Phase II Findings

Missed Opportunities: Of the 10 category III and IVprojects, 30% were rated successful or higher and theonly highly successful project in the sample of 55 projectswas in category III.

Core and Non-Core Sectors: 20 (Core Sectors), 25(Non-Core Sectors)

(i) 50% of the 20 projects in Core sectors rated

successful or higher.

(ii) 60% of the projects in Non-Core sectors ratedsuccessful or higher.



13/24

13

Phase II Findings

(i) Evidence weak on the influences of country gender

assessments to country strategies;

(ii) Capacity to address gender issues, including in

Category I and II projects, is often limited;

(iii) ADB's business processes for project implementationand related information systems, are weak in capturinggender performance data;

(iv) Review of gender categorization of projects duringimplementation is important.



14/24

14

(i) Make more effective use of country and sector levelgender assessments in the preparation of countrypartnership strategies, sector road maps, resultsframeworks, operations business plans, and project

preparation;(ii) Strengthen tracking of gender results in ADB projectperformance monitoring systems; and

(iii) Increase collaboration with development partners,including non-government and community-based

organizations in DMCs, to jointly deepen gender focus incountry policy dialogue, technical assistance activities,and in strengthening capacity of executing agencies.

Phase II - Recommendations



15/24

15

GAD Assessment Criteria



16/24

16

Relevance (20%)

Consistency of a projects impact and outcome with

Governments development strategy

ADBs lending strategy for the country

ADBs strategic objectives at the time of approval andevaluation

Adequacy of the design



17/24

17

Relevance criteria and GADquestions used in Phase II study Do the CGA , CPS (and its gender strategy) identify

the sector/project as priority for gender-relatedinterventions?

How important is gender to achievement of overall

project objectives, as articulated in projectdocuments?

Is there evidence of gender-inclusive consultationsand participation (C&P) on project design andintended outcomes?

How well are the findings of the project genderanalysis (or gender focus of social analysis) reflectedin design of the project?



18/24

18

Effectiveness (30%)

The extent to which the outcomes, as specified in the designand monitoring framework, either as agreed at approvalor as subsequently modified, have been achieved.



19/24

19

Are gender-related outcomes and/or outputsincluded in the DMF, and are there associated targetsand indicators?

Are there missed opportunities for mainstreaming

gender in project design? Are gender-related design features (activities) well

conceived, and appropriately reflected in the GenderAction Plan (GAP)?

Were gender-related outputs delivered as plannedand on time, or, for active projects, are theyconsidered on track?


Effectiveness criteria and GADquestions used in Phase II study


20/24

20

Efficiency (30%)

Describes (ex-post) how economically resources have beenconverted into results, using

the EIRR (relative to 12% standard)

cost-effectiveness of the investment (relative to

industry practice) other indicators (implementation issues)

Describes the resilience to risk of net benefit flows over time



21/24

21

Efficiency criteria and GADquestions used in Phase II study

Were sufficient human and financial resourcesallocated to gender-related activities in the project?

Are gender activities or indicators monitored duringsupervision?

Has the project complied with all gender-relatedcovenants and agreements, or is it on track to do so?

Does the Gender Action Plan (GAP) provide asuitable foundation for monitoring implementation of

gender-related activities?



22/24

22

Considers the likelihood that human, institutional,financial, and other resources are sufficient tomaintain the outcome over the economic life of the

project

Sustainability (20%)



23/24

23

Sustainability criteria and GADquestions used in Phase II study

Does the project build longer-term countryinstitutional capacity to address gender issues in theproject or sector?

Is there continued demand for project-provided

gender-related products and services? Have project-supported policy, legal, and institutional

reforms been implemented or are policy andinstitutional arrangements in place to sustain gender-related activities?

Are there gender-related risks to achievement ofproject outcomes? (Note: scale is inverted--high risk =0; low risk = 3)?



24/24

24

Assessment Rating - Categoriesand Ranges for four criteria

Highly successful (HS>= 2.7)

Successful (1.6=

Documents

ADB's Support to Gender and Development - IED Evaluations of 2009 & 2010