Embed Size (px)
DESCRIPTION
At the 16th Development Assistance Committee (DAC) Evaluation Network, 12-13 February 2014, the African Development Bank presented a short update on the latest progress of the Evaluation Cooperation Group (ECG). The Evaluation Cooperation Group (ECG) was established by the heads of evaluation in multilateral development banks (MDBs) in 1996 to: strengthen the use of evaluation for greater MDB effectiveness and accountability share lessons from MDB evaluations and contribute to their dissemination harmonize performance indicators and evaluation methodologies and approaches enhance evaluation professionalism within the MDBs and collaboration with the heads of evaluation units of bilateral and multilateral development organizations facilitate the involvement of borrowing member countries in evaluation and build their evaluation capacity More on their website: https://wpqr4.adb.org/LotusQuickr/ecg/Main.nsf/h_Toc/c4d2972a55a9514948257bbe0023aa84/?OpenDocument
Citation preview
ECG Update on Progress and
Latest Development
Dac Evaluation Network 12-13 February 2014
Outline
Latest Development and Progress
Use of GPS by ECG Members
Harmonizing Public and Private Project Level
Evaluation
Piloting the Combined Approach to Evaluation
in IADB and other IFIs
2
www.ecgnet.org 2
ECG -Latest Development and
Progress
Tracking and Following Up Evaluation
Recommendatios
Evaluation of Knowledge Services and
innovative Approaches
Harmonizing Public and Private Project Level
Evaluation
Special Event on Impact Evaluation as Learning
Tool for Development Effectiveness3
www.ecgnet.org 3
A Pilot Benchmarking Exercise was conducted during September
2011 and has served to:
(i) Review and fine-tune the EP and OP (review definitions,
underlying concepts and applicability of standards in different
contexts) including rewording of several EP and OPs;
(ii) Use successfully evaluation principles (EPs) and Operational
Practices (OPs) in the GPS for benchmarking, with only minor
issues on terminology, scoring, and weights;
(iii) Visually presenting each institution’s standing within each
area of the GPS, such as through a spider diagram or a bar
chart.
The New GPS (4th Edition) defined Generic Principles for:
• Independence of evaluation departments,
• Reporting and corporate learning,
• Evaluation Guidance and Rating Systems
With Private Sector Principles for:
• Planning and Executing Project Evaluation
• Evaluation Metrics and Benchmarks
And defined straightforward methodology for next
benchmarking (Three Benchmarking exercises completed)
Methodology for benchmarking:
ExaminetheIFI’sOpera onalPrac ces
StandardPrescribedOPs:FullyImplementedPartlyImplementedNotImplemented
Alterna veNon-PrescribedOPs:*
Implemented
DetermineCompliancewithEvalua onPrincipleElements
ScoreeachEPElement:MateriallyCompliant(1pt)PartlyCompliant(0.5pt)NotCompliant(0pt)
DetermineAreasofStrengthorWeakness
EPswherecompliantwithvirtuallyallElements
EPswherecompliant
withfewornoElements
Report Report
NameofIFI Compliance(%)=ΣScoresforEPElements
ΣEPElements**
ListofStrongEPsListofWeakEPs
Scorecard
*Alterna ve,non-prescribedprac cesshouldbeconsideredforincorpora onintofuturerevisionsoftheGPS.**Asrelevanttoeachins tu on.
The Boards of Directors in some IFIs have asked for greater consistency in
public and private sector evaluation approaches
To ensure that differences in ratings for public and private sector
operations are due to actual differences in performance rather than
differences in how these operations are evaluated
To be able to use the same evaluation criteria and ratings systems in
higher-level evaluations
Piloting the Combined Approach to
Project level Evaluation
• Engaging with Policy, Quality Assurance, M&E Staff and Lending Officers in explaining the approach
• Selection of private and public sector interventions in various real sectors and financial intermediation operations
• Provide guidance on evidentiary requirements, analytical methods, and benchmarks for a positive rating
• Clarify how to assess and rate the development outcomes and results attributable to the project
• Discuss the constraints and challenges in apprehending concepts, definitions, and use of rating methodology
• Fine-tune the combined evaluation framework and prepare for adoption of Combined GPS
A Common Framework: Option 1
Relevance
Effectiveness
Efficiency
Impact
Sustainability
Relevance
OECD-DAC CriteriaPublic GPS Criteria Private GPS Criteria
IFI Performance
Borrower Performance
Fulfillment of Project Business Objectives
Effectiveness
Efficiency
Achievement of Corporate Goals (opt.)
Unintended Outcomes (opt.)
Sustainability
Contribution to IFI Mandate Objectives
Financial Performance
Economic Sustainability
Environmental and Social Impact (opt. suppl. criterion)
(the forward-looking part of) Fulfillment of Project Business Objectives and
Financial Performance
Environmental and Social Performance
IFI Additionality
IFI Investment Performance
IFI Work Quality/Bank Handling
Overall Project PerformanceRating
Public Sector Sub-Criteria Harmonized Criteria Private Sector Sub-Criteria
Relevance
Results
Achievement of Outputs
And intended and non-intended
Outcomes
Operational Performance
Achievement of Outputs
Fulfillment of Project Business
Objectives
Contribution to Intended Outcomes
Contribution to Corporate Goals
Unintended Outcomes
Economic Efficiency
Implementation EfficiencyEfficient Use of Resources
Financial Performance of
Project/Company
Economic Efficiency
Implementation Efficiency
Outcome Sustainability
Compliance with SafeguardsSustainability
Outcome Sustainability
Commercial Sustainability
Compliance with Safeguards
Overall Project Performance Rating
Quality at Entry
Quality of SupervisionIFI Performance
Quality at Entry - Additionality
Quality of Supervision
Government and Implementing Agency
PerformanceClient Performance
Non-Financial Performance of the
Company
IFI Investment Profitability
A Common Framework: Option 2
