Building Evaluation Capacity in Africa: Strategies and Challenges By Dr. Frannie A. L©autier Executive Secretary The African Capacity Building Foundation

  • View

  • Download

Embed Size (px)

Text of Building Evaluation Capacity in Africa: Strategies and Challenges By Dr. Frannie A. L©autier...

  • Slide 1

Building Evaluation Capacity in Africa: Strategies and Challenges By Dr. Frannie A. Lautier Executive Secretary The African Capacity Building Foundation (ACBF) AFDB Evaluation Week on the theme: Evaluation for Development December 3 6, 2012, Tunis, Tunisia Slide 2 How should evaluation be structured catching the short dash but sustaining the long march Slide 3 Order of Presentation General overview of Evaluation Capacity issues in Africa Introduction (The challenge) Status of Evaluation in Africa Why Evaluate? Levels and Dimensions of Evaluation Capacity Opportunities Strategies Challenges The ACBF Case Conclusion and Way forward Slide 4 General issues on Evaluation Capacity in African 4 Slide 5 Elevation Slope, aspect, drainage Settlements, ports, markets Road, rail, river, ICT networks Market travel times & costs Introduction: The Challenge Port travel times & costs Terrain, Demography, Infrastructure, Admin Units Production Environment & Constraints Production Systems & Performance Interventions/ Responses Agroecological Zones Cropland extent & intensity Pests & Diseases (Maize Stem Borer) Drought Incidence & Severity Runoff Administrative Units Farming Systems Crop Suitability: Rainfed Wheat Crop Distribution & Yields Value of Production per Rural Person Yield Responses to Inputs, Management, CC Profitability of small scale irrigation Quantity of Nutrients Removed Fertilizer Profitability Distribution of Welfare Benefits Linkage to Macro Models Aggregate to FPUs Source: HarvestChoice/IFPRI 2010 Slide 6 Strategic Choices and Evaluation There are generally two approaches to strategy makingthe root method and the branch method--evaluation practice needs to be aligned to the strategy approach in order to capture impacts The root method relies on the ability to define objectives very well, outline a range of options in a comprehensive manner, evaluate the options and select from the one that maximizes the attainment of the objective. The branch method involves building out step by step and in small degrees from the current situation. It is the state of practice used by leaders in political and complex environments. It is many times referred to as muddling through. Lindblom (1959) presents us with the virtues of muddling through in his example of dealing with inflation and Kay (2009) illustrates how companies have used these strategies with very different effect, offering learning strategies for individuals to be better at managing portfolio risks by knowing why some companies succeed and some fail. Slide 7 Evaluation and the Science of Muddling Through There would also be a premium on piloting and learning from trials that can be monitored and scaled up Such an approach would suit very well post conflict and fragile states, investments with long gestation periods like infrastructure and education, as well as investments in building capacity Growing out a results chain gradually and systemically over time would be valued as it would show a resilient and robust approach to results If we were to use the science of muddling through then what would matter would be trends in results achievements and the sustainability of results Slide 8 The Challenge Contd Understanding the root causes, impediments, and enablers to Africa's development require good data analysis Going beyond physical achievement to also gauge long term achievement and implications for the future requires good quantitative and qualitative information Need to deal with Africas problems of yesterday, today and tomorrow simultaneously required evidence- based information 8 Slide 9 Status of Evaluation in Africa Evaluation capacity is varied across the continent- the statement of lack of evaluation capacity can not be applied across board There are increasing expectations for countries to develop national evaluation systems Planning and monitoring have received more resources and have been strengthened more that evaluation Technical soundness is a cornerstone for crediblility but does nt gurantee use Evaluation of public policies and programs must beembedded in the political process. Both technical and political dimensions of evaluation needs to be considered. Slide 10 Why Evaluate? Evaluation is a powerful tool for public accountability and learning African Governments have improved their knowledge and appreciation of the value of evaluations as enhancing the efficiency and effectiveness of public policies, programs and projects. Evaluation provides a means of assessing which public initiatives work well, which do not work well and most importantly, why. provide credible way of demonstrating the outcome of government effort in a transparent and consistent manner including how public resources were used and what informed the prioritization in the allocation of public resources. 10 Speaking the Truth to the King Slide 11 11 Enabling environment (institutional framework, power structure & influence) Organizational level (systems, procedures & rules) Individual level (experience, knowledge & technical skills) Systemic factors, i.e., relationships between the Enabling environment, organizations and individuals Successful capacity development requires not only skills & organizational procedures, but also incentives & good governance Successful capacity development requires not only skills & organizational procedures, but also incentives & good governance Influences by means of incentives it creates Levels of Evaluation Capacity (OECD paper - 2006) Slide 12 Michael Q. Patton, 2010 Demand Lens Supply Lens Seeing Through Lens Slide 13 Dimensions of Evaluation Capacity 13 Evaluation capacity can also be categorized into three dimensions : Capacity for Conducting Evaluations - Conducting an evaluation involves both producing the study and communicating and disseminating it, which requires specialized technical capacity; Capacity for Managing Evaluations: Managing evaluations requires a broad understanding of evaluation but can be done without the specialized skills to conduct evaluation; and Capacity for Using Evaluations -The capacity to use evaluations is completely different; users of evaluations are mainly decision-makers and in some cases policymakers. Slide 14 Building Evaluation Capacity- Key Considerations Evaluation capacity must be unbundled: Different evaluation capacities should be taken into account. One size fit all approach should be avoided. It is important to distinguish between the capacity to manage evaluations and the capacity to conduct them, as well as capacity to use evaluations. These are all different capacities; it is not practical to lump them all together under the single term capacity Individual training on how to conduct evaluations is not sufficient for development of national evaluation capacity: For quite some time evaluation capacity was reduced to the capacity to carry out evaluations and to a certain extent this continues today. Experience shows that enhancing individual capacities without strengthening the organization and the enabling environment can be counterproductive as the individual experts may be frustrated by the institutional arrangement and processes Individual training on how to conduct evaluations is not sufficient for development of national evaluation capacity: Also needed is the capability to use evaluations for learning and adapting methods to objectives 14 Slide 15 Opportunities Growing interest among universities and national and regional research institutions to provide services in evaluation. This provides opportunities to work with these institutions in further developing capacities and promoting specialized training in evaluation. Increasing appreciation and demand from African Governments to reinforce institutional capacities to develop evaluation policy and evaluation coordination at national level Increasing pressure on Governments to be transparent and accountable in the use of national resources as well as demonstrating results of their policies, programs and projects. The complexity of governance in the modern world requires officials to have more knowledge for optimal decision-making. Strong and growing demand by donors in civil society organizations (a requirement for accountability of public action); 15 Slide 16 Challenges The current trend in evaluation capacity across the continent is broadly rooted on two main challenges : First is the low demand for credible evidence about performance and the scant use of the information generated through evaluation efforts to inform public decision-making. Of particular concern are, on the one hand, the poor quality of the evidence generated by M&E systems, and on the other, the lack of interest from legislative bodies and citizens, key players in democracies with the authority to demand accountability for results vis--vis public investments such as the media. (National Evaluation Capacities, Proceedings from the International Conference, UNDP-2011) The second problem is the poor integration of institutions and actors associated with the effective evaluation of public policies, programmes and institutions, as well as the lack of convergence among cycles of various public administration processes relevant to broad M&E efforts, such as planning, budgeting and personnel. 16 Slide 17 Demand & Supply of Evaluation Excess supply, or surplus, is the condition that exists when quantity supplied exceeds quantity demanded at the current price (EUROPE & N. AMERICA). Asia at equilibrium, where quantity (E Capacity) supplied equals quantity (E Capacity) demanded. Excess demand, is the condition that Africa has where quantity of E Capacity demanded exceeds quantity supplied at the current price Slide 18 Challenges Contd The third problem is the lack harmony among donors and the national evaluation system. Donors tend to use their own evaluatio