EVALUATION OF DANIDA SUPPORTED … OF DANIDA SUPPORTED RESEARCH ON AGRICULTURE ... by ordering from or ... of agriculture and natural resource management

EVALUATION OF DANIDA SUPPORTED RESEARCH ON AGRICULTURE AND NATURAL RESOURCE MANAGEMENT 2006-2011

2013.04Evaluation

August 2013

Evaluation of Danida supported Research on Agriculture and Natural

Resource Management 2006-2011

© Ministry of Foreign Affairs of DenmarkAugust 2013

Production: Evaluation Department, Ministry of Foreign Affairs of DenmarkCover: Jørgen Schytte/DanidaGraphic Production: BGRAPHIC Print: Rosendahls – Schultz Grafisk

ISBN: 978-87-7087-760-2e-ISBN: 978-87-7087-761-9

This report can be obtained free of charge by ordering from http://evaluation.um.dk/ or from www.danida-publikationer.dk.

This report can be downloaded through the homepage of the Ministry of Foreign Affairs www.um.dk or directly from the homepage of the Evaluation Department http://evaluation.um.dk/.

Contact: [email protected]

Responsibility for the content and presentation of findings and recommendations rests with the authors.

3

Table of Contents

Acknowledgement 5List of abbreviations 6Executive Summary 8

1 Introduction and Context of Evaluation 181.1 Introduction 181.2 Background to Evaluation 21

2 Intervention Mapping 232.1 Introduction 232.2 FFU Projects (North-driven) 272.3 FFU Pilot Research Cooperation Programme (South-driven) 282.4 Building Stronger Universities 292.5 Support for two KU-LIFE Centres 302.6 Research Networks 322.7 Minor Studies 33

3 Methodology and Approach 343.1 Analytical Framework 343.2 Field Visits to Burkina Faso and Tanzania 383.3 Limitations and Challenges 39

4 Centres 404.1 Introduction 404.2 Relevance 404.3 Effectiveness 414.4 Efficiency 424.5 Impact/Sustainability 424.6 Conclusions – Centres 43

5 Networks 445.1 Introduction 445.2 Relevance 445.3 Effectiveness 455.4 Efficiency 455.5 Impact/Sustainability 465.6 Conclusions – Networks 46

6 Minor Studies 476.1 Introduction 476.2 Relevance 476.3 Effectiveness 486.4 Efficiency 496.5 Impact/Sustainability 496.6 Conclusions 49

7 FFU Research Grant Projects (North-driven) 507.1 Introduction and Background 507.2 General Management Issues and Findings 537.3 Relevance 54

4

Table of contents

7.4 Effectiveness 557.5 Efficiency 577.6 Impact and Sustainability 587.7 Conclusions 61

8 FFU Pilot Research Cooperation Programme (South-driven projects) 638.1 Introduction 638.2 Relevance 638.3 Effectiveness 648.4 Efficiency 668.5 Impact/Sustainability 678.6 Conclusions 67

9 Building Stronger Universities 699.1 Introduction 699.2 Relevance 699.3 Effectiveness 709.4 Efficiency 749.5 Impact/Sustainability 759.6 Conclusions – BSUEC and BSUGE 76

10 Synergy and Coherence 7710.1 Introduction 7710.2 Linking Different Modalities 7710.3 Nature of External Links with Development Partners 8010.4 Conclusions 81

11 Conclusions and Recommendations 8211.1 Introduction 8211.2 Strategic Level Conclusions 8211.3 Recommendations 85

Annex A Abbreviated Terms of Reference 88Annex B Evaluation Matrix 101

The following annexes to the evaluation report can be downloaded from http://evaluation.um.dk/

Annex C Major Documents Utilised and Reviewed Annex D Responses to Evaluation eSurvey Annex E Projects Sampled as Case Studies during the Evaluation Annex F Summary of SWOT Workshop Recommendations Annex G Emerging Issues Workshop Report Annex H Programme of Incountry Meetings and Visits Annex I Summary of DDRN Performance against Targets 2007-2009 Annex J Examples of Re-worked Objective and Output Statements

from FFU projectsAnnex K Summary of Calls and Responses by BSUEG and BSUEC Platforms

during Phase 1 of BSU Annex L Summarised History of Burkinabè Institutions

5

Acknowledgement

The Evaluation Team would like to acknowledge the many individuals who have contributed with their time, knowledge and views in support of this Evaluation.

In particular we would like to thank the large number of researchers and other key stakeholders in Denmark, Burkina Faso, Tanzania and other partner countries, who have contributed with valuable inputs to the evaluation process through their participation in the eSurvey, workshops, interviews and site visits.

We also wish to thank the members of the Evaluation Reference Group for lively discussions and constructive comments during our meetings and correspondence.

Last, but not least, we wish to thank the Evaluation Department of the Danish Ministry of Foreign Affairs for very useful interactions and support throughout the process.

6

List of abbreviations

AAU Aalborg UniversityAGI-V Agricultural Genetics Institute, VietnamAU Aarhus UniversityBSU Building Stronger UniversitiesBSUEC Building Stronger Universities, Platform on Environment and ClimateCAADP Comprehensive African Agricultural Development ProgrammeCBNRM Community Based Natural Reseource ManagementCBS Copenhagen Business SchoolCCIAM Climate Change Impacts, Adaptation and MitigationCGIAR Consultative Group on International Agricultural ResearchCNRST National Centre for Scientific and Technological Research (Burkina Faso)DAC Development Assistance Committee (OECD)DDRN Danish Development Research NetworksDFC Danida Fellowship CentreDHI Danish Hydraulic InstituteDIIS Danish Institution of International StudiesDKK Danish KroneDMI Danish Meteorological InstituteDMU Denmark’s MiljøundersøgelserDSHC Danish Seed Health Centre for Developing CountriesDTU Technical University of DenmarkDWF Danish Water ForumENRECA Enhancement of Research Capacity (programme)EPINAV Enhancing Pro-poor Innovation in Natural Resources

and Agriculture Food ChainsEU European UnionEVAL Evaluation Department (Danida)FAAP Framework for African Agricultural ProductivityFFU Consultative Research Committee for Development ResearchFLD Danish Centre for Forest, Landscape and PlanningGEP Growth and Employment Platform, Building Stronger UniversitiesGEPPA Research Network for Governance, Economic Policy

and Public Administration

7

List of abbreviations

GEUS Geological Survey of Denmark and GreenlandGW Group WorkIMF International Monetary FundINERA Institute of Environment and Agricultural Research (Burkina Faso)IRSAT Research Institute of Applied Science and Technology (Burkina Faso)KNUST Kwame Nkrumah University of Science and Technology (Ghana)KU University of CopenhagenKU-LIFE University of Copenhagen, Faculty of Life SciencesLSP Larger Strategic Project, FFU-projectMU Mzumbe University, TanzaniaNARS National Agricultural Research SystemNETARD Network for Agricultural Research for DevelopmentNorad Norwegian Agency for Development CooperationNUFU Norwegian Programme for Development, Research and EducationOECD Organisation for Economic Cooperation and DevelopmentPANTIL Programme for Agriculture and Natural Resource Transformation

for Improved LivelihoodsPREMACA Programa Regional de Medio Ambiente en CentroaméricaPRCP Pilot Research Cooperation ProgrammePWG Platform Working GroupReCom Research and Communication on Foreign AidReNED Research Network for Environment and DevelopmentRIA-V Research Institute for Aquaculture, VietnamRUC Roskilde UniversityRUFORUM Regional Universities Forum for Capacity Building in AgricultureSC Steering CommitteeSDU University of Southern DenmarkSUA Sokoine University of Agriculture, TanzaniaSWOT Strengths, Weaknesses, Opportunities and ThreatsToR Terms of ReferenceUDSM University of Dar es Salaam, TanzaniaUFT Technical Advisory Services (Danida)UG University of GhanaUN United NationsVAST Vietnam Academy of Science and TechnologyWTO World Trade Organisation

8

Executive Summary

Introduction and Approach – Danida has supported development research through various modalities and channels since the 1960s. In order to assess more recent aspects of this support and provide recommendations to feed into the current process of formulating a strategy for development research, Danida’s Evaluation Department (EVAL) commissioned an external evaluation of support for research within agriculture and natural resource management from 2006 to 2011.

The Evaluation was implemented from January to June 2013 with dual objectives, to assess, document and explain the relevance, effectiveness and efficiency – and where possible sustainability and impact – of Danish support to development research within the thematic areas of agriculture and natural resource management (NRM) and to provide lessons learned and recommendations which may feed into on-going discussions on how to improve support to development research, and more specifically into the current process of developing an overall strategic framework for support to development research.

The Evaluation considered the historical and future aspects and implications of two elements of Danida support, the mechanisms and processes of providing it and the products and outcomes of successful delivery. Data collection and analysis was based on a mixed-methods approach, combining quantitative data analysis with qualitative methods, and included site visits to Tanzania and Burkina Faso.

Five funding instruments were considered by the Evaluation: 1) Support to Centres, 2) Support to Networks, 3) Minor studies, 4) Projects under the Consultative Research Committee for Development Research (FFU), both North- and South-driven and 5) Building Stronger Universities (BSU), specifically two of the four platforms.

76% of the 602.1 million Danish funding for agricultural and NRM research under these modalities in the period from 2006 to 2011 was allocated for the directly-funded research cooperation projects under the FFU. Country-wise about 40% has gone to three countries (Tanzania 17.5%, Vietnam 12.6%, and Burkina Faso 9.8%). In Tanzania, 77% of the research funding was linked to Sokoine University of Agriculture (SUA).

Main FindingsCentres – The two centres covered by the Evaluation were the Danish Forest Seed Centre and the Danish Seed Health Centre for Developing Countries. Changes in the Danish national research environment, changing Danida and international development policy, a lack of requests from Danida and Sector Support Programmes for services and the integration of the centres into KU-LIFE resulted in the decision to phase out funding for the support of the centres from 2011 onwards. The main outputs (PhD, MSc and publications) and activities of the centres under their performance contracts were achieved and positive results have been delivered. Collaborating research institutions have been strengthened and the strong personal links with Danish researchers established have been the driving force behind the successful establishment of research projects (including FFU projects).

The centre support modality provided a strong platform for capacity building which has made the collaborating research institutions stronger when competing for funds.

9

Executive Summary

The approach relied on North-driven technical support to develop physical as well as intellectual capacity which over time became an inappropriate mechanism for supporting the building of Southern research capacity for reasons mentioned above.

Networks – From 2006 to 2011, the networks supported by Danida have been variously evolved, merged and disbanded. This started in 2007 and was driven by a number of mechanisms, ending with the merging of three networks which created the Danish Development Research Network (DDRN). In broadening its technical base, DDRN became much more of an information hub, well-placed to promote multi-stakeholder dialogue, meetings and networking. Development-based research projects have been successfully generated out of the links established through the network platforms. However, in becoming an information hub it became less effective as a focussed technical platform of skilled specialists in a position to provide targeted advice.

In contrast, the Danish Water Forum (DWF) continued as a relevant forum for net-working among water sector stakeholders in the North and South, but shifted its focus in response to the demands of its stakeholders and in doing so became a network more relevant to Danish private sector companies in the water sector. Arguably, this has reduced its immediate relevance to Southern stakeholders, especially in Africa, as well as reducing DWF’s interests in development.

The networks established relevant platforms for multi-stakeholder dialogue, networking and exchange of research information across and between South and North that were effective in linking groups and individuals with common research interests. However, mechanisms were never established for the networks to inform Danida-funded sector programme formulation processes. Loss of focus within some networks and changes in aid delivery mechanisms further challenged the networks potential for influencing policy.

Opportunities for utilising DDRN/DWF capacity and skills in research communication and in the dissemination of information and promotion of multi-stakeholder dialogue has not been utilised by the emerging BSU funding modality, although this has been identified as a constraint for the modality.

Minor Studies – Minor studies funding is intended to strengthen the quality of Danish development cooperation by providing guidance and input into strategy development and planning. It is a flexible instrument to promote internal learning, influence policy and strategic thinking and to encourage innovation. Operationally, studies under this modality tend to build on known information, compiling, consolidating and analysing existing knowledge and placing it into a specific policy context. Four studies in the area of agriculture and NRM have been completed in the evaluation period, and each was clearly linked to Danida policy and priorities. All were highly relevant. From the available evidence, these studies all appear to have made an impact on Danida policy and raised awareness amongst the organisation’s decision-makers, which is what they were designed to do. However, with only four studies commissioned for agriculture and NRM between 2006 and 2011, the modality, for all its potential usefulness, appears to be underutilised.

FFU Research Grant Projects (North-driven) – Calls for research proposals are advertised in Denmark, based on Danish priorities and selected themes and administered by the Danida Fellowship Centre (DFC). Proposals are checked for relevance by the Danish embassies and technically assessed by FFU and peer reviewers based on three

10

Executive Summary

criteria of the Quality of the research being proposed, the Relevance of the research to national and Danish priorities for development cooperation and policy and the potential Impact of the research. There is no logframe or result framework for the FFU approach and no clearly articulated objective or set of outputs which describe what it is trying to deliver overall, apart from a very broad sense that it is about capacity strengthening and research. This, and the lack of indicators, makes it difficult to measure the full extent of its effectiveness. Successful projects are by definition relevant, but gender issues were poorly dealt with in both research calls and in the subsequently approved proposals despite the fact that Danida had a strategic focus on women’s rights and access to resources during the evaluation period.

DFC’s role in the general administration and support to logistics and finances of this modality was positively assessed by stakeholders. DFC monitors compliance with administrative and financial requirements of the modality but has no role in technical monitoring or evaluation. This area of oversight and quality assurance is very limited with no formal structures or mechanisms in place. In general, resources have been used as planned and agreed in project proposals.

The embassies have no official role in the operation of the FFU projects and there is no official interaction between project and embassy staff, although some limited personal interactions do take place. The lack of a mechanism to support embassy-FFU project interaction has meant that opportunities to incorporate or promote research output in programme support planning have not been realised.

Within the FFU project structure and the other funding modalities under evaluation, there have been several shifts in strategic direction. These changes have had implications for stakeholders and those evaluating proposals, but despite this there has been no deliberate change management process to support them.

The mechanisms and platforms for information exchange are often limited in partner countries, and this reduces stakeholder access to research output. There is an inherent tension between the need for immediate outcomes and benefits at farm-level, often a priority for Southern researchers, especially those in national research institutes, and the production of high quality research for publication in peer-reviewed journals, which is a key requirement of the Danish university system.

The FFU projects have provided an attractive, bilateral funding modality in a relatively closed institutional environment that has benefited researchers in Denmark and selected Southern countries. It is a modality that has been largely driven from Denmark and places emphasis on quality research and publications linked to postgraduate degree train-ing. It has been successful at providing this training and at the same time has generated a number of research publications; the outputs in terms of PhD degrees and publications have been documented as part of the Evaluation for Burkina Faso, Tanzania and Vietnam as an illustrative example.

Impact of the FFU projects is difficult to measure. For example, although Danida support to research activities is relatively high at Sokoine University of Agriculture (SUA) in Tanzania between 2006 and 2011, attribution has proved difficult/impossible. Norway is by far the most influential development partner at SUA providing a large share of funding through comprehensive institutional research programmes, which have aimed at addressing capacity development issues at the institutional level, while

11

Executive Summary

the Danish research funding to FFU projects at SUA has focused on support to individual r esearchers.

Without a comprehensive strategy for support to development research, any modality which is functioning well and delivering postgraduate degrees and publications can be judged a success. The question remains as to whether, in the participatory and holistic environment which characterises modern development research, the FFU project approach remains an appropriate modality.

There are two separate, but related issues. Firstly, the FFU project research is driven by a thematic approach based on shifting Danida priorities, and projects are linked to the skills and strengths of Danish researchers, the majority of whom are in universities. Secondly, the linear approach to research assumes that the responsibility for dissemination and uptake of successful outputs is outside the project boundary and the responsi bility of others. Both of these issues need to be addressed in designing the new Danish Strategy for Development Research.

FFU Pilot Research Cooperation Programme – PRCP (South-driven projects) –The South-driven PRCP research projects under FFU are seen as a mechanism for making support to research more relevant to Southern partners and represent a new approach for Danish funding. The calls have required that the project proposals were developed in the context of Southern priorities linked to Danish strategic development priority areas, although the link to Danish priorities has now been dropped.

The modality operates in Ghana, Tanzania and Vietnam, and the Evaluation has focussed its analysis on Tanzania, where there were five projects implemented within agriculture and NRM during the evaluation period. In all five cases, the South-driven research projects were of high relevance and provided value-addition to the key research areas in their respective institutions.

Concept Notes for the PRCPs have been prepared by Tanzanian researchers, and Danish partners have played an active role in converting the notes into project proposals. On occasion, the Concept Notes have been of poor quality, pointing to a wider issue on the need for support to related change management and capacity development processes within national partner institutions.

The interest among Danish researchers to respond to the open PRCP calls has been limited due to a number of reasons, although there are also good examples of how the South-driven research project modality has created new partnerships and links between Danish and Southern researchers. The modality has proved mutually beneficial and encouraged continued cooperation between the project partners, and where projects have been approved and implemented, satisfaction amongst stakeholders is high.

The South-driven research project modality represents a new and relevant approach, which creates strong ownership in the South, positive outcomes and new partnerships. It addresses issues linked to ownership and relevance as part of Danida’s current approach to development research and alignment with the Paris Declaration on Aid Effectiveness.

By being more responsive to Southern demands and better reflecting national priorities and issues, it has created potential for leveraging additional funding. At the same time it has sparked new partnerships and opened up the possibilities of wider collaboration

12

Executive Summary

between Danish and Southern institutions. However, although the modality has many positive features it remains somewhat trapped within the same research framework (academically-focussed linear model involving, for the large part, university-based research) as the FFU projects. There is the additional problem that many Danish researchers feel that incentives to engage are limited.

Building Stronger Universities – Environment and Climate (BSUEC) and Growth and Employment (BSUGE) Platforms – The initiative originated from the Danish Universities Rectors’ Conference and was developed further in consultation with partners in the South and in dialogue with Danida. The process resulted in a compromise structure with four platforms, a co-funding arrangement and a focus on institutional capacity building and not research. The Evaluation considered only two of the four platforms, and findings and recommendations should be viewed in this context.

Detailed interviews and discussions were held with Danish administrators and researchers involved in the two platforms in Tanzania and Ghana, but due to travel limitations, only Tanzanian partners were interviewed in-country. The findings from interviews with Danish stakeholders who were working in both Tanzania and Ghana were similar.

The BSU aim is to strengthen institutional capacity at Southern universities through partnerships with those in Denmark. However, the platform approach is thematically based on Danish strategic priorities, and as such the platforms are not necessarily relevant to this aim. Despite the fact that later inception workshops were held at Southern partner universities, the overall framework for the BSU concept and Phase 1 design was devel-oped and agreed in Denmark, and no systematic capacity needs assessment was done.

The ownership of the BSU initiative within the Southern partner universities has been weak. Institutionalisation of new courses developed under BSU has been hindered, at least in Tanzania, by the lack of plans or resources for incorporating them into the mainstream of the university academic and administrative structures. The generic and specific PhD courses that have been taught by Danish senior researchers for PhD students and supervisors at the partner universities in the South, however, have been popular and well-attended.

At the operational level the objectives of the BSU are not supported with clear and measureable outcome indicators (short-term and long-term), nor have baseline data been collected against which to measure improvements.

Although BSU key stakeholders at both SUA and University of Dar es Salaam considered the BSU to be a useful gap filler, the BSU platforms in Tanzania are not coordinated or collaborating with other related interventions funded by other development partners within the same universities. The administration costs of the BSUEC and BSUGE platforms are both above 25% due to a complex governance structure, considerably higher than for the other modalities covered by this Evaluation.

BSU was established without a systematic assessment of Southern university needs, and the assumption was that Danish universities had the skills and capacity to address those unknown Southern priorities. Requests for support are demand-driven from the South, and implementation depends on the willingness and ability of the seven Danish universities under BSU to respond.

13

Executive Summary

For different reasons, the incentives for the majority of senior researchers in the South and Denmark to engage in the process are insufficient. This has led to a low level of participation, except by PhD students. Wider attempts to improve institutional skills and capacity have been hindered by a lack of integration in Southern university systems.

The idea behind BSU, of strengthening institutional and research capacity of Southern universities in key areas by calling on the skills and capacity available in Denmark, is a sound one. However, the operationalisation of this concept has been flawed and threatens its immediate and medium-term success. The existing BSU governance structure is not appropriate for the aims of BSU, and is both expensive and cumbersome. There were no indications from the platforms visited that BSU in its current form will produce any lasting and documentable results within the South partner universities.

Synergy and Coherence – The Evaluation has considered five funding modalities (six if the Minor Studies component is included) over a six year period. These have not all been operating at the same level or for the same length of time and they represent a succession of funding and activity levels. Although the Evaluation found evidence of interactions between these different modalities, the synergies were mostly on an ad hoc basis and because the same individuals are involved in multiple modalities.

There is no systematic or formal set of procedures which give guidance on how information should or could be shared, and the limited formal role for embassies means that in-country knowledge of agricultural and NRM research by embassy staff is limited to initiatives stimulated by personal interest. Synergies and coherence are dependent on individuals and not institutions.

Under the current system for implementing centrally funded research to agriculture and NRM, Danish embassies in-country have no formal role in management of the research activities and there is no obligation to connect with other development partners.

Overall Conclusions and Recommendations Based on the findings above, the following main conclusions and recommendations are drawn from the Evaluation of Danish support to development research between 2006 and 2011 within the thematic areas of agriculture and NRM:

Paradigms for Agricultural and NRM research – The focus of the current research paradigm under which the bulk of FFU projects are operating is a North-driven, thematically-organised, academically-focussed linear model involving, for the large part university-based research in Denmark and the South. Current development thinking has moved away from this approach to a more holistic view. Similarly, capacity strengthening under the BSU also needs to be reconsidered, and the appropriateness of the current model reviewed.

The most appropriate research paradigm and approach to capacity strengthening will depend on the strategy for development research which Danida adopts. One option, however, is provided by an approach widely used throughout Africa and South Asia, which is built on a broad-based stakeholder grouping that considers issues in a wide context (value chain) and then utilises those best placed to develop and implement different elements to provide a combined solution. Sometimes referred to as integrated agricultural research for development or agricultural innovation systems, this type

14

Executive Summary

of approach is applicable where the need exists for a strong agricultural sector which can drive economic growth.

The extent to which cross-cutting issues have been dealt with between 2006 and 2011 is mixed. Some have received priority treatment (climate change, environment), others have been less obviously considered (gender, youth). The current Strategy for Denmark’s Development Cooperation identifies quite clearly several key priorities which cut across sectors and which influence and effect support to research in agriculture and NRM. Of these, Green Growth, Stability and issues linked to the Human-Rights Based Approaches and Gender Equality are perhaps the most clearly relevant and should be integrated into the new paradigm.

Recommendation 1. In developing its new Strategy for Development Research, Danida should consider institutionalising a research paradigm which moves away from the current linear model, to one that is holistic, participatory, linked to value chains and largely driven by Southern priorities.

Recommendation 2. As part of its new Strategy for Development Research, Danida should consider including a particular focus on the need for support to strengthening of national institutional frameworks and capacities for planning and coordination of development research within Southern partner countries. This would include support to formulation and implementation of relevant strategies and policies for prioritising and coordinating of research interventions within and across research institutions in the partner countries. Such a focus would benefit from stronger embassy engagement in research activities (see also Recommendation 3).

Mechanisms for Communication and Coordination – The interaction and sharing of information across and between modalities and stakeholders has been sub-optimal, although there have been exceptions. There are two main reasons for this: firstly, the nature of individual projects largely focussed on research as an end in itself and, secondly, the lack of institutionalised mechanisms which place specific requirements on those implementing projects to share. This needs to be addressed at a strategic and operational level.

Recommendation 3. The roles and responsibilities of Danish embassies in relation to planning and implementation of Danida research initiatives should be redefined and institutionalised to become a more useful platform for follow-up and sharing of information as well as for potential application of relevant results from in-country research activities, including in relation to Danish-funded sector support programmes. Specific issues to consider:

3.1 If the potential for a more programme-based approach to development research will be positively considered by Danida in countries with high levels of Danida supported research activity (see also Recommendation 2 and 8), the relevant Danish embassies should become more involved with coordination, follow-up and contact to supported national research institutions to ensure synergy and coherence, including with research activities supported by other Development Partners within the partner countries.

3.2 Annual circulation, by DFC to embassies, of 20-line summaries abstracted from the FFU progress/completion reports and BSU annual project reports.

15

Executive Summary

3.3 Requirement for more systematic briefing of embassies on planned country visits from FFU and BSU project coordinators and staff.

3.4 An annual in-country research event (e.g. workshop or seminar) with partici-pation of representatives from BSU and FFU projects, national governmental institutions, the embassy and possibly other stakeholders as well (e.g. other Development Partners, private sector actors, national research institutes).

Planning, Implementation and M&E – The evidence shows that one of the most significant features influencing the weaknesses identified during the evaluation period has been the lack of a clearly articulated strategy and plan on which funding modalities and implementation can rely. Such a plan could have linked and supported coherence of the various projected activities and provided a basis for monitoring and evaluation. This is currently being addressed.

None of the funding modalities (past or present) have, or had, indicators which allowed the assessment of changes or outcomes linked to funding. At best, product indicators existed in some cases, measuring the number of degrees, publications or workshops. A loose theory of change approach, linked to broad objectives, has been implied, but the Evaluation finds that a causal framework based on a logframe approach (LFA) would have provided a number of clear and distinct advantages not only for monitoring and evaluation but also for planning, implementation, communication and coordination.

In the context of M&E, monitoring of administrative compliance has been done by DFC, and although there are gaps and inconsistencies in the reports available, generally this has worked well. There is, however, no mechanism in place for technical evaluation of research projects due to the lack of baseline data and the inconsistency and inappro-priate nature of indicators. Even projects pre-2009 with logframes lacked indicators other than those measuring product and the quality of the logframes themselves which were of little practical use as monitoring tools. Clear and coherent planning for interventions will be a prerequisite for the success of Danida’s support to development research in agriculture and NRM in the future. An LFA at strategic level will make it feasible for a similar, nested results-based approach to be developed for research projects and activities. This would also be in line with how development research is currently being planned and implemented by other development partners and Southern organisations.

Recommendation 4. In the development of its new strategy for development research, Danida should use an LFA including a stakeholder and problem analysis. The develop-ment of such a framework should precede the writing or formulation of any strategy.

Recommendation 5. The LFA and Result-Based Management (RBM) should be institutionalised within Danida’s modalities for funding development research and utilised from the strategic level down to projects and other funded activities. It should be used to support and encourage more coherent nesting and linking of activities and funding and used to demonstrate clear causal links between inputs, activities, outputs, outcomes and objectives.

Recommendation 6. Specifically for the institutionalisation of the LFA, the change management support (see Recommendation 7) should be provided with adequate resources and include:

16

Executive Summary

6.1 Training and capacity strengthening in LFA and RBM, and the sensitisation of stakeholders to the new Strategy.

6.2 Agreement on clear definitions and instructions on what constitutes an objective, output, outcome and indicator for inclusion in DFC guidelines.

6.3 Technical monitoring and evaluation of research projects against agreed product and outcome-based indicators linked to logframes should be included in Annual Reports (against milestones or intermediate indicators) and Project Completion Reports, requiring baselines at project start.

Change Management – There have been several changes in the modalities and nature of funding during the period under evaluation with limited formal support, consultation and guidance. For new approaches and modalities to succeed, ownership and under-standing are critical. Implementing change requires deliberate management of the process and future changes in direction, introduction of new modalities and ideas. Introduction of the new Danish Strategy for Development Research will need to be accompanied by sensitisation and consultation workshops involving a broad-base of stakeholders.

Recommendation 7. The introduction of any new strategies, funding instruments, tools or guidelines should be deliberately managed and institutionalised using change management principles and fully supported with well-resourced integrated workshops, documentation, capacity strengthening and technical support, as appropriate.

Funding mechanisms – Currently, both the North- and South-driven FFU projects are funded on a project by project basis, which can make coordination and coherence at programme level difficult for organisations, especially in the South. There are several options and models for the disbursement of funds to support research and capacity strengthening. Some development partners utilise a basket funding approach through externally managed multi-donor trust funds (DFID, EU, USAID, CIDA), others, including Danida, provide funds at programme-level to national or sub-regional organisations (Norad, World Bank, AusAID). For FFU projects between 2006 and 2011, Danida has relied on project funding on an individual researcher basis with project- designated funds channelled through Southern institutional systems for specific activities.

The Evaluation has interacted with a number of organisations and administrators, both North and South, and concludes that a more institutional and programme-based approach would be closer to current trends in development assistance.

Recommendation 8. Where feasible, development research funds should be provided directly to organisations in support of programmes, rather than projects and individuals, in parallel with developing the appropriate institutional capacity to manage them. As an interim step, resources for South-driven FFU projects should continue and be increased by reducing, or merged with, North-driven project support in those countries.

Capacity Strengthening – Currently, the key mechanism for capacity strengthening is the BSU initiative, and whilst the idea underpinning BSU is sound, operationalisation, at least for the two platforms evaluated (BSUEC and BSUGE), has been flawed. The Evaluation finds a significant shift in BSU’s strategic approach should be considered after the current phase has been completed.

17

Executive Summary

Capacity strengthening is not just about PhD and MSc degrees, and many stakeholders in the South and in Denmark expressed the view that a broader definition should be adopted, which considered capacity strengthening as an aspect of empowerment which provides stakeholders with the skills to access and use information, work effectively and efficiently within their institutional systems and interact and respond to wider challenges. These are issues that could be, and to some extent are being, addressed under BSU.

Recommendation 9. In the short term, the BSU governance structure should be simplified. Specifically the administrative and technical functions of BSUGE and BSUEC platforms should be merged and a common secretariat established that has a communication function linking to the other platforms.

Recommendation 10. A comprehensive, independent, technical review of the whole BSU initiative should be implemented as soon as possible to inform a decision as to whether it should be continued in its current form. Issues to be considered should include:

10.1 The cost-effectiveness of including the BSU concept as a new capacity develop-ment and empowerment modality nested within the Danida development research strategy, technically under FFU and administered by DFC, as they have the experience and skills to do this.

10.2 Narrowing the Southern-focus of BSU to permit larger, institution-based inputs at fewer Southern partner universities and reviewing the current group of Southern partner universities1 to determine whether support should be to smaller, under-resourced universities with greater potential for generating internal change and impact.

10.3 The nature and options for improving incentives and ownership of BSU.

1 The current Southern partner universities are relatively large universities which already receive significant amount of external funding.

18

1 Introduction and Context of Evaluation

1.1 Introduction

Denmark’s development cooperation (Danida2) has supported development research across a range of subjects for several decades, primarily through account §06.35 of the Danish Finance Act. Sub-accounts correspond to research and information activities in Denmark (§06.35.01) and international development research (§06.35.02), and these are further subdivided into more specific channels, each with a specific purpose.

An overview of the specific channels presented in the Finance Act of 2012, is given in Table 1 below. The channels covered by the present evaluation are listed in bold type.

Table 1 Overview of channels of support for development research

Four-digit account Six-digit account Eight-digit account

§06.35Research and information activities

§06.35.01Research and information activities in Denmark

§06.35.01.10 Projects in Denmark

§06.35.01.11 Research activities

§06.35.01.13 Information activities

§06.35.01.14 Intercultural cooperation

§06.35.01.15 Fact-finding activities (minor studies)

§06.35.01.17 Seminars, courses, conferences etc.

§06.35.01.18 Evaluation

§06.35.02 International development research

§06.35.02.10 The consultative group on interna-tional agricultural research (CGIAR)

§06.35.02.11 Other international development research

Source: Danish Finance Act of 2012.

Together §06.35.01.10 and §06.35.01.11 constitute the overall frame for support to research on development related topics and capacity building.

Between 2006 and 2011, annual support to all development research initiatives ranged between DKK 200 and 285 million, and has been disbursed through various multilateral and Danish channels. For the institutes and mechanisms based in Denmark, including the Consultative Research Committee for Development Research3 (FFU), Danish universities and research networks, the annual figure for support to research in agriculture

2 The name Danida appeared in 1963 as a contraction of Danish International Development Agency and, subsequently, Danish International Development Assistance. Today Danida is no longer a contraction but has been retained as the term for Denmark’s development cooperation (http://um.dk/en/danida-en/about-danida/history/).

3 Rådgivende Forskningsfaglige Udvalg, appointed by the Danish Minister for Development Cooperation.

19


and natural resource management (NRM) ranges from DKK 69 to 116 million, with an average of DKK 100 million per year.

Denmark cooperates with research institutions and think-tanks to enhance Southern countries’ own research and to generate new knowledge that can be applied in development. The broad objectives of this approach are:

• To strengthen countries’ own research through graduate training and PhD programmes and to ensure that developing countries have access to knowledge.

• Create new knowledge that can be applied in supporting development including for example issues of climate change, sustainable energy, the use of mobile telephone technology and value chains in the private sector.

Working with, and supporting, international research organisations aims to provide research results, to perform consultancy and to implement education and capacity building that benefit developing countries. From 2006 to 2011 Danida channelled DKK 321 million to the CGIAR4 system through the CGIAR Fund, as well as DKK 414 million through FFU North-driven projects.

Research is also supported through multilateral initiatives including the European Union (EU), the United Nations (UN), the World Bank, International Monetary Fund (IMF), the World Trade Organisation (WTO) and other global fora such as the G20. Support is given to the Global Environment Facility (GEF) which is one of the most important instruments for furthering environmental and climate initiatives in the developing countries.

In order to assess support via Danish institutions and organisations, as well as to provide recommendations to feed into the current process of formulating an overall strategy for development research, Danida’s Evaluation Department (EVAL) commissioned Orbicon A/S and ITAD to undertake an external evaluation5. The Evaluation has been limited to support for research within agriculture and natural resource management during the period 2006 to 2011, abbreviated Terms of Reference are presented in Annex A.

4 The CGIAR consists of an aligned global partnership among 15 international agricultural research centres (CGIAR Centres) that conduct research into agriculture, forestry and fisheries. Further-more, research centres looking into economic, policy and institutional aspects related to global food and nutrition security are also included. The CGIAR Fund is a multi-donor, multi-year trust fund with the World Bank, Washington D.C., as a trustee and administrator of the Fund. The CGIAR Fund is financing the portfolio of 15 thematic research programmes aiming at four selected system level outcomes: reducing rural poverty; improving food security; improving nutrition and health; assuring sustainable management of the natural resources.

5 The Team from Orbicon-ITAD comprises: John Sutherland, agricultural research and evaluation specialist (Team Leader); Carsten Schwensen, development economist and evaluation specialist (Deputy Team Leader); Anne Højmark Andersen, agriculturalist/natural resource management and evaluation specialist (Burkina Faso); Damian Gabagambi, agricultural economist and evaluation specialist (Tanzania).

20


The dual purpose of this Evaluation has been to:

• Assess, document and explain the relevance, effectiveness and efficiency – and where possible sustainability and impact – of Danish support to development research within the thematic areas of agriculture and natural resource management.

• Provide lessons learned and recommendations which may feed into on-going discussions on how to improve support to development research, and more specifically into the current process of developing an overall strategic framework for support to development research, which is expected to be published in September 2013.

The Evaluation was carried from January to June 2013 and included field visits to Burkina Faso and Tanzania.

Structure of reportThis report contains details of the methodology and approach used by the Evaluation and the findings, conclusions and recommendations based on a range of information and data sources. It is split into 11 chapters as follows:

• Chapter 1 Introduction – this provides the background to the Evaluation.

• Chapter 2 Intervention Mapping – this provides an overview of the five modalities which were at the heart of the Evaluation, and characterises these in terms of their history and the resources used.

• Chapter 3 Methodology and Approach – which covers the tools used in collecting and analysing data, and explains the limitations and assumptions of the methodo logy.

• Chapters 4 to 9 Modality Evaluation – these chapters cover the five modalities in some detail and measure performance against the five Organisation for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria. The interventions under the FFU modality are split between North- and South-driven projects. At the end of each chapter there is a section on the Conclusions that the Evaluation has drawn from the findings reported.

• Chapter 10 Synergy and Coherence – considers how the various modalities have interacted, supported and to some extent developed, from each other, highlighting areas relevant to the current issues.

• Chapter 11 Conclusions and Recommendations – draws overall Conclusions from the Evaluation at strategic and operational levels with Recommendations which are also medium- to long-term (strategic) and short- to medium-term (operational).

• Annexes – a number of annexes provide supplementary information and data not included in the main report.

21


1.2 Background to Evaluation

Changing strategiesThe overall objective of Danish development assistance was articulated in 1971 in the Law on International Development Assistance and is to reduce poverty with the inten-tion that this will contribute to social progress and political independence6. Development research was considered as one of several tools, but no detail was provided in the Law itself. During the evaluation period, Denmark has had two development strategies, Partnership 20007 in 2000 and Freedom from Poverty – Freedom to Change in 20108,9. Partnership 2000 highlights the increasing importance of research and research-based knowledge for development cooperation and calls for a closer connection between research and policy development. The main objective of Danish support to development research has been to contribute to the solution of developing countries’ problems, both in terms of new research results and in the building of capacity. Partnership 2000 cites the need to utilise improvements in the professional and political basis for decision making, to improve the overall performance of Danish development cooperation.

In 2006, at the start of the evaluation period, three priority focus areas for development assistance10 were identified by the Danish Government, within the context of poverty eradication:

• Good governance – a fundamental prerequisite for development

• Women – a driving force for development – focus on strengthening women’s rights and access to resources – access to education – strengthening position of women in Africa (Danida’s five-point plan on gender equity)

• HIV/AIDS.

It gave high priority to promoting a sustainable environment in developing countries linked to a thematic area which targeted efforts contributing to sustainable management of the environment and natural resources. It included recognition of the significance of climate change as a growing challenge that makes demands on adaptation to ensure that the living conditions of vulnerable population groups in developing countries do not deteriorate.

More recently Freedom from Poverty – Freedom to Change mentions the importance of documenting and communicating results, and that the internal learning process should be enhanced through research and evaluation as well as focusing on applying research results in developing countries with an approach that uses value-chains in a farm-to-fork modality.

6 The Danish Law on International Development Assistance (Lov om internationalt udviklings-samarbejde; First version, 1971).

7 Denmark’s Development Policy Strategy – Partnership 2000. The Danish Ministry of Foreign Affairs, Copenhagen, 2000.

8 Freedom from Poverty – Freedom to Change Strategy for Denmark’s Development Cooperation, The Danish Ministry of Foreign Affairs Copenhagen, July 2010.

9 A new strategy for Danish Development Cooperation, “The Right to a Better Life”, was launched in 2012. A new Act on Danish Development Cooperation was passed by the Danish Parliament in 2012 and entered into force on 1st of January 2013.

10 Commitment to Development – Priorities of the Danish Government for Danish Development Assistance 2007-2011. Ministry of Foreign Affairs, Copenhagen, Denmark. August 2006.

22


The Technical Advisory Services (UFT) is currently working on a strategic framework for Danish support to development research to create clear direction for the future prioritisation of the diverse elements within the Danish Government’s support. The strategic framework for development research is expected to strengthen the opportunity for development research to contribute to the overall objective of the development assistance11 as well as making it easier to monitor the relevance, quality and effect of the support to development research.

Past evaluations Two key technical evaluations cover some of the modalities under consideration by the present study. They are an evaluation of the ENRECA (Enhancement of Research Capacity in Developing Countries) modality in 200012 which focuses on one funding option that was managed by the fore-runner to the FFU, and the more significant Commission on Development-Related Research in 2001, often referred to as the Hernes’ Report13 after its senior author, which is a wide ranging and detailed look at development research supported by Danida.

Since 2001, many of the recommendations of the Hernes’ Report and its annexes have been dealt with by Danida, but others re-emerged as issues during the current Evaluation and are dealt with in the following chapters, most notably those relating to:

• Communication and the need for a coherent and structured approach to infor mation management.

• The research paradigm – during the evaluation period there has been a tendency for a linear model to be used in which research identifies issues, develops solutions, passes on responsibility for uptake to extension and extension passes information on to farmers. Current thinking, and approaches suggested in the Hernes’ Report, is for a more holistic approach involving a broad base of stakeholders and value chains.

• The strengthening of partnerships.

• The development of broad-based institutional capacity.

• The linking of research to sector programmes.

11 To strengthen research capacity in developing countries and create new knowledge which can contribute to solving developmental problems.

12 Evaluation of Danida’s Bilateral Programme for Enhancement of Research Capacity in Developing Countries (ENRECA), Ministry of Foreign Affairs of Denmark. December 2000.

13 Commission on Development-Related Research (2001): Partnerships at the Leading Edge: A Danish Vision for Knowledge, Research and Development (the Hernes Report). Ministry of Foreign Affairs of Denmark. April 2001.

23

2 Intervention Mapping

2.1 Introduction

Within development research, the most important areas during the period of the Evaluation have been agriculture and food, health, environment and climate, and economic growth and employment. The total amount granted for the activities carried out within the scope of this Evaluation is approximately DKK 600 million (see Table 2), or 42% of the budget allocated under the Finance Act. This chapter describes and summarises the key characteristics of the five modalities under evaluation.

The purpose14 of funding channel §06.35.01.10 is to support the use of Danish competencies in development research. Specific modalities over the period covered by the present Evaluation have included support to research networks and to research centres. From 2011 this support was replaced by a new modality, Building Stronger Universities in Developing Countries (BSU), comprising four thematic platforms, of which two, Environment and Climate and Growth and Employment, were covered by the Evaluation.

The FFU guides funding through §06.35.01.11, the purpose of which is to support the development of new knowledge for the benefit of developing countries and to maintain and expand the capacity of these countries with respect to producing research. The FFU assesses which applications are worthy of support and forwards their recommendation for the formal consent of The Danish Council for Strategic Research (DCSR). The Ministry of Foreign Affairs makes the final decision on which applications to support. Since 2008 these funds have been split between North-driven and South-driven research, prior to this all research was North-driven.

Table 2 Overview of Fund allocation for five modalities under evaluation for Agriculture and NRM research and Research Networks, 2006-2011

Modality and Budget code 2006 2007 2008 2009 2010 2011 Total

1. Centres (§06.35.01.10) 15.5 15.5 17.0 14.0 12.0 0.0 74.0

2. Networks (§06.35.01.10) 6.5 13.0 0.0 14.0 0.0 0.0 33.5

3. BSU* (§06.35.01.10) n/a n/a n/a 2.5 1.5 30.0 34.0

FFU North-driven (Agriculture & NRM)

45.2

88.1

76.2

70.8

61.9

71.6

413.8

FFU South-driven (Agriculture & NRM)

n/a

n/a

5.5

14.3

14.8

10.1

44.7

4. FFU (§06.35.01.11) Total 45.2 88.1 81.7 85 76.8 81.7 458.5

5. Minor Studies** (§06.35.01.15)

0.0

0.0

0.6

0.0

0.4

1.1

2.1

TOTAL 67.2 116.6 99.3 115.5 90.7 112.8 602.1

* Environment-Climate and Employment-Growth Platforms only. ** Funds for agriculture and natural resource management related studies.

14 According to the Finance Act of 2012.

24


Together §06.35.01.10 and §06.35.01.11 constitute the overall frame for support to research on development related topics and capacity building.

Funding through §06.35.01.15 is for minor studies of a fact-finding nature, implemented with the main objective of strengthening the quality of Danish development cooperation. The utilisation of this fund under agricultural and natural resource management has been very limited during the evaluation period.

Allocation of funding and projects for FFU in Tanzania, Vietnam and Burkina FasoIn this section there is a brief analysis of FFU project activity from 2006 to 2011 in Tanzania, Vietnam and Burkina Faso. It should be noted that the funding indicated is the total value of the project, and is not an indication of the funding that was received and managed by the institutes directly. This varied according to the type of project, for example with small project PhD awards 100% of funding was allocated to expenditure in Denmark and with other projects between 25 and 60% for funding activities in southern partner countries.

In Tanzania both North- and South-driven projects have been supported between 2006 and 2011, and by way of a case study and illustration, the resources and project activities have been mapped and are summarised in Table 3 which shows the number of projects, resources and degrees/publications by institute and project modality, over the period.

Table 3 Summary of Project type and resource allocation for FFU projects (North- and South-driven) in Tanzania 2006-2011

Institute

Projects Value

DKK million PhD MSc

Peer Reviewed

PapersNorth South

Ardhi University 1 0 2.8 5 0 n/a*

Mzumbe University 0 1 5.3 n/a n/a n/a

NPGRC** 3 0 12.4 5 10 3

SUA 15 4 104.6 21 35 73

UDSM*** 2 0 6.3 1 n/a 2

Undesignated 4 – 5.3 2 7 18

Totals 30 136.7 34 52 96

* n/a = not available from the documentation to which the Evaluation had access. ** NPGRC = National Plant Genetic Resources Centre, Tropical Pesticides Research Institute. *** UDSM = University of Dar es Salaam.

During the period the total funding to FFU projects in Tanzania has been DKK 136.7 million of which DKK 104.6 million (77%) went to projects where SUA was a partner institution. Four of the projects in the data sheets had no Tanzanian organisation specified.

The distribution of FFU North- and South-driven projects in Vietnam is shown in Table 4, with the nominal numbers of PhD and MSc awards and the numbers of projects publications in peer-reviewed journals. The total value of projects over the period was DKK 104.9 million. It should be noted that several projects have a multi-institutional

25


base and the funding allocation between the different institutes is not known and is indicative only. At least 16 institutes were involved in Danida supported activities between 2006 and 2011, with the largest nominal funding going to Can Tho University. The spread of institutes and projects is much wider than for either of the two African countries analysed.

Table 4 Summarised data for FFU North- and South-driven, agricultural and NRM research projects in Vietnam, 2006-2011*1516

Institute

Projects Value

DKK million PhD MSc

Peer Reviewed

PapersNorth South

Can Tho University 4 0 25.3 23 90 65

Hanoi Agricultural University 2 1 17.6 6 8 8

Hanoi Medical University 1 0 (shared project)

Hanoi University of Mining and Geology 1 0 8.3 5 2 0

Hanoi University of Sciences 2 0 (shared projects)

Meteorological Department 1 0 (shared project)

Ministry of Health15 1 0 9.4 4 1 5

National Institute of Animal Science 1 0 (shared project)

National Institute of Veterinary Research 1 0 (shared project)

Research Institute for Aquaculture 1 1 13.2 2 1 0

Vietnam Academy of Science and Technology 1 4 12.4 4 6 0

Vietnamese Academy of Agricultural Sciences16 4 1 18.2 12 11 7

TOTALS* 19 104.9 41 107 85

* There are several projects which involve more than one Vietnamese institute. Where possible from available documentation, resources, awards and publications have been assigned to what is believed to be the lead institute.

** The absolute number of projects is shown here, many of the projects involve multiple institutes in Vietnam.

15 Projects at the following institutes: National Institute of Hygiene and Epidemiology, National Institute of Occupational and Environmental Health, National Institute of Nutrition.

16 Projects at the following Institutes: Southern Fruit Research Institute (SOFRI), Soils and Fertilizers Institute (SFRI), Cuu Long Rice Research Institute (CLRRI), Agricultural Genetics Institute.

26


For Burkina Faso, with fewer projects (a total of 13 for which data are available), the total value of North-driven projects is DKK 69.6 million over the period. The largest share (DKK 35.0 million) is going to six projects in the Institute of Environment and Agricultural Research (INERA) and the Research Institute of Applied Science and Technology (IRSAT), part of the National Centre for Scientific and Technological Research (CNRST) complex of institutes. These data are summarised in Table 5.

Table 5 Summarised data for FFU North-driven, agricultural and NRM research projects in Burkina Faso, 2006-2011

Institution ProjectsValue

DKK million PhD MSc

Peer-Reviewed

Papers

AMMA Burkina Faso 1 2.1 1 0 5

Centre national de semence forestière 1 2.9 0 0 0

CNRST (INERA+IRSAT) 6 35.0 7 2 4

University of Bobo 1 2.6 0 0 0

University of Ouagadougou 4 27.0 4 9 6

TOTAL 13 69.6 12 11 15

Country distribution of FFU projectsThe distribution of the 88 FFU projects under the Evaluation by country shows the bulk of the projects in Africa. A number were multi-country within the same continent (20) and in six cases covered Asia, South America and Africa.

The seven main project locations are shown in Figure 1, the category Other covers countries with five or fewer projects, as shown in Table 6.

Figure 1 Distribution of FFU projects in Agriculture and NRM, by country for North- and South-driven support for 2006-2011 (n=88)

35

17.5

12.69.8

8.4 7.7 7.76.3

30.130

25

20

15

10

5

0Tanzania Vietnam Burkina Faso Ghana Kenya Uganda OtherMali

% of All Projects

27


Table 6 Distribution of Projects for Countries with five or fewer projects

CountryAs a % of total projects

for each countryNumber of

projects in Country

Mozambique, Nicaragua 3.5 5

Bolivia, Cambodia, Nepal, Zambia 2.8 4

Benin, Bhutan, Senegal, Thailand 1.4 2

Bangladesh, China, Colombia, Costa Rica, Ecuador, Ethiopia, Namibia, Peru, Philippines 0.7 1

(Note: The percentage of Total projects for each country means that 3.5% of projects were in Mozambique and 3.5% in Nicaragua, 2.8% in Bolivia, 2.8% in Cambodia, and so on. The total is 30.1% of projects in the category Other in Figure 1.)

2.2 FFU Projects (North-driven)

The main share of Danish funding for support to agriculture and NRM research in the period from 2006 to 2011 was to directly-funded research cooperation projects (FFU projects) submitted by researchers attached to Danish research institutions.

Grants were awarded to strategic research cooperation projects/activities which were expected to strengthen research capacity and generate new knowledge relevant to the needs and strategies of developing countries and to Denmark’s development cooperation. It includes, or has included, larger strategic programmes (including ENRECA), smaller projects (initiative grants, post-doctoral studies and PhD awards) and Travel Grants. Grants varied in size from DKK 150,000 to DKK 11 million with nearly two thirds of the FFU projects approved for funding in this period being located in Africa (50/87)17.

Table 7 Distribution and value of all agriculture and NRM North-driven Projects, 2006-2011

Project Type Numbers % Value Million DKK %

Larger Strategic Projects 27 31.0 201.0 48.6

Smaller Projects, PhD 26 29.9 41.0 9.9

Larger Strategic Projects ENRECA 19 21.8 129.0 31.2

Smaller Projects, Postdoc 12 13.8 33.0 8.0

Smaller Projects, Initiatives 3 3.4 10.0 2.4

Total 87 – 414.0 –

Within this modality project grants have, to a large extent, reflected Danish development policy priorities. The screening of project applications and the selection of projects for funding has been based on three equally weighted criteria: relevance, effect and quality of the proposed development research. FFU has endorsed the applications to the Danish Ministry of Foreign Affairs. Individual capacity development has been integrated into

17 In addition there were 20 projects located in Asia, eight in Latin America and nine which were multi-country/global.

28


the research projects, mainly through funding of PhD students. Thematically, the FFU projects show a great diversity although many of the themes are interrelated, just as several of the research projects are inter- and multi-disciplinary in the sense that they approach development problems in an integrated way and work simultaneously to solve problems. Key thematic areas include:

• Agriculture farming, crops

• Forestry

• Climate change

• Drinking water/water resources

• Food security.

Numerically, the largest share of projects approved by FFU in the period falls within the categories of Larger Strategic and Smaller Projects (PhD), although by funding the Larger Strategic and ENRECA Projects have received the main share (Table 7).

During the evaluation period, Copenhagen University (KU) was the dominant lead institution of FFU projects within agriculture and natural resource management. KU, mainly the Faculty of Life Sciences (KU-LIFE) accounted for two thirds of the projects, Aarhus University for 15%, of the projects, Danish Institute of International Studies (DIIS) for 7% and Geological Survey of Denmark and Greenland (GEUS) for 6%.

2.3 FFU Pilot Research Cooperation Programme (South-driven)

In addition to FFU projects a Pilot Research Cooperation Programme (PRCP) was launched in 2008, with the aim of strengthening research capacity in the three countries based on South-driven research cooperation. Separate initiatives under this modality have historically been referred to as FFU Pilot Projects; however the terminology now being used by Danida is South-driven research projects, which is the terminology used in this report.

The programme has been launched in three countries, Tanzania, Vietnam and Ghana. In Tanzania and Vietnam it was launched in 2008 and Ghana was added in 2011; the programme is still active.

Researchers within the three countries engaged in the modality define the research themes to reflect their national and institutional priorities and select the researchers in Denmark with whom they wished to collaborate. The aim is that research themes will be demand-driven and coherent with national priorities of participating countries, enhancing ownership and relevance.

In operational terms, Southern researchers draw up research concept notes to which Danish researchers are invited to submit an expression of interest. The Southern researcher then selects their Danish partner from amongst those who have responded and the final research project proposal is developed jointly with the selected Danish partner. In other respects, projects funded under the pilot programme must meet the same requirements as other Danish research cooperation projects.

29


A total of 10 South-driven research projects in agriculture and NRM were launched within the evaluation period, five in Tanzania and five in Vietnam. Thematically these projects are focused on climate change, markets (trade and value chain), agriculture, husbandry and land use changes.

2.4 Building Stronger Universities

Since 2011, Danida has provided funding through a new modality, Building Stronger Universities (BSU), where the aim is to develop cooperation between an association of Danish universities, Universities in Denmark, and Southern partners. The overall goal of the BSU initiative is to strengthen universities and research institutions in developing countries through the formation of networks of excellence, by developing and strengthening long-term partnerships between universities in developing countries and in Denmark, strengthening:

• Research capacity

• Research-based education

• Dissemination of research-based information.

The BSU programme is fundamentally concerned with how Danish academic institu-tions can contribute to improving the operations and performance within universities in Southern partner countries. The focus is on the academic quality of their research output, training courses, and how they contribute knowledge and skills to meet the national development challenges in their respective countries.

The BSU modality allocates funds on a biennial basis for four thematic platforms: Growth and Employment; Climate and Environment; Health; Stability, Democracy and Rights18.

It is the intention that the BSU initiative supplements Danish support provided to individual researchers by addressing the needs for capacity development at the level of research institutions. This has been in the form of establishing PhD schools, upgrading PhD training, improved research management and environment and improved research dissemination.

An outline of the two BSU platforms covered by this Evaluation is presented below. The emphasis has been on its set-up and preliminary experiences.

Environment and climate platformThe development objective of the BSU Platform on Environment and Climate (BSUEC) is to contribute to informed and improved sustainable natural resource management and livelihoods considering the needs for environmental protection, clean tech solutions, adaptation and mitigation in the face of climate change – through research capacity building in Ghana and Tanzania.

18 The Growth and Employment and Climate and Environment Platforms are thematically the most relevant within the scope of this Evaluation.

30


Five focus areas have been identified as relevant to BSUEC:

• Sustainable land use and management;

• Sustainable water use and management;

• Energy and waste in rural and urban linkages;

• Climate change;

• Sustainable innovations and planning.

The partners of BSUEC in its initial phase are University of Ghana, Kwame Nkrumah University of Science and Technology, Ghana (KNUST), University of Dar es Salaam, Tanzania (UDSM), and Sokoine University of Agriculture, Tanzania (SUA). Participating Danish universities are Aarhus University (AU), (KU), Aalborg University (AAU), Technical University of Denmark (DTU), University of Southern Denmark (SDU), Roskilde University (RUC) and Copenhagen Business School (CBS). Two Danish sector research institutes are also linked to the platform: Danish Meteorological Institute (DMI) and (GEUS). Platform Working Groups (PWGs) have been established at some specific African universities19.

During the inception phase, several processes were undertaken to develop activity plans, platform governance structure and the detailed budget.

During the first two-year implementation phase, from 2011 to 2013, BSUEC is focusing on establishing the capacity to train PhD students and further foster research networks and collaboration between Danish researchers and the participating African Universities. Activities are being implemented in five work packages.

Growth and employment platformThe development objective of the BSU Platform on Growth and Employment (BSUGE) is to enhance capacity of the African institutions to promote sustainable growth and employment through research, education and dissemination of research findings.

The partners of BSUGE in its initial phase are UG, KNUST, UDSM, SUA and seven Danish universities: KU, AU, SDU, RUC, (AAU), DTU and CBS. Platform working groups have been established at UG, KNUST, UDSM and SUA.

During the first two-year implementation phase, from 2011 to 2013, BSUGE has been focusing on PhD education, PhD scholarships, Research collaboration, Dissemination and Platform Governance which are each designated as a specific work package.

2.5 Support for two KU-LIFE Centres

History and origins of the KU-LIFE centresThe Danida-funded centres covered by the Evaluation are the Danish Seed Health Centre and the Danida Forest Seed Centre Danish.

19 There is a PWG at UDSM and one at SUA each with five participants.

31


Established in 1981, the Danish Forest Seed Centre (DFSC) evolved from the Danish FAO Forest Seed Centre. The establishment was in line with Danish priorities in development assistance in relation to forest management and forest conservation and DFSC activities were closely linked to Danida sector support programmes.

DFSC provided research and technical support to developing countries in the fields of procurement and handling of seed of tropical and subtropical tree species, basic tree improvement and conservation of forest gene resources and national centres were established in many of the programme countries, including Tanzania and Burkina Faso.

From 2004 the DFSC was integrated into the newly established Danish Centre for Forest Landscape and Planning under the Royal Veterinary and Agricultural University.

In 2004 the Danish Seed Health Centre for Developing Countries (DSHC) was created by merging the Danish Government Institute of Seed Pathology (DGISP) with development activities of the Department of Plant Biology at the Royal Veterinary and Agricultural University, in order to address the losses to farmers in developing countries caused by seed-borne and seed-transmitted diseases, as well as the danger involved in the movement of disease organisms with seeds within a country or across international boundaries. The overall goal was to increase research capacity and capability of national research institutions in the project countries and of developing close working relationships between seed producers, seed testing laboratories, research institutes and extension services.

In 2007 the Royal Veterinary and Agricultural University was integrated into the University of Copenhagen and became the Faculty of Life Sciences (KU-LIFE).

In 2012 the KU-LIFE was separated into two parts. The veterinary part merged with the Faculty of Pharmaceutical Sciences and the Faculty of Health Sciences into a new Faculty of Health and Medical Sciences. The remaining part of LIFE – under which DSHC and DFSC belong – merged a new larger faculty, entitled Faculty of Science.

Funding for two KU-LIFE centresThe Danida-funded activities at the two KU-LIFE centres covered by the Evaluation have been guided by Performance Contracts (2008 to 2010 and 2011 to 2012) and have taken place primarily, but not exclusively, in Danida partner countries.

The centres have combined many disciplines, worked across sectors and collaborated with universities in Denmark and abroad, with international institutions, and with institutions, networks and relevant sector ministries in a number of countries in Africa and Asia.

In the Performance Contract from 2008 to 2010 the centre activities were grouped under the following objectives:

• Generating new knowledge and developing new methodologies, approaches and models (Applied Research)

• Building capacity for research, use of research and interventions (Capacity Building)

• Promotion of the use of research based knowledge for development (Knowledge Management).

32


2.6 Research Networks

Since the mid-1990s, Danida has funded a number of research networks. The members of these networks have been researchers and aid practitioners in Denmark and developing countries. The aim of the networks has been to enhance the utilisation of development-oriented research in development assistance, focusing on the Danish resource base within research for development. This aim has been accomplished by increasing the dialogue and interaction between the research community and aid practitioners in Denmark and abroad.

In April 2006 Danida circulated a discussion paper presenting some thoughts on the future of the research networks. The discussion paper was particularly focused on possibilities for improvement by merging the networks. Following this, the Network for Agricultural Research for Development (NETARD), the Research Network for Environment and Development (ReNED) and the Research Network for Governance, Economic Policy and Public Administration (GEPPA) decided to merge and became the Danish Development Research Network (DDRN). The Danish Research Network for International Health20, Danish Water Forum (DWF) and the Poultry Network continued as separate entities although Danida’s support for the Poultry Network ended in 2007.

The main function of the networks has been to help communicate research findings, as well as making the capacity of the Danish resource base known to potential users within development assistance, although the focus of activities has differed across the networks.

The Evaluation has considered the DWF and DDRN as the key networks receiving support from 2006 to 2011 since the Poultry Network was winding down at the start of this period. DDRN comprised the three main networks which merged and DWF also received funding and was active during the period (see below).

Danish Water ForumSince 2004 the Danish Water Forum has administered the Knowledge Network for Water and Development with funding from Danida. The Knowledge Network was established at the end of 2003 to enhance the exchange of knowledge in the water sector, create synergy and support the technical quality in Danish development aid, in particular in relation to Danida-funded water sector programmes as well as in sector programmes where water was an issue, including agriculture, environment, and health.

The first phase, from 2004 to 2006, focused on establishment of the network, knowledge sharing in Denmark and extraction of international knowledge for the Danish resource base. These activities were also of relevance for the second phase of support from 2007 to 2009 but with enhanced focus on facilitation of research cooperation between researchers from Denmark and partner countries. Since 2007 the network has aimed at fostering collaboration and knowledge sharing between the Danish researchers and practitioners, and between researchers and practitioners from North and South. This has been in the form of enhanced communication, joint projects, working groups and networks, and facilitation of the integration of research-based knowledge in the water sector in development assistance in Denmark.

20 This network is not covered by the Evaluation.

33


Danish Development Research Network The DDRN’s overall objective was to increase the contribution and inclusion of research and knowledge in the design and implementation of development assistance in response to partner countries’ needs. More specifically to contribute to the dissemination and exchange of information between development programmes and the research community within agriculture, environment and governance and to foster an engaged and committed network of members through enhanced community building, interaction and collaboration, and through establishment of the necessary information channels, mechanisms and tools21.

2.7 Minor Studies

The Danish support to development research includes an allocation for minor research-based special studies. A study in this context is considered as a smaller research-based activity carried out by Danish or foreign researchers or consultants, typically linked to current development policy issues. The overall responsibility for the account and the authority to approve applications for study grants lies with the head of UFT. The research studies are normally prepared to elucidate special concerns raised by embassies and departments with a view to strengthening the quality of Danish development cooperation.

21 DDRN Intermediate Objectives. See Producing knowledge for development together – lessons from the Danish Development Research Network. DDRN, July 2011.

34

3 Methodology and Approach

3.1 Analytical Framework

Overall analytical frameworkIn the context of the OECD/DAC evaluation criteria22 the analysis considered two broad aspects of Danida support:

• the mechanisms and processes of providing support

• the products and outcomes of successful delivery

The DAC criteria are now fairly standard across evaluation work and ensured compliance with international best practice. Data collection and analysis was based on a mixed-methods approach, combining quantitative data analysis with qualitative methods (see below).

Quantitative methods measured products23 and qualitative methods were used to explore issues of process, creating a better understanding of possible changes by which a particular intervention triggered a series of events that ultimately resulted in an observable changes or outcomes24.

Between 2006 and 2011, support to the activities of the Centres and Networks gradually reduced as these were merged and absorbed into different structures, however the links and output they created continued to support and synergise other areas of support.

By 2011 the FFU projects and BSU had become the key focus for Danida, with some activity under Minor Studies. The position of this support which aimed to strengthen capacity and produce research output at organisations in target countries, which then fed into a general strengthening of National Agricultural Research Systems (NARS), is shown in Figure 2 as part of a theory of change diagram.

This broad and simplified theory of change25 links the support to capacity strengthening and research provided by the five modalities being evaluated to the wider issue of creating stronger NARS. It has been articulated here by the Evaluation to reflect the policy and focus of development support during the period under consideration.

The definition of a NARS is extensive and inclusive, involving universities, national agricultural research institutes, extension services, private sector including NGO and processers and civil society. It is also important to include, within the concept of a NARS, the enabling policy environment in which the system operates and the research

22 Relevance, Effectiveness, Efficiency, Sustainability and Impact.23 Products such as the number of PhDs completed or the number of peer-reviewed papers published.24 Outcomes such as the level and quality of new grants obtained by newly qualified researchers for

research addressing demand-driven needs or the number and quality of fresh initiatives building on published research findings.

25 Figure 2 is illustrative and simplified, but is included to demonstrate the role that Danida support to agriculture and NRM research has in the wider context of development, and economic growth, and the part it plays in contributing to the strengthening of NARS.

35


which is linked to it including for example, research on the development of appropriate land rights policy and issues linked to cross-border and internal trade and markets.

The creation of stronger NARS underpins much of the approach in Africa which falls under the Comprehensive African Agricultural Development Programme (CAADP)26, supported by the Framework for African Agricultural Productivity (FAAP)27, which aims to use agricultural growth as the engine to provide economic growth and poverty reduction, and the same principle applies elsewhere.

Figure 2 Broad Theory of Change for Danida Support to Agriculture and NRM Research

Danida Support

Danida Sector Programmes

Non-agriculture and NRM-based contributions

from other sources

Capacity StrengtheningAgricultural & NRM Research

Stronger HR base

Economic Growth

Agriculturally-based Growth

StrongerNARS

Othercontributions

More productive and competitiveagriculture and markets

Reduction in Poverty

International publications

Dual aspects of the Evaluation – history and future perspectiveThere were two broad aspects to the Evaluation. The first was to evaluate the historical performance of the activities being funded, which was a conventional look at perfor-mance, based on the DAC criteria. Measuring and assessing that the extent interventions were relevant in the context of the contemporary strategies and priorities, how effective they were in delivering the required results, how efficient was the use of resources and to what extent were the desired outcomes and impact achieved and potentially sustained?

The second was to take the evaluation information and analyses and use them to look forward and consider the lessons learned from past experience and how the best use could be made in formulating a new strategy for Danida support to development research designed around a new and innovative approach.

An Evaluation Matrix (see Annex B) developed during the inception phase from the 24 questions posed by the ToR, identified key issues to be addressed.

26 Comprehensive Africa Agriculture Development Programme. NEPAD Midrand, South Africa. August 2002.

27 Framework for African Agricultural Productivity. June 2006. Forum for Agricultural Research in Africa, Secretariat, Accra, Ghana.

36


Data collection instruments and analysisSeveral different tools and modalities were used during the Evaluation; these are briefly outlined below.

Document and literature reviewA large number of documents were reviewed and discussed including policy, strategy, review and project reports. The key reports are listed at Annex C, and are referred to in footnotes in this report, where particularly relevant. The review of documentation allowed, inter alia, an analysis of the relevance of design, the efficiency with which resources have been used to create the necessary and sufficient products for delivery and the extent to which results have been delivered and objectives achieved. Prior to country visits (see below) the review facilitated the development of project fact sheets.

Online survey – eSurvey An eSurvey was conducted using proprietary software designed for such work28. The aim was to provide data for triangulating information collected from other sources as well as increase ownership of the evaluation process by engaging with a wide cohort of stakeholders. It was not intended as a major data-collecting instrument.

The focus of the eSurvey was stakeholders who had been, or who were currently, engaged in FFU North- and South-driven projects29, 324 invitations were sent out by email of which approximately 111 (34%) were from Denmark and 213 (66%) from partner countries. A total of 86 responses (27%) were received, of which 23% were women. The majority of respondents were from universities (74%) with the bulk of the balance being national research institutes (18%) and ministries (4%). The completion rate was slightly higher from Denmark (32%) compared with respondents in partner countries (24%)

Responses to the survey are shown in full at Annex D, and are included, where appro priate, in Chapters 7 and 8; a summary of the responses to the open question on Danida support is given also given at Annex D and have been taken into consideration in the analysis and development of recommendations.

Overall the eSurvey proved less useful than had been anticipated, see below, although it broadly confirmed information from other sources.

Key informant interviews This tool used a semi-structured approach with a checklist, based on the Evaluation Matrix, to guide the process. It provided one of the major data collection methods.

Key Informants were the project coordinators, country-based project leaders and contact and management level stakeholders associated with the FFU and network/BSU/KU-LIFE funding mechanisms. Interviews were conducted in person (Denmark, Burkina Faso and Tanzania), and via video and telephone links where this was not possible. In addition to these key informants, departmental and institution heads and directors not directly engaged in activities, were interviewed to gain their perspective of Danida support in the broader picture of development partner activities and priorities.

28 SurveyMonkey – see www.surveymonkey.com for further description and overview of the tool.29 North-driven projects were those identified and led by Danish organisations, the South-driven projects

were those under the Pilot Research Cooperation Programme, and were formerly referred to as Pilot Projects.

37


Table 8 shows a summary of the numbers of individuals interviewed and with whom the Evaluation interacted.

A table summarising the selected projects is given at Annex E.

A total of 41 FFU projects (out of 83) and all six South-driven projects were engaged with directly. In addition the views and responses from 28 participants at a one day Emerging Issues Workshop in Copenhagen (see Annex G) provided input into the Evaluation. Many of those interviewed had interests in different activities funded by Danida, and this was especially true for the BSU initiative where the 18 people shown in Table 8 refer only to those directly engaged in Platform management. Overall the Evaluation took into consideration input from a diverse selection of stakeholders.

Table 8 Numbers of stakeholders interviewed during the Evaluation

Category/Country Burkina Faso Denmark Tanzania Total

FFU Projects 19 32 22 73

FFU Southern-driven Projects n/a 6 5 11

Centres – Management position 2 3 1 6

Networks Management n/a 5 n/a 5

BSU EC and GE – Management n/a 7 11 18

Danish Fellowship Centre n/a 3 n/a 3

Danida-Danish embassy 1 6 3 10

Donors/Other embassy 2 n/a 3 5

Senior Management 4 1 6 11

Total 28 63 50 142

SWOT analysis workshop In addition to interviews and site visits, SWOT Analysis Workshops in which stake-holders analysed the strengths, weaknesses, opportunities and threats of Danida support were held in Burkina Faso (15 participants), Tanzania (14) and Denmark (10). The workshops provided opportunities for informal discussions and interaction between a broad-base of stakeholders who had been through similar experiences and processes. A summary of the recommendations made by workshop participants is given at Annex F.

In order to assess potential or actual impact at the level of end-users or beneficiaries field visits were made to two project sites in Burkina Faso and two in Tanzania.

Emerging issues workshopAfter field visits and interviews had been completed, a Workshop was held in Copen-hagen to share emerging issues with key stakeholders. The objective was to sensitise both Danish and Southern partners to some of the main conclusions of the Evaluation and to seek their responses and comments. The report of the workshop is at Annex G, and findings were incorporated into this report, where appropriate.

38


Bibliometric analysisAn assessment of FFU Project Completion Reports indicated that inconsistent reporting on publications, and the sometimes long delays in research being accepted for publication, made such an analysis unrealistic. However, the Evaluation does present some information on publications as a potential indicator of the academic quality of research.

Triangulation of findingsThe core tools for collecting data were the Key Informant Interviews, SWOT Analyses and Literature and Document Review. The Site Visits and eSurvey provided sources of additional information and data feeding into the overall evaluation and providing triangulation of the other findings.

Interplay between in-depth and overall portfolio analysisThe need to understand and document the big picture performance of Danida support to research in agriculture and natural resource management has been key to the Evaluation; to provide the evidence to guide and support the development of a fresh strategic approach is a key objective.

To this end the tools described in this section have been used on projects and initiatives selected using carefully developed criteria for in-depth analysis. Coverage was triangulated through the use of the multiple mechanisms described above. Two key instruments in developing the overall picture have been interaction and feedback with EVAL, UFT, the new strategy development team, Reference Group Committee and DFC as the Evaluation progressed, and a workshop on Emerging Issues (see Annex G) in the latter stages of the process.

3.2 Field Visits to Burkina Faso and Tanzania

The programmes for the country visits are shown at Annex H. Interactions with stakeholders took place through meetings with individuals and groups engaged with work supported by Danida, the SWOT Analysis Workshops and site visits. It should be noted that the larger number of North-driven FFU projects in Tanzania is reflected in the reporting on findings in Chapter 7 and the absence of South-driven FFU projects in Burkina Faso is the reason for the emphasis on Tanzania in Chapter 8.

39


3.3 Limitations and Challenges

The Evaluation has been limited to a fixed time period of 2006 to 2011 and five different funding mechanisms. Given the time available for the Evaluation, the number of funding instruments, their geographic coverage and the six-year time span it was not possible to make a comprehensive, detailed evaluation of all activities, and a compromise had to be established which balanced detail of coverage with the scale of the study.

To this end the focus has been on stakeholders in Denmark, Tanzania and Burkina Faso. Danish stakeholders were able to provide their perspectives of activities based in the non-visit countries (as well as Tanzania and Burkina Faso) but it was not possible to interview stakeholders in these other countries due to time and travel constraints. Every effort has been made to provide as much detail as possible and caveats are placed on conclusions where it is deemed appropriate.

The assessment of the BSU modality was limited to two of the four Platforms, and does not necessarily reflect the status or condition of the others. Recommendations and conclusions specifically address constraints and issues facing the Environment and Climate and Growth and Employment Platforms in Tanzania and Ghana. Only Tanzanian stakeholders were interviewed about these two platforms in-country although most of the Danish interviewees had had experience of both countries.

The Evaluation has also been limited to support for agriculture and natural resource management and the detailed assessment of the South-driven research projects under PRCP is limited to the five projects in Tanzania.

The lack of a strategic framework or logframe/results framework for support to development research, and for the individual modalities, has created difficulties during the Evaluation. Without a clear statement of what the support between 2006 and 2011 was intended to deliver and achieve both overall and within each of the modalities and the absence of indicators that would demonstrate success, assessments of effectiveness have been difficult. Where indicators have been identified they have been limited to measures of product such as the number of degrees awarded or meetings held. There are few, if any baseline figures or data sets on which to base judgements of improvement.

The stakeholders, duration, size, location and institutional homes of the North- and South-driven FFU projects created a highly variable cohort of potential respondents for the eSurvey. This created difficulties in articulating simple and easily understood questions that could be answered clearly and quickly. This has limited its usefulness and it has only been possible to draw broad conclusions from responses, however nothing in the responses contradicts information collected using other methods.

Despite the limitations and challenges, the Evaluation is confident that the findings and recommendations described in this report fairly reflect the views and data made available and represent an accurate assessment of the period and modalities being evaluated.

40

4 Centres

4.1 Introduction

The two centres covered by the Evaluation, Danish Forest Seed Centre (DFSC) and Danish Seed Health Centre for Developing Countries (DSHC), have a long history and are now part of the University of Copenhagen (see Chapter 2). This chapter documents the finding of the Evaluation as it relates to them and draws conclusions on their performance and contribution to development.

4.2 Relevance

Both centres were established by Danida as part of Danish development assistance to provide research and technical support to developing countries in their respective fields; namely forests, trees and land-use planning (DFSC) and seed health and seed pathology (DSHC).

When they were created the two centres’ activities were coherent with Danida’s policies and strategies, aiming to contribute to poverty alleviation and sustainable development, and bridging the gap between research and development cooperation within the centres’ fields of expertise. The centres also played an important role as service providers to Danida Sector Programme Support and cooperated closely with Danida (UFT).

Through their focus on capacity building and knowledge management, the centres provided an important and internationally recognized contribution to knowledge generation, capacity building and the use of research-based knowledge for development within their respective fields. The centres functioned more as development assistance sources, providing educators and an applied research approach, than as research institutions.

Subsequently however, several factors reduced the relevance of their approach including changing Danida policies30, changes in the Danish research environment that demanded greater competition31 and a lack of requests from Danida and Sector Programme Support for their services, as a consequence the support to the centres was phased out from 2011 onwards, as they became integrated into KU-LIFE.

30 Moving from a project to a programme approach, as it is also the case with the BSU, as well as broader policy issues in development including the Paris Declaration on Aid Effectiveness.

31 A new political agreement in 2008 changed the way universities were covered for incidental expenses, which from 2009 and were treated as an overhead contribution of 35% when grants were obtained from research councils. This rate replaced the previous overhead that was a general rate of 20% plus building overheads of 12.65%. The overhead contribution from 2009 applied to research councils, the Danish National Research Foundation, the Danish National Advanced Technology Foundation, future grants from the globalisation pool and the ministries’ research funding that is subject to competition. To finance this redistribution, cuts were made in the universities’ capital subsidy, basic grants and reorganisation reserve.

41

4 Centres

4.3 Effectiveness

The main outputs and activities of the centres, under the performance contracts, have been achieved and many positive results have been reached in regards to capacity building and knowledge management. Capacity building includes research capacity, development of the ability to respond to demand (empowerment), monitoring and evaluation, and capacity for the use of research for development. A summary of accessible data on PhD and publications is presented in Table 9. These data are incomplete, and it is a reflection of the lack of an institutionalised database for output that these figures are not readily available.

The first support to national centres started in the 1990 and this long-term commitment has created strong institutional and personal links and helped build research capacity, in the form PhD, MSc and lower-level training, as well as institutional capacity in the form of infrastructure support and publications.

There is evidence that collaborating research institutions have been strengthened and that the strong personal links with Danish researchers established as a result of the long-term commitment, have been the driving force behind the successful establishment of research projects (including FFU projects). This has made collaborating research institutions stronger when competing for international financial resources, and thus helped make them financial independent.

Table 9 Partial summary of PhD, MSc and publication records for DFSC and SHC

Description Performance 2006 2007 2008 2009 2010

Danish Seed Health Centre

PhD Target 7 9 11 1 0

Actual 9 13 9 4 0

MSc Target 0 5 7 7 5

Actual 2 5 3 7 4

Articles/abstracts published (Only reported as output from 2009 onwards)

Target n/a n/a n/a 7 6

Actual n/a n/a n/a 12 16

Danish Forest Seed Centre

PhD* Target n/a n/a 4 4 4

Actual 2 4 4 4 2

MSc** Target n/a n/a 12 12 11

Actual n/a n/a 14 12 12

Articles/abstracts published Target n/a n/a 9 7 7

Actual 2 4 9 17 15

* No targets were set for PhD in 2006 and 2007. ** Annual reports do not report on numbers of MSc.

42

4 Centres

4.4 Efficiency

The centre modality had a narrow, specialised and technical focus which ensured that all resources used in providing support were carefully targeted. With management costs of approximately 16%, this was a relatively low-cost modality; however, the funding allocation was non-competitive, relying on self-reporting of performance against targets for disbursement from Danida.

The long period of support to the centres (more than 40 years) has provided Danida with a unique database32. Geographically the focus of support was broad; the DFSC has been involved in tree seed projects and helped establish national forest seed centres in some 20 countries worldwide33 and the DSHC has been involved in setting up seed pathology laboratories in 17 countries34. This does not appear to have reduced efficiency in terms of supporting and establishing national centres in targeted countries.

4.5 Impact/Sustainability

The performance indicators are based on outputs instead of outcomes which make it difficult to measure impact. There is limited documentary evidence for impact or outcomes on policy or of any strategic influence; however this does not mean that there have been none. Reforestation policy in Thailand and participatory forest management initiatives in Tanzania suggest some influence from this Danida support.

The centres have, through long-term investments and commitments, provided a platform for capacity building which has made the collaborating research institutions stronger when competing for international financial resources, and thus helped make them financially independent and viable. The core funding through the modality also made it possible to test ideas on a small scale, which later developed into FFU projects. Strengthened capacity has also empowered centres to make contributions to strategy and policy development and human and institutional capacity strengthening in a number of countries.

The long-term collaboration, and the creation of strong personal links, has made collaborating researchers better able to compete for international funds and helped make them financial independent.

The sustainability/viability of the national centres varies considerably. Most centres have been integrated into universities or national research institutes, although this has not always happened. For instance the Seed Health Centres in India and Burkina Faso are fully operational and well integrated into research institutions but the future of the Seed Health Centre in Tanzania is more uncertain, with a reduction in both staff and funding.

32 At present the DSHChasa database with 48,000 seed samples collected over a 40-years period.33 Centres established in Nicaragua, Costa Rica (CATIE), Sudan, Burkina Faso, Eritrea, Ethiopia,

Uganda, Tanzania, Kenya (ICRAF), Nepal, Laos, Vietnam, Cambodia, Thailand and Indonesia. 34 Bangladesh, Burkina Faso, China, Egypt, Ghana, India, Indonesia, Ivory Coast, Kenya, Nepal,

Nigeria, Pakistan, Philippines, Tanzania, Uganda, Vietnam and Zimbabwe.

43

4 Centres

Most National Forest Seed Centres in Asia, Africa and Latin America (for example in Burkina Faso) are still functioning and there are examples (Nicaragua) where centres have been able to continue as private service providers. At least one national centre (Burkina Faso) has the potential for becoming a regional centre of excellence under World Bank funding and the West African Agricultural Productivity Programme.

4.6 Conclusions – Centres

The Centre approach relied on technical support to develop physical as well as intel lectual capacity. Infrastructure and equipment were provided and the technical skills to utilise it were developed. The modality has, through long-term investments and commitments, provided a strong platform for capacity building which has made the collaborating research institutions stronger when competing for funds. As a mechanism for capacity strengthening in focussed areas the centres were highly effective.

Partnerships between institutions were established which created an enabling environment for the development of individual skills and collaborative links continue to exist between individual researchers in the South and Denmark that enable research initiatives to be developed and implemented.

The whole approach was driven by the skills and interests of Danish centres, and created institutions which were not always absorbed into national systems or able to maintain investment levels, although this is variable.

With the change in Danida’s, and wider, development policy reflecting a more southern and demand-driven agenda35and shifts in Danish research policy, the non-competitive funding of standalone activities and creation of infrastructure, such as the centres, is no longer an appropriate mechanism for supporting Southern research capacity. Where institutional strengthening is necessary and appropriate, the approach now is for it to be South-driven and for Danish professionals and support to respond to demand from stakeholders.

The crucial difference between this modality and that of newer approaches such as BSU and the South-driven FFU projects is that, in theory at least, the demand and process is driven by Southern institutions who seek specific skills and inputs to address their own institutional priorities.

35 The Paris Declaration plus the recognition of the importance of sustainable development in terms of human development in which resource use aims to meet human needs while ensuring the sustainability of natural systems and the environment, so that these needs can be met not only in the present, but also for generations to come. Sustainable development is defined as development: that meets the needs of the present without compromising the ability of future generations to meet their own needs.

44

5 Networks

5.1 Introduction

The networks supported by Danida from 2006 to 2011 have variously evolved, merged and disbanded; this chapter documents the key findings and conclusions relating to these processes. Further details of the background to the networks are given in Chapter 2.

5.2 Relevance

The merging, discontinuation and evolution of the networks36, which started in 2007 was driven by a need, recognised by stakeholders, for a more relevant and efficient approach to networking and interaction. This level of demand for the networks is a useful, positive proxy indicator of their relevance.

Since its formation, DDRN has provided a relevant and useful forum for research information and communication, largely responding to the needs of stakeholders and achieving most of its stated objectives37.

DDRN has functioned as a forum in which Southern countries could articulate research demands and the broad membership of DDRN consisting of all types of stakeholders (government institutions, NGO, researchers, development practitioners, private com panies) has made it an important and highly appreciated mechanism for sharing and exchanging of research related information.

However, the broad focus of DDRN has also made it difficult to maintain the original idea behind the three networks, which was to facilitate research-based knowledge inputs into Danish development co-operation, and specifically to ensure that research findings were utilised in Danida programming and national sector programmes, this has reduced its relevance.

The DWF, which maintained its independence when DDRN was formed, continued as a relevant forum for networking among water sector stakeholders in North and South. In contrast to DDRN, the DWF has a narrower scope and focus within a more defined sector, facilitating a closer link between development research and business development.

After Danida funding to DWF was phased out, there has been a gradual shift in its focus in response to the demands of its members. In doing so DWF has become a network more of relevance to supporting and promoting the export of Danish water sector know-how and products to Asian countries, than to supporting development research within the water sector in Africa.

36 Network for Agricultural Research for Development (NETARD), Research Network for Environ-ment and Development (ReNED), Research Network for Governance, Economic Policy and Public Administration (GEPPA), Danish Research Network for International Health, Danish Water Forum (DWF); Network for Smallholder Poultry Development (NPSD).

37 Intermediate Objectives. See Producing knowledge for development together – lessons from the Danish Development research Network. DDRN, July 2011, pp. 62.

45

5 Networks

5.3 Effectiveness

Based on the review of the network contracts and work plans combined with personal interviews with key stakeholders from the networks, the Evaluation finds that the networks to a large extent have produced what was identified in work plans. The information provided by DDRN for example, illustrates this (see Annex I for a summary), however the targets and expectations of the work plans in terms of deliveries have not been high. Most targets were associated with events and activities and the establishment of groups and links. Although these deliverables have been beneficial for strengthening networking relations and sharing of information, there has been no real attempt made by the networks to assess the effectiveness of the channels for uptake of the research information produced by, and through, them.

In merging the networks and broadening its technical base, DDRN became much more of an information hub and was well-placed to promote multi-stakeholder dialogue, meetings and networking. Development-based research projects have been successfully generated out of the links established through the network platforms. This is particular true for a number of FFU North-driven projects, where there are several examples of partnerships and collaboration that developed from network activities (workshops, seminars) and continued into lasting collaborative activities and projects. However, in becoming an information hub it became less effective as a focussed technical platform of skilled specialists in a position to provide targeted advice.

The DDRN has experienced a steady increase in its member base over time and by the end of 2011 it had more than 2,000 members, most of them from the South.

Less effective has been DDRN’s role in promoting the uptake of research into Danida’s development programmes and policy. There has not been, and there still is not, any formal mechanism to facilitate the use of research outputs in the development of Danida programmes, because whilst UFT has the mandate for this role it has limited influence over embassies which have the responsibility for programme development. There is the additional issue of also needing a mechanism that ensures opportunities exist for the uptake of Danida supported research output, where appropriate, for incorporation into national sector programmes.

The process of decentralisation of Danish development assistance over the past decade has made it even more difficult for the networks to reach out and influence sector programme formulation processes in the South.

5.4 Efficiency

The original three networks (NETARD, ReNED and GEPPA) had a number of duplicating functions and overlap and many stakeholders were members of all three networks. These inefficiencies led finally to the decision to merge into DDRN. The merging, however, produced a much broader scope and the loss of ability to provide focussed support to policy making, as discussed above.

During the design of the BSU, there were a number of discussions and consultations with the networks to see whether some of the functions, skills and contacts developed through DDRN activities could fit into the BSU context, and it was the original intention that the communication and networking skills of DDRN would be put

46

5 Networks

to use in the BSU. However, it was not possible to agree a mechanism for this while DDRN was still operating, and communication remains something of an issue across all the modalities being evaluated (see Chapter 11).


Despite being able to pay for a part-time administrative assistant, the DDRN has after 2011 not been able to leverage funding to maintain their Secretariat. This has resulted in a sharp decrease in the interactive activities established at the website and in the type of level of events (workshops, seminars etc.) that DDRN could contribute to. Likewise, DDRN’s ability to serve as an effective portal for information exchange has been much reduced.

The DWF has continued, mainly based on a fee (DKK 5,000) paid by its 5038 members, as well as by funding from a few other projects and sources. The membership level has been maintained at 2011 levels. However, the focus of DWF has gradually shifted away from development research within an African context, to more commercially oriented focus on export opportunities in growth countries in Asia. DWF has, through securing of other external funds, been able to keep some administrative functions and organising of thematic meetings, albeit at a lower activity level than in the period up to 2011.

General assembles and regular board meetings are still being held by both DDRN and DWF, similarly newsletters are still being produced by both networks, although at a lower frequency and with less substantial content than up to 2011.

With limited DDRN funding, some of the links with Southern partners (such as RUFORUM39) have been lost, removing or severely limiting the opportunities for Danish stakeholder representation where it might be of value, although there is little value in links without supporting action and resources. Some limited functions have been taken over with funds from Sweden, Norway and the EU.

5.6 Conclusions – Networks

The networks established relevant platforms for multi-stakeholder dialogue, networking and exchange of research information across, and between, South and North that were effective in linking groups and individuals with common research interests.

Effective mechanisms were never established for the networks to inform the Danida-funded development programme formulation processes. Loss of focus within some networks, and changes in aid delivery mechanisms, further challenged the networks potential for influencing policy.

Opportunities for utilising DDRN/DWF capacity and skills in research communication and in the dissemination of information and promotion of multi-stakeholder dialogue has not been utilised by the emerging BSU funding modality, although this has been identified as a constraint in the BSU.

38 Mainly Danish private sector companies in the water sector.39 The Regional Universities Forum for Capacity Building in Agriculture (RUFORUM)

is a consortium of 30 universities in Eastern, Central and Southern Africa, established in 2004.

47

6 Minor Studies

6.1 Introduction

Minor studies funding is to support initiatives which contribute to Danida’s strategic development objective that aims at strengthening the quality of Danish development cooperation and provides, or should provide, guidance and input into strategy development and planning. It is an instrument to promote internal learning, influence policy and strategic thinking and to encourage innovation. As such it is distinct from the other modalities being evaluated as it supports Danida policy development rather than directly supporting Southern institutions or individuals.

It is a flexible instrument, which can be applied quickly, within an annual planning cycle, and focuses on relevant and often innovative issues within development assistance. It is an instrument for quick learning and uptake of knowledge, especially with respect to priority, topical issues within development assistance. Operationally, studies under this modality tend to re-analyse and build on known information, compiling and consolidating existing knowledge and placing it into a specific policy context.

There is an annual process within Danida to allocate funding for these studies which are initiated by representatives and embassies. Funding also follows a structured approach to communication which includes a written report and a pro-active approach to com munication with at least one meeting where results are presented. As well as internal communication, where the results and outcomes of the study justify it, wider international sharing of findings is encouraged.

The modality is dependent on evidence from existing information sources which relies on proper data collection within aid programmes, well-performed and structured reviews and state-of-the-art evaluations. This may be a limitation where data or information are inadequate, not available or inaccessible.

Four studies relevant to the Evaluation have been identified by UFT, and have been completed during the period40; the total cost of these was DKK 2.1 million.

6.2 Relevance

Each of the four studies was clearly linked to Danida policy and the priorities identified in two key strategic documents41. The studies addressed:

40 Low Carbon Development and Poverty Alleviation – Options for Development Cooperation in Energy, Agriculture and Forestry. Mikkel Funder, Jacob Fjalland, Helle Munk Ravnborg and Henrik Egelyng, DIIS Report 2009:20; PREMECA, Green Growth in Practice under Danida’s Regional Envi-ronmental Programme in Central America. Jakob Grosen, Development Associates ApS, March 2012; Addressing Climate Change and Conflict – experiences from natural resource management. Mikkel Funder, Signe Marie Cold-Ravnkilde and Ida Peters Ginsborg – in collaboration with Nanna Callis-en Bang, DIIS REPORT 2012:04; Land Tenure under Transition – Tenure Security, Land Institutions and Economic Activity in Uganda, Helle Munk Ravnborg, Bernard Bashaasha, Rasmus Hundsbæk Pedersen, Rachel Spichiger and Alice Turinawe, DIIS Working Paper 2013:03, March 2013.

41 Climate change is a priority area in both Commitment to Development and Freedom from Poverty – Freedom to Change).

48

6 Minor Studies

• Important aspects of climate change in Africa;

• The multiplier effect climate change issues can have on conflict when they become part of a complex and volatile situation and secondly the important part climate change can play in exacerbating poverty;

• Documenting the lessons learned from supporting green growth42 through the implementation of eco-enterprises in Guatemala and Honduras;

• The mechanisms by which land and property rights stimulate and sustain eco-nomic activity of individuals and businesses by assess the impact of different issues Land and property rights, tenure security and economic behaviour in Uganda.

6.3 Effectiveness

A study carried out through the regional environmental programme in Central America (“Programa Regional de Medio Ambiente en Centroamérica (PREMACA)”) is a good example of how results from a Danida supported programme are put into a new context, in this case that of green growth, and the study was used to shape the approach and defini-tion of the green growth paradigm. The PREMACA study increased existing knowledge on green value chains producing a short and results-oriented study demonstrating how green growth and greener value chains could be successfully pursued in development programmes, especially within agriculture and private sector support. Effectiveness of the study was increased through well-designed communication and it is often cited as a case study of how positive results can be achieved within a greener growth paradigm.

This is also the case for the three other studies, although effectiveness is on a more general level. The relatively fast process of this funding modality made output from them quickly available and permitted the information to feed into important policy processes within Danida, with implications for strategic thinking around low carbon development and climate change.

The Climate Change and Conflict study output was particularly timely and served to highlight climate change as an important factor that is likely to exacerbate existing conflicts in regions where Danish development aid and humanitarian interventions operates, such as the Horn of Africa. The considerations for the Climate Change – Conflict nexus are now increasingly embedded in bilateral interventions, for example in Northern Kenya and Somalia, and in regional activities on the Horn of Africa. Although the study was not the first to highlight the importance of this linkage it certainly raised awareness within Danida and significantly contributed to bringing the issue on the agenda and further into consideration in aid programmes.

42 Green growth should be understood as an integral part of sustainable growth which promotes general economic growth and development in a manner that enables the environment today and in future to deliver the resources and environmental services on which our welfare depends. Green growth should catalyse investments, innovation and job creation, which not only sustain contin-ued growth, but also give rise to new economic opportunities. According to the United Nations Environment Programme (UNEP), a green economy is one which is low carbon, resource efficient and socially inclusive. (The Right to a Better Life – Strategy for Denmark’s Development Cooperation, Danish Government, June 2012).

49

6 Minor Studies

The Land and Property Rights study provided a more recent insight into the influence that policy in these areas has on economic growth driven through agriculture and natural resource management, providing useful material through three publications43.

6.4 Efficiency

In terms of the quality of the outputs and the uses to which they were put, the total cost of the four studies (DKK 2.1 million), appears to represent extremely good value for money.


From the available evidence these studies all appear to have made an impact on Danida policy and raised awareness amongst the organisation’s decision-makers, which is what they were designed to do. One in particular (PREMACA) provided clearly articulated information which helped drive and develop focussed, evidence-based policy. The others provided more general information with less direct benefit, but did serve to consolidate information and raise awareness.

6.6 Conclusions

It is important that Danida policy and decision-makers have access to reliable and evidence-based information and the Minor Studies modality provides this opportunity. In general the Evaluation has found only limited institutionalised mechanisms for com-munication and information, and this modality provides one option for consolidating and sharing relevant and focussed knowledge; as such this is a positive characteristic.

Historically this role was partly held by the now phased-out technical networks, but in their absence the ability for embassies and representatives to commission independent consolidation of evidence in support of policy/strategy decisions is an important one.

The modality, for all its potential usefulness appears underutilised, only four studies were commissioned for agriculture and NRM in the six years covered by the Evaluation, and one of these on Policy and land rights was at the very end of the six-year period (August 2011). The reasons are linked to the fact that studies need to be demand-driven through Danida’s system and there is insufficient capacity to use and manage them.

Whilst there is an unwillingness to utilise the funding instrument for the type of studies for which it is designed, there is clearly a need for the facility to exist and evidence of clear, cost-effective benefits from its use. It is possible that the new overall strategy for development research, which is under development, will identify priority issues that can provide an easy entry-point for those with access to the fund in the embassies and Danida in Copenhagen.

43 Land tenure security and development in Uganda, DIIS Policy Brief, April 2013, 4pp. Land Tenure under Transition – Tenure Security, Land Institutions and Economic Activity in Uganda, Helle Munk Ravnborg, Bernard Bashaasha, Rasmus Hundsbæk Pedersen, Rachel Spichiger and Alice Turinawe, DIIS Working Paper 2013:03, March 2013.

Land Tenure and Economic Activities in Uganda: a Literature Review, Rasmus Hundsbæk Pedersen, Rachel Spichiger, Sarah Alobo and Michael Kidoido, with the collaboration of Bernard Bashaasha and Helle Munk Ravnborg, DIIS Working Paper 2012:13, November 2012.

50

7 FFU Research Grant Projects (North-driven)

7.1 Introduction and Background

Calls for proposalsThe earlier calls, from 2006 to 2008, for research proposals identified capacity strengthening projects (ENRECA) as well as research project grants and initiative grants in Denmark and developing countries. The ENRECA grants were abolished in 2009 and the Initiative Grant in 2010; from that point, the focus became research grants for development research, although development research is not defined.

The FFU is an ad hoc Committee under the Strategic Research Council, which is appointed to ensure the strategic use of the funds designated for development research, in which importance is attached not only to technical quality but also to the relevance of the research in the context of development assistance.

The scope of those eligible to apply remains consistent as …an organisation, such as a governmental institution, business enterprise or private organisation in Denmark and a main applicant attached to the Danish organisation, although the wording changes slightly. Despite this the principal recipients of grants have been from universities (86%) with the remainder going to research institutes44.

Research themes and focus From 2006 to 2008 the calls state that research should generate knowledge for the pro motion of Danish development assistance in line with strategic planning documents45 with an overall objective of combating poverty. In 2006, the call invites applications for research or for building research capacity in developing countries with topics covering the development and role of the private sector, children and young people and market-based agricultural production. In 2007 and 2008 calls focus on Danish programme countries and research in fields in which research and new knowledge relevant to Denmark’s development assistance may contribute to solving the problems of developing countries. From 2009 onwards the objective is refined to generate knowledge to promote the overall objective of the Danish development assistance to reduce poverty and to support research in fields in which research and new knowledge that is relevant to Denmark’s development assistance and may contribute to solving the problems of developing countries. The importance of projects that contribute to the enhancement of the research capacity in developing countries, and that it is driven by the countries’ own demands and strategies, is stressed. In 2011, support to encouraging sustainable development is also included in the wording.

Research areas and themes are identified in all the calls, in some detail and are summarised in Table 10 below46. The most significant change over the period is the

44 KU (65%), Aarhus University (14%) and Roskilde, South Danish and Danish Technical Universities receiving the balance (7%). The institutes receiving 14% of project funding were DIIS, GEUS, DMU and Risø. See Table 5 in Broegaard for details.

45 Strategy for Denmark’s Development Policy – Partnership 2000 and A World for All – Priorities of the Danish Government for Danish Development Assistance 2008-2012.

46 Note it has not been possible to assign values to this table as the thematic information on the project database provided to the Evaluation is inconsistent with the stated titles of the themes, as advertised.

51


appearance of climate and the environment with more socially linked topics such as fragile states, conflict, employment and rights.

The type of projects which could be supported under the FFU grants has evolved over the period under evaluation, and is summarised in Table 11. The key changes here are the removal of ENRECA and small grant projects, coupled with the reduction in the number of singleton post-doctoral and PhD studies.

It is made clear in the earlier calls, in 2007 and 2008, that the Ministry of Foreign Affairs reserves the right to merge multiple applications, where appropriate and appoint a single project manager. The length of large projects was set at three to five years duration. Applications for initiative grants up to a maximum of DKK 200,000 could be sent to the Steering Committee (SC) of the FFU throughout the year and the SC meeting twice a year would decide which projects to support.

Table 10 Summary of Themes and Research Topics 2006-2011

Theme or Research Area 2006 2007 2008 2009 2010 2011

Agriculture and Natural Resource Management

Market-based agricultural production X

Environment and sustainable use of natural resources and the development of energy in Africa

X X

Agriculture and Sustainable Development X

Food Security X

Climate, Energy and sustainable use of natural resources

X X X

Capacity Strengthening

Building Research Capacity (ENRECA) X X X

Youth and Gender Roles

Youth Education and Employment X X

Children and young people X

Civil Society and Rights

Good governance at Central and/or decentralised level

X X

Fragile States, Conflict and Civil Society X X

Economic Growth, Employment and Property Rights

X

Development and role of the private sector X

Health

Health systems in Africa X X X X

52


Table 11 Summary of Grant Types 2006-2011 (*signifies only limited support)

Theme or Research Area 2006 2007 2008 2009 2010 2011

Projects for building research capacity (ENRECA) X X X

Major research projects (>DKK 5 million) X X X X X X

Smaller research projects (<DKK 5 million) including singleton Post-doctoral and PhD

X X X X X X*

Small Initiative Grants for the preparation of ENRECA-projects and/or major research projects with institutions in developing countries (Maximum DKK 200,000)

X X X X

From 2009 support to PhD awards was tightened so that students from partner countries could only be supported if they were part of a larger strategic project and preferably enrolled at an institution in their own country.

From 2010 all larger strategic research applications had to go through a prequalification process, with priority being given to larger strategic research projects/programmes with joint collaboration between several Danish institutions and partners in the South. Selection criteria favoured projects where PhD and post-doctoral studies were included in larger research programmes instead of being submitted as separate small projects. In the 2011 call, it is stated that larger strategic research programmes should have substantive elements of capacity building, with a focus on national priorities and ownership in developing countries.

In the assessment of the quality of the applications, individual PhD and post-doc as well as larger strategic applications, the innovative nature of the research was considered to be central.

Requirements for applicationsIn 2007 and 2008 the application format and process required a logical framework matrix with indicators and milestones; however in 2009 the guidelines and formats were revised to comply with those of the Danish Council for Strategic Research in order to ease the administrative burden on the applicants, and the logframe requirement was dropped, and with it a potentially useful M&E tool.

In 2008 the Ministry of Foreign Affairs outsourced the administration of the support to development research to Danida Fellowship Centre (DFC). The requirements regarding the applications as well as the application procedure, formats and assessment criteria in the guide to applicants, as well as the formats for the application, budget form, and guide were made available on the internet.

The technical assessment of applications has been done by FFU since 2007, with the assessment of large, prequalified applications, being done by external professional assessors followed by part-consultation. Applications are also sent for evaluation to relevant departments and embassies in the Ministry of Foreign Affairs

From 2007 to 2009 comprehensive criteria were listed in the calls for proposals themselves, and these included not only reference to the overall objective of the support,

53


but went into some detail on the specific items. In 2010 this was simplified to three criteria used by the Danish Council for Strategic Research, with the detail to be found in the Application Guide. The three criteria were:

• Quality of the research being proposed

• The Relevance of the research to national and Danish priorities and policy

• Potential Impact of the research.

7.2 General Management Issues and Findings

Relevance and nature of callsThere have been several shifts in strategic direction and these changes have had implications for stakeholders and those evaluating the proposals, but despite this there has been no deliberate change management process to support them. With any change there will be winners and losers, and in terms of both effectiveness and efficiency, it is important that this is deliberately managed. Reviewers need to be aware of the nature and rationale for the criteria they are using in assessing applications, for example.

The themes for the calls are decided and drafted by the Technical Advisory Services of the Ministry of Foreign Affairs, in May of each year. The draft is discussed with relevant departments and submitted to FFU in June. The FFU does not decide the themes but can propose formulations and sometimes suggests themes which are then included. The calls under this funding modality are linked to Danida priorities (see Table 10) and are not necessarily based on, or driven by, Danish research capacity.

The role of the Danida Fellowship CentreStakeholder responses to the role of the DFC in the general administration and support to logistics and finances of this modality and the PRCP, was very positive, and well appreciated. The only minor issue being that for activities outside of Copenhagen, DFC support for visitors was not of the same high quality. DFC monitors compliance with administrative and financial requirements of the modality but has no role in technical monitoring or evaluation. This area of technical oversight and quality assurance is very limited, with no formal structures or mechanisms in place, especially for individual projects.47

Communication and informationDanida has initiated an information portal which was setup by DDRN and launched in July 2011. It contains some information on projects that were granted after January 2008 and a limited number of older projects where coordinators were able to confirm project details. It is currently maintained by DFC; prior to this, there was no adequate system.

47 The last formal evaluation of FFU projects was done in 2000, and for Danida overall support in 2001. See 1) Evaluation of Danida’s Bilateral Programme for Enhancement of Research Capacity in Developing Countries, December 2000, Ministry of Foreign Affairs, Danida. 2) Partnerships at the Leading Edge: A Danish Vision for Knowledge, Research and Development – Report of the Commission on Development-related Research Funded by Danida, April 2001, Ministry of Foreign Affairs, Danida.

54


The FFU research projects have produced a large amount of information and know-ledge48 but except where projects have specifically planned and budgeted for uptake or dissemination, there is no formal or institutionalised communication-information system, or requirement, for the sharing or promotion of output. Dissemination and uptake of research findings has taken place in FFU projects through a range of mechanisms including policy briefs49, workshops, publications and posters. However the approach is not systematic or guided by an overall strategy, it very much depends on the individual researchers and research topic.

Of the 88 FFU projects for which the Evaluation had data, 20 have had specific budgets for dissemination, ranging from 0.1% to 7.4% (with an average of 3.6%) of the total project cost. Of these 12 are larger strategic projects, where the average was 4.8% and the remaining eight being PhD and post-doctoral studies (average 2.4%). These specific budgets were in projects started after 2009.

Likewise, there is no requirement for researchers or embassy staff to interact and exchange information on outputs or outcomes, even where research may be relevant to the development of Danida’s country programmes, national sector programmes or more general planning and policy.

7.3 Relevance

The thematic approach of the calls for FFU proposals, based on Danida policy and priorities, means that successful projects are, by definition, relevant in this context. The task of assessing relevance to Southern partners falls mainly to embassy staff, which can sometimes create difficulties if proposals are highly technical or if no clear statement of national policy or priorities is in place.

From the review of project documents and objectives, and based on broad national priorities of poverty reduction and food security, it is reasonable to assume that in general all FFU projects are relevant to development issues and priorities. However at the detailed level, it should be noted that gender issues were poorly dealt with in both research calls and in the subsequently approved proposals, despite the fact that Danida had a strategic focus on women’s rights and access to resources50; Broegaard51 notes that micro-finance, property rights and disaster prevention are also marginalised, and the Evaluation can confirm this.

48 Knowledge is defined as information with the tools on how to use it.49 These are not always appropriate. However, in one case, a project produced policy brief documents

for policy makers and published them on a website but there was no mechanism for them to share with the targeted decision makers, in fact the concept of a policy paper was new to Francophone countries so there was an additional requirement for promotion and explanation.

50 Women – a driving force for development – focus on strengthening women’s rights and access to resources – access to education – strengthening position of women in Africa (Danida’s five-point plan on gender equity), from: Commitment to Development – Priorities of the Danish Govern-ment for Danish Development Assistance 2007-2011. The Danish Ministry of Foreign Affairs, Copenhagen, August 2006.

51 Issues Paper for future evaluation of effect of Danida supported research on agriculture and natural resource management. August 2012, Rikke Brandt Broegaard.

55


7.4 Effectiveness

General management issuesThere is no logframe or result framework for the FFU approach as a whole and no clearly articulated objective or set of outputs which succinctly describe what it is trying to deliver, apart from a very broad sense that it is about capacity strengthening and research. This, and the lack of indicators, makes it difficult to measure the full extent of its effectiveness. This shortcoming is not unique to the FFU modality.

The logframe also ceased to be a requirement for FFU projects from 2009. Of the 24 larger projects required to include a logframe between 2006 and 2008, examples were only found in six, although complete project documentation was not available for all projects. The sample did, however, provided a useful insight into the way in which the logframes had been generated and used.

Within the sample there were a range of format and formulation issues including large numbers of specific objectives and results, and in one rather extreme case52 there were three overall objectives, two specific objectives and one result. Indicators where they were present were exclusively expressed in terms of completed activities, and in one case activities were listed under the specific objective. It is clear from the associated reports that the logframes were not understood or considered a key tool for project cycle management and M&E, they appear to have been treated as a box-filling exercise once project documentation had been completed.

Financial management by Danish partners is done well and is effective; however, the development of budgets in project proposals was not always done jointly with Southern partners. This resulted in Southern partners not understanding the nature of the budgets and with control coming from the North the limited ownership created misunder-standings and in some cases resentment that certain activities or purchase were not possible. Despite the fact that guidelines on expenditure and financial management are available online, understanding of Danida financial systems amongst Southern partners was variable.

Technical support and collaborationOverall the FFU projects can be seen to have developed strong partnerships at an indi-vidual level, with good collaboration and support being provided by Danish researchers. Some of these partnerships have a long history, and although not institutionalised, young researchers taking part in projects also get opportunities to establish links.

There is a requirement for FFU projects that proposals should be implemented with a Southern partner; however the selection of those partners by a lead Danish organisation is on a non-competitive basis. This has created something of a closed-shop which can make it difficult for new or alternative organisations in the South to access FFU project funds or Danish partners. This non-competitive selection is the opposite of the PRCP modality which allows Danish organisations to submit expressions of interest to Southern organisations when concept notes from the South are part of a call.

The lack of a requirement for project proposals to include a logframe or result framework has implications for measuring not only project success but also the contribution each project makes to the overall thrust of FFU project support by Danida. The proposal

52 DFC reference 207-LIFE.

56


format requires Objectives, Outputs and Indicators but lacks guidance on how these should be defined and articulated, and whilst implying a results-based management approach the Evaluation could find no evidence that this was understood by applicants in either the North or South. Broegaard noted the lack of indicators and the variability in defining and measuring objectives and outputs whilst also recording the fact that in many cases progress is measured by reporting on completed activities, judging success by the delivery of product, not outcome.

A study of research proposals indicated that the necessary information for appropriately articulated objective, output and outcome statements is present, but that there is a lack of clarity and consistency in the way they are presented (see Annex J for two typical examples).

Several senior managers at Southern institutes, who are working with other development partners, stated that they missed having clear frameworks with indicators. The World Bank now uses a reduced form of a logframe, referred to as a Results Framework, and other development partners all have some form of causal structure which links the use of inputs with the delivery of results, outcomes and impact53.

For projects operating in Francophone countries, effectiveness can sometimes be hampered by language issues, with communication between Francophone and Anglo-phone countries, especially in Africa, being an issue. Language training has not generally been included in project budgets, but there are no administrative reasons why this could not be done.

Use of research findings in Danida programmes is very limited although at national level there were positive examples with follow-up research activities, with or without Danida funding. Similarly there were instances where FFU project outputs became incorporated into larger multilaterally-funded projects. Responses to the eSurvey (Figure 3 and Figure 4) confirmed that project outputs were largely utilised in one way or another, with less than 5% indicating no measurable outputs. An analysis of how the research outputs were used shows that new research initiatives and research into use constituted the main areas, although there were interesting differences when the responses from Danish and non-Danish respondents was considered.

53 Indicator 11 for the Paris Declaration is Results orientated frameworks – an indicator which assesses the degree to which partner countries have results-oriented frameworks align with those of development partners.

57


Figure 3 Responses to What have project outputs led to? (n=191 responses)

No measurable outputs

Research into use

Personal promotion/benefit

New research initiatives – internationally

New research initiatives – nationally

0% 5% 10% 15% 20% 25% 30%

Percentage response

Figure 4 Responses to What have project outputs led to? filtered by nationality

No measurable outputs

Research into use

Personal promotion/benefit

New research initiatives – internationally

New research initiatives – nationally

0% 10% 20% 30% 40%

Non-Danish

Percentage response

Danish

7.5 Efficiency

In general, resources have been used as planned and agreed in project proposals. DFC has received very few requests for significant changes, and these have largely been justified and agreed, although in one case communication delays caused some transitory difficulties54.

Although no major changes or budget costs have been incurred, approximately half of the FFU projects have had to request no-cost extensions in order to ensure completion of PhD studies. This is largely due to the original project length, typically three years, and the time taken to identify personnel and implement a PhD award, a minimum of three and a half years.

73% of respondents to the eSurvey indicated that they were aware of other development partner initiatives in the same technical area as their project and also indicated that they engaged with these to some (43%) or a large extent (22%), see Annex D. Often researchers, particularly in Southern organisations, were engaged on several different projects funded from different sources. Based on analysis of the situation in Burkina Faso and Tanzania, there has been little official harmonisation of FFU projects with research initiatives supported by other development partners, although neither was there evidence of duplication or conflict of interests.

54 On one project the leadership changed, and although initially DFC passed a no-objection, 12 months later FFU indicated that the change was not appropriate and funding was frozen. After further consultation the decision was reversed and the project continued.

58


The embassies in these countries have no official role in the implementation of the FFU projects and there is no official interaction between project and embassy staff, although some limited personal interactions do take place. The lack of a governance structure to support embassy-FFU project interaction has meant that opportunities to incorporate or promote research output in programme development or national sector planning have not been realised.

Table 12 shows the reported output from FFU North-driven projects for which data are available. The Evaluation reviewed all the Project Completion Reports available for the period from 2006 to 2011. The data are incomplete, however from the information available the ENRECA projects produced higher numbers of publications and PhD awards at a lower cost than the Larger Strategic Projects which replaced the funding instrument. The lowest cost for PhD awards came, not surprisingly, from the projects designed specifically to deliver this output. In general terms all projects delivered the outputs in terms of degrees and produced an overall average of 3.2 publications per project at a nominal cost of DKK 1.3 million.

Table 12 Numbers of PhD awards and publications for agricultural and NRM FFU North-driven projects 2006-2011

PhD Awards Publications

Category Projects Planned Awarded Average* Papers Average**

ENRECA Projects 7 27 21 3.0 49 7.0

Larger Strategic Projects 4 6 5 1.3 18 4.5

Smaller projects, PhD 8 8 8 1.0 10 1.3

Total 24 41 34 1.4 77 3.2

* Average number of PhD awards/project category. ** Average number of papers in peer reviewed journals/project.

7.6 Impact and Sustainability

At an institutional level it has been difficult to measure the extent and nature of impact attributable to the FFU projects, with little baseline data and fewer targets against which to make judgements. Some proxy indicators exist however and a case study for SUA in Tanzania is presented below since SUA has a long and extensive history of collaboration with Danish researchers and institutions.

The PhD and MSc training through FFU projects has undoubtedly improved the human resource base at Southern institutions where projects were located, in terms of the number of degrees awarded55. The perception amongst stakeholders (including those who responded to the eSurvey) was that engagement with Danida supported projects had improved their personal performance and to some extent that of the organisation they were based in.

55 At SUA, the relatively large support provided through Danida FFU projects over several years has undoubtedly contributed to a documented increase in number of academic staff with PhD degrees, number of courses supplied and number of student uptake.

59


Institutional status and Danida links with SUADuring the period from 2006 and 2011, SUA has had between 100 and 150 research projects within agricultural and NRM, including environment and climate change, for improvement of livelihoods and the reduction of poverty producing on average of 60 scientific papers each year56.

Table 13 Staffing and Degree Information SUA, Tanzania 2006-20115758

Category/Description 2006 2007 2008 2009 2010 2011

Number of Undergraduate Programmes n/a n/a n/a 20 n/a 30

Number of Postgraduate Programmes n/a n/a n/a 36 n/a 48

Student enrolment, BA 820 712 1,470 1,918 2,078 1,517

Student enrolment, MSc 225 348 236 547 562 377

Student enrolment, PhD 31 12 16 52 48 23

Student output (PhD) 13 13 8 7 14 8

Student output (Masters) 151 150 195 151 252 204

Student output (BA) 657 657 696 695 655 1,103

Academic staff, total n/a n/a n/a 476 n/a 508

Academic Staff with Bachelor degree n/a n/a n/a 33 n/a 93

Academic Staff with Master degree n/a n/a n/a 148 n/a 155

Academic Staff with PhD degree n/a n/a n/a 235 n/a 320

Financial support to SUA from Government of Tanzania (DKK million) 60.8 59.1 70.2 91.3 89.9 98.1

Financial support to SUA from development partners58 (DKK million) 18.6 18.6 26.0 94.5 57.3 81.0

Own income generated at SUA (DKK million) 0.6 8.7 17.0 22.2 29.0 37.0

During this period, the financial support to SUA has increased by more than 150% reflecting significant increases in contributions from both Government of Tanzania and development partners, mainly through additional funds from Norway, EU, DFID and USAID.

Income generated at SUA has also increased by more than 300% due to a sharp increase in the demand for SUA consultancies, production activities and services charged from research activities.

56 SUA estimate.57 From Annual Reports and documentation provided by SUA, Tanzania.58 In 2009 the Danish contribution was <10%. No official figures could be provided to the Evaluation

on contributions from individual development partners for other years, however perceptions from SUA management is that the Danish contribution has been no more than 10% over the period from 2006 to 2011. The Norwegian contribution is perceived to have been more than 50%. The discrepancy between this estimate and the overall value of support to SUA-based projects lies in the amount of funding that is channelled directly through the SUA system.

60


As a consequence of the sharp increase in the inflow of funds, SUA has been able to increase its capacity over the same period shown by increases in the number and qualifications of the academic staff employed at SUA, increases in the number of Undergraduate and Postgraduate Programmes offered and increases in student enrolments and outputs at Bachelor and Master levels. In terms of PhD awards, neither the enrolment rate nor the output number has increased.

Researchers at SUA are now more inclined to stay at the university rather than move to other organisations, because the university’s reputation provides ample opportunities for staff to generate additional income as consultants as well as accessing grants and funding from development partners.

Although Danida-funded research activities have contributed59 to the capacity development achievements at SUA between 2006 and 2011 attribution is difficult/impossible. Norway is by far the most influential development partner at SUA60. Funding provided through Norwegian programmes61 has composed more than 50% of the total external funding to SUA within the period. In addition, a large share of the Norwegian funding has been through comprehensive institutional research programmes, which have aimed at addressing capacity development issues at the institutional level (management and administration of research) as well as at the individual level (research projects). This is contrary to the Danish research funding to SUA within the period, which has focused on support to individual researchers and research projects.

Additional outcomes and benefitsIndividual links continue post-project and allow continued research from both Danida and other sources. The contacts allow networking and exchange of ideas and there were many examples of FFU projects stimulating access to funding from a variety of additional and new resources, even where FFU funding was declined because proposals were too focussed on uptake and dissemination. This implies an impact on institutional capacity through training and publications.

The mechanisms and platforms for information exchange are often limited in partner countries and this reduces stakeholder access to research output. In addition there were also limited budgeting and dissemination-uptake activities designed into in many projects (see above), further reducing access to research output by those who might directly benefit.

59 26 PhD and 36 MSc degrees during the period.60 Norway provides support to Tanzania through the Norwegian Agency for Development

Cooperation (Norad) for research based in Norway (10% of the budget) as well as in-country (90%). The in-country funding is managed by the embassy. Support to universities is untied and based on programmes developed by the universities who work together to produce a programme of what they need. Support to PhD studies encourages field work in Tanzania, and there is an emphasis on the need to make research output available and accessible. PhD and research have specific components for the production of policy briefs and information outputs.

61 For example the Programme for Agriculture and Natural Resource Transformation for Improved Livelihoods (PANTIL), Enhancing Pro-poor Innovation in Natural Resources and Agriculture Food Chains (EPINAV), Climate Change Impacts, Adaptation and Mitigation (CCIAM) and Norwegian Programme for Development, Research and Education (NUFU).

61


Although impact has not always been obvious and is not commonly financed or supported through project design (these are research projects) there is a visible, positive shift in some of the more recent projects62.

7.7 Conclusions

Design and monitoringThe lack of indicators not only hinders evaluation but prevents adequate technical monitoring during implementation. Although the monitoring of administrative and financial compliance has been done efficiently by DFC, there is little evidence of technical monitoring beyond the numbers of degrees and publications produced; outcomes have generally not been considered. The use of planning tools such as result frameworks or logframes would have made it easier to do so, and to institutionalise such technical monitoring.

FFU project successThe FFU projects have been largely driven from Denmark and have provided an attractive, bilateral funding modality in a relatively closed institutional environment that has benefited researchers in Denmark and selected Southern countries. It has functioned well within the scope of the Calls for research proposals, and has had an impact on the human resource capacity of some Southern universities and research institutes at an individual level through the award of postgraduate degrees and the gaining of research experience; the focus in most Southern countries has been at university-level. It has been successful at providing training and at the same time has generated a number of research publications.

The lack of baseline data, the multiplicity of donor and other funding sources and the multiple roles of individuals funded from these different sources make it very difficult to measure and attribute institutional impact from Danida support.

The researcher-researcher approach has focussed on individual research interests and operated in a research paradigm which emphasises the importance of making research output available for dissemination and uptake by others beyond the project boundary. This linear model of researcher-extension-farmer/user is discussed further below, and in relation to the other modalities and potential solutions, in Chapters 10 and 11.

62 One recent project included a budget for dissemination activities, which it considered very important. Farmers were involved via surveys and plantation of oil-trees, the forestry/environment administration via information and reports and other stakeholders via classical media, such as radio, TV, scientific publications and brochures.

62


Future focus for FFU projects Throughout the period from 2006 to 2011 Danida and FFU have lacked a detailed strategy for support to development research and have relied instead on a set of broad objectives63 and a number of thematic areas.

Under such circumstances any modality which is functioning well and delivering post-graduate degrees and publications, can be judged a success, but the question remains as to whether, in the participatory and holistic environment which characterises modern development research, the FFU project approach remains an appropriate modality.

FFU project research is driven by a thematic approach based on shifting Danida priorities, and projects are linked to the skills and strengths of Danish researchers, the majority of whom are in universities. The research focus aims to satisfy academic criteria of quality research publications and PhD/MSc degrees with the links and partnerships being largely built at a personal-project, rather than institutional- programme, level. This linear approach to research assumes that the responsibility for dissemination and uptake of successful outputs is outside the project boundary and the responsibility of others.

The PRCP has started to address some of these issues (see Chapter 8) but a longer- term view, built around Danida’s new strategy for development research, is required which addresses not only the issues of ownership, but also the issues of livelihoods, partici pation, value chains and the role of national research institutes (see Chapters 10 and 11).

63 The basic objective of support as defined in the calls for proposals was to generate knowledge to promote the overall objective of the Danish development assistance to reduce poverty and to support research in fields in which research and new knowledge that is relevant to Denmark’s development assistance and may contribute to solving the problems of developing countries.

63

8 FFU Pilot Research Cooperation Programme (South-driven projects)

8.1 Introduction

The South-driven research projects under the PRCP are seen as a mechanism for making support to research more relevant to Southern partners (see Chapters 2 and 7 for further detail on the background). This chapter documents the findings and conclusions of the Evaluation on the performance of the projects funded under this modality.

8.2 Relevance

The South-driven project concept represents a new approach for Danish funding of joint development research among South and North researchers. The calls have required that the project proposals are developed in the context of southern priorities in the context of Danish strategic development priority areas. This, by definition, means that all projects fulfil the relevance criteria from both the Danish and Southern perspective. In Tanzania, for example, although there was no explicit research strategy document, research activities were guided by two documents64 ensuring broad relevance.

However, while the Evaluation finds it reasonable to link the South-driven research project themes to identified knowledge gaps and needs in the South, it seems much less relevant to require that the South-driven research projects should also reflect Danish strategic priorities65.

During the country-visit to Tanzania, the Evaluation assessed the relevance of the five Tanzanian South-driven research projects, implemented at SUA (four projects) and Mzumbe University (one project) in terms of their relevance to the Southern partner institution. In all five cases it was found that the South-driven research projects were of high relevance and provided value-addition to key research areas in their respective institutions.

The South-driven research project modality also responds to the principles set forth in the Paris Declaration on Aid Effectiveness. The modality has a strong focus on ensuring Southern ownership both in terms of development of the project concept as well as in relation to the process of selecting Danish project partner(s). All South-driven research projects in Tanzania, clearly come from a Southern demand.

Alignment of institutional procedures and regulations of Southern institutions with those required by Danida has caused some difficulties, for example in relation to financial management and reporting, and this initially created some tensions between the Southern institutions and the DFC. Most of these issues have been resolved as understanding of the modality increased and the mechanisms of the modality evolved and developed.

64 Tanzania Agricultural and Livestock policy of 1997 (pp. 13-15) and Tanzania Agricultural Sector Development Strategy (pp. 31-32 or Section 6.1)

65 This requirement has been abandoned in the latest call for proposals in 2013.

64


In some of the South-driven research projects the Southern partners found that the training in Denmark based on state of the art equipment was not sufficiently tailored to reflect local capacities and conditions. Although it was an interesting experience for the PhD students from the South in these cases to get access to high-tech Danish equipment, opportunities to use these new skills in their own institutions was very limited, which made the technology input appear less relevant.

8.3 Effectiveness

Concept Note and Proposal preparationThe South-driven research projects have demonstrated a high degree of Southern ownership. For four of the five Tanzanian projects, the original project idea and Concept Note has been developed by the Tanzanian researcher in the fifth case, the project idea and Concept Note were a consequence of discussions between Danish and Tanzanian researchers who were already implementing an FFU project together. Although Concept Notes have been prepared by Tanzanian researchers, the Danish partners have all played an active role in converting the concept notes into project proposals, involving close cooperation and interaction with the Tanzanian researchers without fundamentally changing the original project idea. This approach has also built a strong sense of team ownership.

The quality of Concept Notes and subsequent project proposals, submitted to DFC, has often failed to meet the requirements and a number of the notes and proposals have been rejected. This reflects a mismatch between expectations on the Danish side to the proposal writing capacity of the Southern researchers, who have overall responsibility for submitting these documents. The BSU initiative includes activities on proposal writing for PhD students and researchers and Southern partner universities which provides potential for synergy between the two modalities.

There is a wider issue here linked to change management support where a new initiative assumed that capacity existed and that all that was necessary for success was to announce the new modality. Some kind of capacity needs assessment in the context of the switch in emphasis to South-driven projects, linked to training to address constraints, may have reduced this failure rate and improved quality.

Calls for expression of interest The mechanism used for advertising the calls for the South-driven research projects has not been effective at reaching a wide audience of relevant researchers in Tanzania (see Table 3). Many researchers with potential for engaging in the process who were met by the Evaluation, even within SUA and UDSM, were not aware of the South-driven research project modality. Indications were that those who received information did so more by luck than through any systematic approach.

The interest among Danish researchers to respond to the open calls for expression of interest for the South-driven research projects has been limited. For the five successful South-driven research projects in Tanzania between one and three Danish institutions showed interest in participating. The main reasons for the low response is said, by them, to be a combination of low incentives, limited ability to control the quality of the research in the project, lack of influence in the selection of PhDs at the Southern institutions and no previous working relations between the Danish and Southern partner researchers.

65


The nature of research tends to be more downstream and offer fewer opportunities for publication in high-quality journals and there were suggestions that the lack of students registered at Danish universities also reduces the financial and performance incentives, from a Danish perspective.

There are, however, good examples of how the South-driven research project modality has created new partnerships and links between Danish and South researchers. It was very evident from the interviews with project partners in Denmark and Tanzania that the modality has been mutually beneficial and encouraged continued cooperation between the project partners. In the case of the Tanzanian South-driven research projects, four out of the five established South-North South-driven research project partnerships did not exist previously. Where projects have been approved and implemented, satis faction amongst stakeholders is high.

Broader context of PRCP modality in researchAlthough the South-driven research project modality represents a new approach for Danish funding of joint research initiatives between Danish and Southern researchers, the modality still contains some of the inherent weaknesses that were identified for the FFU North-driven project modality (see Chapter 7). These include the fact that:

• Project designs and presentation are generally weak at presenting a clear overview of project activities, outputs and outcomes and the interaction between them.

• The success of the projects (according to the project documents) is mainly judged on completion of project activities and delivery of products (PhD and MSc degrees and publications) and not on outcomes (changes in behaviour).

• Time frames are usually unrealistic, mostly due to the time needed for the PhD recruitment and processing procedures, and almost all projects are requesting no-cost extensions of the project period.

• There is an inherent tension between Danish researchers’ academic ambitions which value published articles in high ranked international journals and the Southern researchers focus on the applied research within a national/local context.

The role and influence of Danish researchers in the selection of PhD and MSc candidates at the Southern institutions has generally been limited and most of the Danish researchers have found the selection processes to be insufficiently transparent and participatory. Although in one instance, Southern researchers adopted an unconventional approach to MSc awards which doubled the number of candidates involved66.

The calls for the PhD positions have generally been open and announced outside the home institution in the South, although the majority of the positions have ended up being filled in by existing staff members in those institutions. Although this may be seen as an advantage in terms of developing internal staff capacities at the Southern institutions, there are indications that the selection process has in some cases also served as an internal reward system.

66 On one project, there was an innovative selection of MSc candidates by the Southern university, where individuals were selected from those who had already self-funded for one year (and therefore showed commitment) and the project was thus able to double the number of MSc that it supported. This was marked as a change in the project report to DFC but not queried.

66


8.4 Efficiency

Resource allocationTime and resources have been allocated specifically for Southern researchers to come to Denmark and personally meet and interview Danish researchers who have shown interest in becoming project partners, allowing them to make a choice of partners based on personal interaction. This process has created a strong element of Southern ownership in the projects.

The majority of researchers interviewed, from both Denmark and the South, emphasised the importance and need for allowing sufficient time at an early stage in the project cycle for getting to know each other. This ensured a common understanding of the major aspects of the project development, including budget and resource allocation and the technical issues. This initial investment is considered by the partners to be of key importance for subsequent successful project implementation.

The control and responsibility for funding in the South-driven research projects was appreciated by the Tanzanian researchers. However, they also indicated that the administrative work approving Danish requests and funding was burdensome. This view is supported by DFC, where it has been noted that the administration of the South-driven research projects has required substantially more work and follow-up than in case of the FFU North-driven projects.

The average administrative cost level for the South-driven research projects however, at just below 15%, is slightly lower than that of the North-driven FFU projects, reflecting higher overhead costs for the Danish research institutions.

Capacity and communicationAs in the case of the proposal writing (see above), the need to strengthen administrative capacity is addressed within the BSU work plan, providing another potential for generating of synergies between these two modalities.

In Tanzania, the South-driven research project modality has included a yearly meeting between all Danish and Tanzanian South-driven research project partners and with participation of the Tanzanian Government and the Danish embassy. This yearly event has been an effective mechanism for disseminating of project information and has been important for developing of synergies and relevant interaction across some of the South-driven research projects.

All the South-driven research project partners interviewed in Denmark and South rated the working relations and cooperation as highly positive, a fact also borne out by the eSurvey. The Southern researchers appreciated the role and participation of the Danish researchers and there was a feeling of strong engagement and commitment to the projects from both sides.

67



New and extended partnerships established under the South-driven research project have developed into enhanced collaborative relationships and attracted additional external funding; four examples67 are given in Box 1.

Box 1 Examples of Positive outcomes from South-driven projects in Tanzania

The Development of Enterprise in Solar Drying of Fruits and Vegetables for Employment Creation Project (Project P9-08-TAN) implemented with Danish funding has been a catalyst for attracting additional funding from the World Bank, Norad and DFID. The recently started Sokoine University Graduate Entrepreneurs Cooperative (SUGECO) is also a product of the project.

Two other projects, the Rural-Urban Complementarities for the Reduction of Poverty and Identifying the Contribution of Savings and Credit Facilities and Productivity, Market Assess and Incomes for Small farming businesses through Contracts which were partnerships with the new Danish partners that have allowed the Southern partner to be enrolled in other funding applications and partnerships with EU and DFID funding as a consequence of networking through the Danish partner (P11-09-TAN).

In the Opportunities and challenges in peri-urban livestock farming in Tanzania Project (P6-08-TAN), Danish funding is considered key to establishing baselines and basic research required to create foci for subsequent funds application to USAID and UN Health.

The Monitoring the Environment of Mount Kilimanjaro region and its association with Climatic Changes Project (P10-08-TAN) although still seeking additional external funding for continued running of the monitoring stations has received substantial international interest as well as nationally through the Tanzanian Science Commission.

The experience from the South-driven research projects implemented within the evaluation period shows that one of the benefits from them is that they have the potential to create impact through addressing Southern priorities directly. Some outcomes are already showing a positive impact (see Box 1). Examples include encouragement of young entrepreneurs who have graduated at SUA to start their own businesses and generate income from market sales of their products and the establishment of the first Tanzanian controlled meteorological station on Kilimanjaro. In this latter case, the station is generating increased exposure to the international community because of Kilimanjaro’s significance to global climate change monitoring.

8.6 Conclusions

The South-driven research project modality represents a new and relevant approach, which creates strong ownership in the South, with positive outcomes and new partnerships. It addresses issues linked to ownership and relevance as part of Danida’s current approach to development research and aligns with the Paris Declaration on Aid Effectiveness. Some issues linked to coherence of administrative systems still need

67 Further project information is available through the Danida Research Portal at http://drp.dfcentre.com/ using the title or project number.

68


to be addressed but there is potential for doing this through other funding modalities such as the current BSU approach (see Chapter 9).

By being more responsive to Southern demands, and better reflecting national priorities and issues, the South-driven research project modality has created potential for leveraging additional funding and for generating impact through changes in behaviour. At the same time it has sparked new partnerships and opened up the possibilities of wider collaboration between Danish and Southern institutions.

Although the modality has many positive features it remains somewhat trapped within the same research framework as the FFU projects, with similar shortcomings (see Chapter 7). There is an additional problem that many (but not all) Danish researchers feel that incentives to engage are insufficient.

69

9 Building Stronger Universities

9.1 Introduction

This chapter documents the Evaluation findings and conclusions for two of the four platforms under the BSU initiative active in Ghana and Tanzania, the Environment and Climate Platform and the Growth and Employment Platform, and should be viewed in this context.

Detailed interviews and discussions were held with stakeholders involved in the two platforms in both countries. This included Danish scientists and administrators working in both countries but due to travel limitations, only scientists and administrators in Tanzania. The findings from interviews with Danish stakeholders who were working in both Tanzania and Ghana were similar. The focus of the Evaluation has been on these platforms and the situation in Tanzania which has been, and is, a major recipient of Danida support (see Chapters 2, 7 and 8).

The initiative originated from the Rectors’ Conference and was developed further in consultation with the partners in the South and in dialogue with Danida. The process resulted in a compromise structure with four platforms, a co-funding arrangement and a focus on institutional capacity building and but not research.

The co-funding is a condition of the Danida grant and requires that the Danish partner universities and institutions co-fund the initiative with an in-kind input equivalent to their share of the grant (excluding overheads). Part of the in-kind contribution is provided in human resources, although initially at least Aarhus University has also provided direct financial contributions. The in-kind contribution falls to departmental budgets and resources and has caused difficulties where staff have to justify non- productive (in terms of research papers or Danish-registered students) activities.

9.2 Relevance

The BSU initiative represents a new approach for strengthening institutional capacity at Southern universities, by creating partnerships with those in Denmark. The BSU initiative has the potential to provide inspiration and learning on institutional cooperation between universities in the South and North, which could be of benefit to other development partners.

The focus during BSU Phase 1 between 2011 and 2013 on strengthening capacity for PhD training and supervision at the Southern partner universities addressed an area of key concern and was perceived as highly relevant by both management and researchers at (SUA) and the (UDSM).

The BSU platforms are thematically based on Danish strategic priorities for development and as such are not necessarily relevant to the BSU aim of strengthening the institutional capacity of Southern universities, in response to their demands68. The platform approach

68 For example, if the need is for improved institutional capacity for supervising PhD students, there appears little relevance of having it supported through a Growth and Employment Platform.

70


has also resulted in tensions within and across universities in both Denmark and the South, where the thematic focus areas are an artificial element in the context of how the universities are organised and what their institutional capacity building needs are.

9.3 Effectiveness

Design and roll-outThe roll-out of the BSU initiative and the formulation of Phase 1 were to a large extent done through a top-down approach and Southern partner ownership of the modality has been low during this initial stage of the intervention.

Despite extensive discussions and some tension between the Danish universities, in particular at the initial stages of the BSU, the initiative has facilitated some cooperation between researchers across Danish universities, a cooperation that did not exist previously. In Denmark the BSUEC and BSUGE platforms have managed to establish more formalized cooperation and joint planning and implementation of activities.

Project descriptions are available for both Platforms69,70 which list a number of objectives and output indicators with guidelines and suggestions provided by Danish universities; the dialogue has to a large extent been driven from the Danish side.

There has only been limited delivery of the ambitious targets71 set in the proposals of the two platforms; a recent Review commissioned by Danish Universities72 omitted any mention of specific details on output delivery completed. Full analysis of performance is outside the Evaluation’s terms of reference.

The generic and specific PhD courses given by Danish senior researchers for PhD students and supervisors at the partner universities in the South73 have been popular and well-attended by the students at the partner universities and for some courses

69 Building Stronger Universities in Developing Countries – The Platform on Environment and Climate, Application – Description, Activity Plan and Budget. (2011-2013), Danish Universities, April 2011. http://dkuni.dk/English/Our-Work/~/media/Files/Internationalt/BSU/Environm%20and%20Climate%20Proposal13052011.ashx

70 Universities Denmark Growth and Employment – Description, activity plan and budget for Growth and Employment Platform First Phase (2011-2013), Danish Universities, April 2011.

http://dkuni.dk/English/Our-Work/~/media/Files/Internationalt/BSU/Growth%20and%20Empl%20proposal.ashx

71 For example BSU EG aimed, under the low-case funding scenario for one of its Strengthening PhD Education objective, to complete four baseline and needs assessment reports, develop 12 PhD courses, train 240 PhD students, have 24 faculties able to run courses and train 80 faculty members in PhD supervision. Further details of these can be found in the work plan documents.

72 Building Stronger Universities in Developing Countries – A Program Review Report for Universities Denmark, CMI Commissioned Report David Manyanza (Development Solutions Consultancy) & Johan Helland (Chr. Michelsen Institute), March 2013.

http://bsuud.org/fileadmin/user_upload/bsu-shared/BSU_Review_Report__final__18_03_13.pdf 73 Scientific writing, multivariate analysis.

71


the demand has exceeded the supply, and more courses have been requested (but not yet delivered). Exact participant numbers are not available for all courses as these have not been compiled and submitted by platform committees in the South, the data available are included in Annex K. From anecdotal evidence provided by interviews in Tanzania with Platform managers, responses to Training of Trainer courses to build institutional capacity, was poor (see below for possible reasons).

Due to delays in the submission of Inception Reports by the platforms, funding only started in late 2011 and early 2012, and this has delayed implementation of many of the planned courses although all PhD candidates had been registered by the end of 2012. Currently the 2011 Annual Reports for BSUEC and BSUGE are available online through the Platform websites74, and for the BSUGE platform for 2012.

During Phase I, a small amount of funding was allocated for joint projects between researchers at Southern partner universities and Danish universities. The funds were allocated through a call for proposals and gave researchers the opportunity to develop and submit joint research project proposals for funding elsewhere. The interviews carried out with researchers in Denmark and in South indicated that this funding had been very effective for partners as seed money for approaching larger funding schemes, including the FFU, although this is outside the general ethos of the modality being focussed on research and at the individual rather than institutional level.

Despite the fact that later inception workshops were held at Southern partner universities, the overall framework for the BSU concept and Phase 1 design was largely developed and agreed in Denmark, although some adjustments were made following consultations with southern partners. There was no systematic capacity needs assessment, nor were the needs for support to the changes demanded by the new modality assessed.

Incentives and engagementAccording to those involved at the operational level, only limited incentives to take part in implementation of capacity strengthening activities have been provided for senior researchers in Denmark and the South. The majority of senior Danish researchers interviewed, who had been involved with BSU activities, considered their time and effort spent on BSU insufficiently valued and appreciated by their institutions. Most of them would not be interested in implementing BSU activities again, unless the conditions for their involvement were significantly improved.

All of the interviewed Danish researchers had spent considerably more time on admin-istrative issues than they were compensated for by the BSU. In addition, the agreed 1:1 co-financing by Danish Universities has not yet materialized as planned which has led to further demotivation among the Danish researchers. As a result, some Calls for proposals under the BSU have had no, or very few, applicants. Table 14 summarises the responses to calls, issued during Phase 1, for the two platforms, further detail is provided at Annex K.

74 http://bsuge.org and http://bsuec.org/

72


Table 14 Summary of responses to Calls for Expression of Interest for BSUEG and BSUEC Platforms during Phase 1 of BSU

University Number of calls Average number of Responses (Range)

BSU – Growth and Employment Platform

SUA 9 3.4 (2-6)

UDSM 9 1.8 (1-4)

KNUST 9 1.8 (1-5)

UG 9 1.7 (1-3)

BSU – Environment and Climate Platform

SUA 2 0.5 (0-1)

UDSM 4 2 responses for each call

KNUST 2 2.5

UG 2 2 responses for one call and no responses for second call

Of the 46 calls that were issued, the response rate varied from 0 to 6. The Frequency is shown in Figure 5, which demonstrates that except for one or two of the calls, number of responses to calls was one or two organisations only, with two of the calls getting no response at all.

Figure 5 Frequency of responses to Calls for Expressions of Interest to implement BSUEG and BSUEC initiatives in Ghana and Tanzania.

18

12

10

8

6

4

2

00 1 2 3 4 5 6

14

16

Number of organisations responding to a Call for Expressions of Interest

Freq

uenc

y

One aspect of the BSU institutional capacity development has been to train PhD supervisors (senior researchers) at the Southern partner universities in PhD supervision and teaching. However, based on the experiences from SUA and UDSM the incentives for these senior researchers to attend the training courses implemented through BSU were limited and the opportunity costs to be too high. Researchers were not paid for the time spent in the training sessions organised by BSU, and preferred to spend their time on other activities with better financial rewards, such as consultancy work.

73


Neither of the participating universities in Tanzania (SUA and UDSM) had plans or resources allocated for implementation of the new PhD courses being supported by BSU, giving further justification to those researchers who could have taken part, but chose not to do so.

Limited attendance in BSU training courses by the Tanzanian researchers and the reluctance of senior Danish researchers to commit themselves to conducting the PhD training courses have made their planning and timing a particular challenge. Some courses have been implemented in the middle of holiday periods in Tanzania and announcements have often been made very late.

The first phase of BSU has mainly involved two types of Danish researchers. Firstly those who would like to establish new links and partnerships with a Southern partner, albeit at a loss in terms of research output and/or time inputs, and secondly those who are already engaged in one or more projects in-country and who could make inputs into BSU with minimal additional costs by piggybacking visits and sharing resources.

Approaches and governanceDuring the field visit to Tanzania, the BSUEC and BSUGE platforms at SUA and UDSM raised concerns that there had been a tendency among Danish partners to see the BSU as a one-fit-all model, where common themes/subjects had been picked by Danish universities. This was considered unrealistic as the SUA and UDSM have very different needs and are different in terms of size and resources.

In general, not more than half of the working group members have attended regular BSUEC/BSUGE working group meetings at SUA and UDSM. Compared to other programmes, funded by development partners at SUA and UDSM, the BSU provides no or limited financial incentives for the members to attend these meetings75. This makes it hard for the BSU to compete in a culture where there is more focus on opportunity costs than on opportunities for upgrading of skills. This is an issue that might have been addressed had there been a more intense process of sensitisation and explanation for southern partners and if they had had a broader sense of ownership. The internal contact between the BSUEC and BSUGE working groups at SUA and UDSM has been limited, and has mainly taken place when Danish partners have called for joint meetings/events.

The fact that the BSU operates through two-year contracts has also created difficulties. Being an initiative with the intention of strengthening institutional capacity, the time-frame is too short and has made it impossible to complete planned activities on time, where PhD stipends are required for a minimum of three years.

Operational issues At the operational level, the objectives of the BSU are unclear and not supported by a logframe with clear and measureable outcome indicators (short-term and long-term). Likewise, the terminology is not well-defined by the BSU, what for example does institutional capacity building actually imply? No baseline data have been collected in the partner universities to allow for subsequent assessment of change or impact.

The process of selecting the four Southern partner universities for the BSU has not included a systematic comparative study or assessment of the 10-12 pre-qualified

75 At both universities the chairs of the university platform working groups are receiving up to 60% of the funds allocated to the Platform Coordinators as an incentive to engage with the process. This has been negotiated between Coordinators and Chairs.

74


institutions, and although there was intensive discussion and effort76 in making this selection, there is limited written documentation to justify it; this has been perceived as a lack of transparency by some Southern stakeholders.

The extensive research communication and dissemination skills that have been developed through several years of Danida support to the Networks (in particular DDRN and DWF) have not been integrated into the BSU. Although the reasons for this are not necessarily due to BSU, it represents a missed opportunity for the transfer of relevant learning and capacity skills from one Danida funded development research modality to another.

The current BSU structure makes it difficult for non-participating institutions and stake-holders to become involved in the activities, although technically they are not excluded. It is the case both in Denmark and in the partner countries that BSU is considered by many outsiders to be a rather closed and non-inclusive forum, and this may be necessary for it to remain focussed.

9.4 Efficiency

Although the BSU was discussed at a conceptual level among the Danish universities and Danida over a longer period, the actual planning and design of the operational approach was not based on a thorough assessment of specific needs and conditions within the individual Southern partner universities77. As a result of this, interviews with management and staff at both SUA and UDSM confirmed that the BSU Phase 1 activities had been implemented on an ad-hoc basis without clear linkages to current or planned institutional development processes at the universities.

Box 2 Examples of limited collaboration in capacity strengthening78

1. Norad is currently funding a large Climate Change Impacts, Adaptation and Mitigation programme (CCIAM), managed through the Norwegian embassy and which is imple-mented at both SUA and UDSM. This programme focuses on institutional strengthening and involves a number of the same individuals as the BSUEC. However, no institutional collaboration or coordination takes place between the Norad programme and BSU.

2. USAID is also funding a large Innovative Agricultural Research Initiative (iAGRI) which

aims at strengthening training and collaborative research capacities at SUA. There are no links established between BSU and iAGRI, which has a significant budgetary input into SUA capacity strengthening4 .

76 See http://bsuec.org/fileadmin/user_upload/bsu-ec/Application/BSUPEC_Final_v2.pdf Table A4.3 which represents the mapping that was done of the potential partner institutions. In addition to that, a number of consultations were carried out within the institutions.

77 For example, the need for parallel support to planning and change management processes within the same institutions.

78 iAGRI is USD 24 million over five years with 20 PhD and 100 MSc – focussed on SUA and Ministry of Agriculture, addressing Capacity strengthening – training, equipment, vehicles, library, teaching infrastructure; Gender mainstreaming; Skills improvement for newly qualified lecturers; Leadership skills development for senior academic staff running SUA; Strengthening links to five US universities; Working on S-S links (Brazil and India) working through RUFORUM on other African university links. It has permanent staff and a project office based at SUA.

75


Although BSU key stakeholders at both SUA and UDSM considered the BSU to be a useful gap filler, the BSU platforms at these universities are not coordinated or colla-borating with other related interventions funded by other development partners within the same universities79, two examples (see Box 2) serve to illustrate this. Compared with iAGRI and CCIAM at UDSM and at SUA, the BSU is a relatively small intervention in terms of funding levels and time horizon. The physical distance to Danish partners also makes it challenging to support the institutional integration of BSU activities within the Southern partner universities; both the USAID and Norad projects have Tanzania-based development partner contacts. Collaboration between these initiatives is something that SUA and UDSM should be addressing with the BSU members at those institutes.

The administration costs80 of the BSUEC and BSUGE platforms are both above 25%, including costs for overheads in Denmark and in the South, salaries to administrative staff and audits. This is considerably higher than for the other modalities covered by this Evaluation81.

The main explanation for the relatively high administrative cost level of the BSU is its very heavy and complex governance structure. The four thematic platforms have each been established with their own Secretariat and a Steering Committee for each platform at each Southern and Northern partner institution. It means that within BSUEC and BSEGE a total of 10 Steering Committees have been operating in Phase 1. This has complicated and slowed down communication and decision making procedures, contributing to serious delays in the implementation of work plans.

In addition, the value-added for the Southern partner institutions of having four differ-ent thematic platforms is not obvious, especially as the BSU focus is on strengthening institutional capacity in the South and not on research, often within generic areas.

Under the current structure the workload for BSU administrative staff is high, and staff are working under pressure from both university management and researchers to facilitate the processes. The inappropriateness of the situation is indicated by high staff turnover among BSU Platform Chairs and Coordinators in Denmark, with four out of eight Chairs and Coordinators leaving their posts before the end of Phase 1.


The way the BSU was originally designed and implemented during phase 1 questions the ownership and sustainability of the interventions. BSU Phase 1 was mainly designed from Denmark. Guidelines and suggestions were provided by Danish universities on what could probably be funded and what not and the dialogue during Phase 1 has, to a large extent, been driven from Denmark.

The sustainability of the BSU interventions will to a large extent depend on the extent to which it will be possible to mainstream training courses into the curriculum at the

79 Although on one occasion a course being planned under BSU was cancelled, after discussion with USAID, because a similar larger course was also being run by iAGRI.

80 Administration costs include budget lines for: Administrative fees; External Audit; Work Platform 5 – Platform governance and management; Overhead South, 7%; Overhead Denmark, 20%.

81 The FFU projects, Centres and Networks have/had administrative cost levels around 15%.

76


Southern partner universities. Based on the consultations at SUA and UDSM, there is currently no commitment from SUA and UDSM management that this will happen.

Due to the short implementation period and lack of baseline data, the Evaluation was unable to identify any impact from the BSU interventions. This lack of relevant baseline information, limited institutional anchorage and fragmented interventions compared to the support of other donors, suggests that impact, and its measurement, in the medium-term is unlikely.

9.6 Conclusions – BSUEC and BSUGE

Design and operationThe idea behind BSU, strengthening institutional and research capacity of Southern universities in key areas by calling on the skills and capacity available in Denmark, is a good one. However the operationalisation of this idea has been flawed, and threatens its immediate and medium-term success.

The lack of systematic assessments of Southern needs, and the ability and appropriateness of Danish organisations to address them, has resulted in a mismatch of demand and supply. Support is limited to seven Danish universities who are not necessarily best-placed to provide the capacity development skills demanded from the South, if they are outside specific academic areas.

For different reasons, the incentives for the majority of senior researchers in the South and Denmark to engage in the process are insufficient to encourage involvement by a broad cohort of potential participants and trainers. This is not a direct fault of the BSU idea, however if it is to succeed, it is an issue which needs to be addressed. It has led to poor levels of participation, except by PhD students who have benefited from specific and highly targeted technical training courses relevant to their degrees. Wider attempts to improve institutional skills and capacity have been less successful due to a lack of integration in Southern university systems.

Ownership and institutionalisationThe ownership and institutionalisation of the BSU initiative within the Southern partner universities has been weak and despite inception workshops and special initiatives there was little evidence that BSU had been, or would be, integrated into Southern institutional plans.

Despite the governance structure BSU has relatively low levels of visibility and the operational funding level per institution, is also significantly less than that available from other development partners with a higher local profile.

Governance structureThe existing BSU governance structure is not appropriate for the aims of BSU, and is both expensive and cumbersome. There were no indications that BSU in its current form will produce any lasting and documentable results within the Southern partner universities.

77

10 Synergy and Coherence

10.1 Introduction

The Evaluation has considered five funding modalities (six if the Minor Studies component is included) over a six-year period. These have not all been operating at the same level or for the same length of time, but represent a succession of funding and activity levels. The Evaluation found evidence of the interaction between these different modalities, not surprisingly since many of the researchers in Denmark, and the South, have been, or are, engaged in more than one funding stream.

The synergy between the different modalities, where it occurs, is on an ad hoc basis, and occurs principally because the same individuals are involved in multiple modalities. There is no systematic or formal set of procedures which give guidance on how information should or could be shared, and the lack of a formal role for embassies means that the knowledge of embassy staff of in-country agricultural and NRM research is limited to initiatives stimulated by personal interest. That is not to say that synergies and coherence does not happen, as the case studies below demonstrate, but that it is dependent on individuals and not institutions.

10.2 Linking Different Modalities

Funding levelsFigure 6 shows the development in funding for FFU projects (both South-driven and North-driven projects), the emergence of BSU as a modality and the decline of funding to networks and centres.

Figure 6 Funding levels (DKK million) between 2006 and 2011 for five modalities (agriculture and natural resource management)

60

40

20

02006 2007 2008 2009 2010 2011

80

S-driven projects BSU Centres NetworksN-driven projects

This shifting of funding reflects the role that each modality has played in the overall support Danida has provided to research in agriculture and natural resource management.

78


In the context of the five funding instruments considered by the Evaluation, funding which started in 2006, was split between FFU North-driven projects (66% of total), centres (23%) and networks (11%) but by 2011 this had changed to North-driven projects (64%), South-driven projects (9%) and BSU (27%).

There is ample evidence to show that funding to centres, which provided both the technical capacity and support for researchers in the South, also provided opportunities for small research initiatives and scoping studies which provided the impetus and inspiration for the development of FFU projects. As the role of the centres changed, and they became merged into a university environment, the Danish staff retained the personal links with colleagues in the South and continued to work and develop research activities. Some of these were supported through FFU and others by development partners (principally the EU) with a greater focus on uptake and application of outputs. Some examples of this are given below.

Similarly the networks initially provided a platform for informing Danida technical staff of key issues, and a forum where researchers could exchange ideas and develop proposals for FFU funding.

Case studies of linkages and synergy from Burkina Faso and Tanzania For many years Tanzania has been one of the top recipients of Danish development assistance and Denmark is a significant bilateral donor. In recent years the focus of Danida has been on research activities through participating research institutions, especially universities. For the last 20 years, support to agriculture and natural resources has been through UDSM and SUA and Recently Mzumbe University has started to benefit as a newcomer in this collaboration82, starting with the launching of a pilot research project.

Burkina Faso has also been a major recipient of Danish development assistance and development research since the 1970s and Denmark is today among the largest bilateral donors. The main recipients for Danish support to development research are the National Centre for Scientific and Technological Research (CNRST), the National Forest Seeds Centre (CNSF) and the University of Ouagadougou (see Chapter 2 and Annex L).

Two case studies from Tanzania (Box 3 and Box 4 ) and two from Burkina Faso (Box 5 and Box 6) demonstrate how the different modalities have worked in support of each other and with other funding sources, initiatives and projects. These are not isolated examples.

82 In addition to attending the Evaluation’s SWOT Workshop in Tanzania, the Vice Chancellor of Mzumbe University also travelled to Copenhagen to take part in the Emerging Issues Workshop which was also part of the evaluation process.

79


Box 3 Case Study – Centre, FFU, National Government and Development Partner Links in Tanzania

A small project on forest management started through the Tree Seed Centre in 2003 and developed into an FFU project. The project collected data on PFM and related issued, linking with policy makers in Tanzania who were mainstreaming a participatory forest management (PFM) policy. PFM was used as a tool which linked with Norwegian support to Tanzania’s national initiative to Reduction in Emissions from Deforestation and forest Degradation (REDD) project. The outcome of the project has been a large database which is accessible by professionals to improve the PFM and REDD to provide a baseline of data for the Tanzania National Carbon Monitoring Centre (funded by the Norad for three years) which is bringing together all the REDD-related projects.

Box 4 Case Study – ENRECA, FFU, Multi-stakeholder and Development Partner Links in Tanzania

Original pig project developed under ENRECA in Tanzania and Mozambique addressed the issue of cystercicosis in pigs. The second phase through an FFU project is linking the constraints to wider farming system in order to improve the livelihoods of farmers and reduce the effect of pig diseases to human health. It aims to improve community food security in Mozambique and Tanzania by fostering optimal smallholder pig production model systems through multi-disciplinary and multi-project linkages so that this FFU project has become a cog in a big machine to improve livelihoods.

The project has identified a range of stakeholders including feed millers, drug companies, Minis-tries of Livestock and Fisheries and Local Government. All stakeholders are present at meetings and a multi-stakeholder platform has been established for planning and implementation.

Smallholder pig production is mainly a women’s occupation or a way for the elderly to maintain a pension. Simple farmer-based solutions to increase productivity contribute to secure protein foods, improve farmers’ livelihoods and contribute to poverty alleviation. Research findings are being disseminated to key local stakeholders and internationally via publications, presentations, educational materials and policy briefs.

There has been spill-over from the project into other projects, and some funding is received through ASARECA’s Biotechnology and Biosafety Programme with additional involvement from ILRI. Sharing networking on a sub-regional basis.

One outcome of project has been development of the Cystercicosis Regional Working Group for Eastern and Southern Africa which is a network involving Kenya, Uganda, Rwanda, Mozambique, Tanzania, South Africa which meets every two years and has developed a regional action plan to tackle the problem, with each country having a regional taskforce. In Tanzania it involves Ministries of Medical Research, Local Government, Research, University of Dar es Salaam and SUA. The Working Group’s Secretariat is self-funding and sustainable.

This project is also providing key information and inputs and helping to leverage additional funds from ICONZ, Gates Foundation, Merial Drugs and engagement with One Health. There is also additional EU funding through ADVANZ (EUR 0.5 million) for communication. There are also South-South institutional links with University of Zambia and South Africa where analysis is done for the project. Annual planning meetings involving Mozambique, Zambia, Tanzania, South Africa and Danish stakeholders.

80


Box 5 Case Study – Centre, FFU and Development Partner Links in Burkina Faso

One researcher working at INERA received MSc and PhD training from Danida under Seed Health Centre support. From this position a small project was also developed with Centre support which evolved into an FFU project using a botanical fungicide, which targets the crops of small-scale sorghum and millet farmers and food security.

The project has synergies with other academic research cooperation programmes/projects including one in Tanzania and there have been several South-South exchanges. EU-funding has been obtained for the next phase (FFU funding was declined) which will concentrate on dissemination and uptake.

Box 6 Case Study – FFU Project support to National and sub-regional outcomes in Burkina Faso

The Institut de Recherche en Sciences Appliquées et Technologies (IRSAT) in Burkina Faso is a member of the Centre National de la Recherche Scientifique et Technologique – a national grouping of scientific research institutes. Over the last seven years IRSAT has received support from three FFU projects which have developed its human resource capacity as well as strengthening equipment and laboratory facilities. Currently a laboratory for food analysis has received international accreditation, and leverage EU funds for continued development as well as support from the West African Agricultural Productivity Programme (WAAPP) funded through the World Bank and managed and administered by CORAF/WECARD, a sub-regional organisation for agricultural research covering 22 countries in West and Central Africa. WAAPP support is to recognise the laboratory as a Centre of Excellence for national and sub-regional use.

10.3 Nature of External Links with Development Partners

Under the current system for implementing centrally funded research to agriculture and NRM, there is no role for staff at the Danish embassies in-country in the management of the research programme, and to this extent there is no obligation to connect with other development partners. Donor coordination for agriculture and NRM in Tanzania is limited to an information sharing platform and there are no links to FFU on this platform.

81


10.4 Conclusions

The links and collaboration that have been developed through Danida-funded activities, irrespective of the funding instrument, have had a synergistic influence, encouraging further collaboration and creating informal networks between researchers. The leveraging of non-Danida support for research is one of the positive consequences. At any one time between 2006 and 2011, a more formal structure for information and knowledge sharing, including a simple mechanism for communicating key research findings to Danida staff in embassies, would have increased these benefits.

Currently BSU initiatives and FFU North- and South-driven projects are the operational funding instruments but these also lack a formal mechanism for sharing information with each other and the embassies, relying on personal contacts and interests, and whilst this appears to some extent to function, a more formal structure, however simple, would almost certainly enhance both synergy and coherence.

82

11 Conclusions and Recommendations

11.1 Introduction

The Evaluation acknowledges that Danida has, or is, already addressing many of the issues, both strategically and operationally, identified in this Evaluation, one consequence of covering an historic period. For example, the development of a new strategy for development research has started and operational issues for project-based FFU and BSU initiatives are being addressed.

The following evidence-based conclusions and recommendations should be taken in this context, and are intended to provide support and guidance to the process.

11.2 Strategic Level Conclusions

Paradigms for agricultural and NRM researchA significant part of the Evaluation’s purpose has been to look forward to the development of Danida’s new strategy for development research. In this context, and before such a strategy is developed, it is worth considering the current research paradigm under which the FFU North-driven, and to a certain extent the South-driven, projects are operating. As described above (Chapter 7) the focus of the current paradigm is a North-driven, thematically-organised, academically-focussed linear model involving, for the large part, university-based research in Denmark and the South. Current development thinking has moved away from this approach to a more holistic view based on a broad-base of stakeholders, value chains and wider consideration of policy environments.

Similarly capacity strengthening under the BSU also needs to be reconsidered, and the appropriateness of the current model reviewed on the basis of the findings and conclusions of this Evaluation (Chapter 9).

The most appropriate research paradigm and approach to capacity strengthening will depend on the strategy which Danida adopts83. One option is provided by a research paradigm widely used throughout Africa and summarised as a Case Study in Box 7. This approach is equally applicable to situations outside Africa, where the same need exists for a strong agricultural sector which can drive economic growth.

The extent to which cross-cutting issues have been dealt with between 2006 and 2011 is mixed. Some have received priority treatment (climate change, environment) others have been less obviously considered (gender, youth). The current Strategy for Denmark’s Development Cooperation identifies quite clearly several key priorities which cut across sectors and which influence and effect support to research in agriculture and NRM. Of these green growth, stability and issues linked to the human-rights based approach84

83 If, for example, the decision is made by Danida that development research should aim to achieve an increase in the competiveness and productivity of agriculture and agricultural markets which will contribute to expanding the agricultural economy and drive economic growth (see Figure 2), then a fresh look at how research is supported is needed.

84 Participation, Non-discrimination, Transparency, Accountability.

83


are perhaps the most clearly relevant and should be integrated into the new paradigm and Strategy.

Planning, implementation and M&EIt is clear from the evidence that one of the most significant features (which is in the process of being addressed) influencing the weaknesses identified during the evaluation period has been the lack of a clearly articulated strategy and plan. This could have formed the basis for the implementation and management of the funding modalities and could have linked and supported coherence of the various projectised activities, providing a basis for monitoring and evaluation.

Box 7 Case Study – Support to sub-regional agricultural research (East and West Africa)

Agricultural research in Sub-Saharan Africa is covered by the CAADP and FAAP policy and framework documents, with a focus on NARS rather than just universities. At a Pan African level this is coordinated by the Forum for Agricultural Research in Africa (FARA) and at sub-regional level by three organisations (SRO) the Association for Strengthening Agricultural Research in East and Central Africa (ASARECA), Conseil Ouest et Centre Africain pour la Recherché et le Développement Agricoles/West and Central Africa Council for Agricultural Research and Development (CORAF/WECARD) and Centre for Coordination of Agricultural Research and Development For Southern Africa (CCARDESA) linked to various regional economic communities.

Both ASARECA and CORAF/WECARD have well-developed, logframe-based strategies and operational plans built on CAADP and FAAP principles, and which use a holistic approach to research variously described as integrated agricultural research for development or an agricultural innovation system approach. At the heart of these approaches is a participatory, demand-driven and prioritised paradigm which addresses agricultural systems and value chains and the wide range of constraints which address them including cross-cutting issues such as gender, environment and climate change. Implementation is projectised but built around platforms for the exchange of information and knowledge. Capacity strengthening is a one of nine key FAAP principles, and focuses not only conventional issues but also on improving access and generating empowered stakeholders. Funding is provided by member states and development partners and is either managed through a multi-donor trust fund (DFID, EU, USAID), or by direct payments managed by the SRO (AusAID) itself.

None of the funding modalities (past or present) have, or had, indicators which allowed the assessment of changes or outcomes linked to funding85. A loose theory of change approach, linked to broad objectives, has been implied but the Evaluation finds that a causal framework based on a logframe approach (LFA) would provide a number of clear and distinct advantages not only for monitoring and evaluation but also for planning, implementation, communication and coordination.

In the context of M&E, monitoring of administrative compliance has been done by DFC, and although there are gaps and inconsistencies in the reports available, generally

85 Whilst it is true that Centre contracts did have performance frameworks, indicators were limited to measures of completed activities, such as PhD and MSc awards and meetings held and no evidence that these had been reported against could be found. The same is also true of project indicators, where these exist.

84


this has worked well. There is, however, no mechanism in place for technical evaluation of research projects due to the lack of baseline data and inconsistency and inappropriate nature of indicators. Even projects pre-2009 with logframes lacked indicators other than those of the numbers of degrees awarded and Workshops held-type, and the quality of the logframes themselves were such that they were of little practical use as monitoring tools.

Clear and coherent planning for interventions is a prerequisite for the success of Danida’s programme of support to research in agriculture and NRM. A LFA at strategic level will make it feasible for a similar, nested results-based approach for projects and other development initiatives. This will bring it in line with many other development partners and Southern organisation.

For projects under FFU support the processes for the identification of partner institutes differs between the North- and South-driven modalities. There is no competition for the Southern partners of North-driven research proposals. This is restrictive and in the view of the Evaluation counter-productive if the ultimate aim is to build capacity in the South.

Change management There have been several changes in the modalities and nature of funding during the period under evaluation with limited formal support, consultation and guidance. For new approaches and modalities to succeed, ownership and understanding are critical. Implementation of the Evaluation recommendations will require deliberate management of the change process, if implementation is to succeed.

Whilst not all aspects can be South-driven, there is ample scope for an increase in the base of stakeholders during planning and for support during implementation of changes. Future changes in direction, introduction of new modalities and ideas, introduction of the new Strategy will all need to be accompanied by sensitisation and consultation workshops involving a broad-base of stakeholders.

Funding mechanismsCurrently both the North- and South-driven FFU projects are funded on a project by project basis, with individual researchers in either the North or the South having responsibility for funds, this makes coordination and coherence at the level of organisations, difficult, especially in the South.

There are several options and models for the disbursement of funds to support research and capacity strengthening. Some development partners utilise a basket funding approach through externally managed multi-donor trust funds (DFID, EU, USAID, CIDA), others, including Danida, provide funds at a programme-level to national or sub-regional organisations (Norad, World Bank, AusAID).

From 2006 to 2011, Danida has relied on project funding on an individual researcher basis, with funds channelling through Southern institutional systems for specific activi-ties. The Evaluation has interacted with a number of organisations and administrators, both North and South, and concludes that a more institutional and programme-based approach would be closer to current trends in development assistance.

85


Capacity strengtheningCurrently the key mechanism for capacity strengthening is the BSU initiative (see Chapter 9) and whilst the idea underpinning BSU is sound, operationalisation, at least for the two platforms evaluated (BSUEC and BSUGE), has been flawed. The Evaluation finds a significant shift in its strategic approach should be considered after the current phase has been completed.

In general the implementation of PhD and MSc degrees under the FFU projects has been straightforward and followed well-established processes and procedure; however there are some operational areas with potential for improvements and several stake-holders, through discussions and the SWOT Workshops, expressed reservations about selection processes.

Capacity strengthening is not just about PhD and MSc degrees, and many stakeholders in the South and in Denmark, expressed the view that a broader definition should be adopted, which considered capacity strengthening as an aspect of empowerment which includes providing stakeholders with the skills to access and use information, work effectively and efficiently within their institutional systems and interact and respond to wider challenges. These are issues that could be, and to some extent are being, addressed under BSU.

Mechanisms for Communication and Coordination – The interaction and sharing of information across and between modalities and stakeholders has been sub-optimal, although there have been exceptions. There are two main reasons for this, firstly the nature of individual projects largely focussed on research as an end in itself and, secondly, the lack of institutionalised mechanisms which place specific requirements on those implementing projects to share. This needs to be addressed at a strategic and operational level.

11.3 Recommendations

Paradigm for Agricultural and NRM ResearchRecommendation 1. In developing its new Strategy for Development Research, Danida should consider institutionalising a research paradigm which moves away from the current linear model, to one that is holistic, participatory, linked to value chains and largely driven by Southern priorities86.

Recommendation 2. As part of its new Strategy for Development Research, Danida should consider including a particular focus on the need for support to strengthening of national institutional frameworks and capacities for planning and coordination of development research within Southern partner countries. This would include support to formulation and implementation of relevant strategies and policies for prioritising and coordinating of research interventions within and across research institutions in the partner countries. Such a focus would benefit from stronger embassy engagement in research activities (see also Recommendation 3).

86 This type of approach is often labelled Agricultural Innovations Systems (AIS) or Integrated Agricultural Research for Development (IAR4D).

86


Mechanisms for Communication and Coordination Recommendation 3. The roles and responsibilities of Danish embassies in relation to planning and implementation of Danida research initiatives should be redefined and institutionalised to become a more useful platform for follow-up and sharing of information as well as for potential application of relevant results from in-country research activities, including in relation to Danish-funded sector support programmes. Specific issues to consider:

3.1 If the potential for a more programme-based approach to development research will be positively considered by Danida in countries with high levels of Danida supported research activity (see also Recommendation 2 and 8) the relevant Danish embassies should become more involved with coordination, follow-up and contact to supported national research institutions to ensure synergy and coherence, including with research activities supported by other Development Partners within the partner countries.

3.2 Annual circulation, by DFC to embassies, of 20-line summaries abstracted from the FFU progress/completion reports and BSU annual project reports.

3.3 Requirement for more systematic briefing of embassies on planned country visits from FFU and BSU project coordinators and staff.

3.4 An annual in-country research event (e.g. workshop or seminar) with participation of representatives from BSU and FFU projects, national governmental institutions, the embassy and possibly other stakeholders as well (e.g. other Development Partners, private sector actors, national research institutes).

Planning, Implementation and M&E for Agriculture and NRM ResearchRecommendation 4. In the development of its new strategy for development research, Danida should use a Log Frame Approach (LFA) including stakeholder and problem analysis. The development of such a framework should precede the writing or formulation of any strategy.

Recommendation 5. The LFA and Result-based Management (RBM) should be institutionalised within Danida’s modalities for funding development research, and utilised from the strategic level down to projects and other funded activities. It should be used to support and encourage more coherent nesting and linking of activities and funding and used to demonstrate clear causal links between inputs, activities, outputs, outcomes and objectives.

Recommendation 6. Specifically for the institutionalisation of the LFA the change management support (see Recommendation 7) should be provided with adequate resources and include:

6.1 Training and capacity strengthening in LFA and RBM, and the sensitisation of stakeholders to the new Strategy.

6.2 Agreement on clear definitions and instructions on what constitutes an objective, output, outcome and indicator for inclusion in DFC guidelines.

87


6.3 Technical monitoring and evaluation of research projects against agreed product and outcome-based indicators linked to logframes should be included in Annual (against milestones or intermediate indicators) and Project Completion Reports requiring baselines at project start.

Change ManagementRecommendation 7. The introduction of any new strategies, funding instruments, tools or guidelines should be deliberately managed and institutionalised using change management principles and fully supported with well-resourced integrated workshops, documentation, capacity strengthening and technical support, as appropriate.

Funding of Support to research in Agriculture and NRM Recommendation 8. Where feasible, development research funds should be provided directly to organisations in support of programmes, rather than projects and individuals, in parallel with developing the appropriate institutional capacity to manage them. As an interim step, resources for South-driven FFU projects should continue and be increased by reducing, or merged with, North-driven project support in those countries.

Capacity Strengthening and BSU Recommendation 9. In the short term, the BSU governance structure should be simplified. Specifically the administrative and technical functions of BSUGE and BSUEC platforms should be merged and a common secretariat established that has a communication function linking to the other platforms.

Recommendation 10. A comprehensive, independent, technical review of the whole BSU initiative should be implemented as soon as possible, to inform a decision as to whether it should be continued in its current form. Issues to be considered should include:

10.1 The cost-effectiveness of including the BSU concept as a new capacity develop-ment and empowerment modality nested within the Danida development research strategy, technically under FFU and administered by DFC.

10.2 Narrowing the Southern-focus of BSU to permit larger, institution-based inputs at fewer Southern partner universities and reviewing the current group of Southern partner universities87 to determine whether support should be to smaller, under-resourced universities with greater potential for generating internal change and impact.

10.3 The nature and options for improving incentives and ownership of BSU.

87 The current Southern partner universities are relatively large universities which already receive significant amount of external funding.

88

Annex A Abbreviated Terms of Reference

1. Background

In order to assess Danida’s support to development research over recent years, as well as to provide recommendations which will feed into the current process of formulating an overall strategy for this support, Danida’s evaluation department has decided to commission an external evaluation. So as to allow the evaluation to pursue an adequate level of analytical depth it has been decided that the thematic scope should be limited to support for research within agriculture and natural resource management. This thematic area has received substantial support and represents an area of strong interest to Danida. The period covered will be 2006 to 2011.

Danida has supported development research within a range of subjects for several decades. Over recent years, annual funds have ranged between DKK 200 and 285 million, which have been disbursed through various channels, including the Consultative Research Committee for Development Research (FFU), research networks, and centre contracts with KU-LIFE.

The research support has not previously relied on a formal strategic framework (a such is currently under preparation), but has been guided by recommendations provided in the so-called Hernes report of 2001, in which an international panel examined Danish development-oriented research. One of the key recommendation was to link support to research closer to priorities in Danish development cooperation, and as a consequence, over the last decade, efforts have been put into ensuring that support to development research has taken shape in line with the International Development Cooperation Act (Lov om Internationalt Udviklingssamarbejde), development policy priorities, other Danida strategies, as well as to ensuring that it conforms to the Danish research grant system in general. In 2012, an overall objective for Danish support to development research was formulated for the first time in the new International Development Cooperation Act (§7), declaring that it should strengthen research capacity in partner countries and create new knowledge capable of alleviating development problems.

Support to development research is distributed primarily through account §06.35 on the Danish Finance Act. Sub-accounts correspond to “research and information activities in Denmark” (§06.35.01) and “international development research” (§06.35.02), and these are further subdivided into more specific channels, each with a specific purpose. An overview of the specific channels, as presented in the Finance Act of 2012, is given in Figure 1 below. Channels to be covered by the present evaluation are listed in bold type.

89


Figure 1 Overview of channels of support for development research

Four-digit account Six-digit account Eight-digit account

§06.35Research and information activities

§06.35.01Research and information activities in Denmark

§06.35.01.10 Projects in Denmark

§06.35.01.11 Research activities

§06.35.01.13 Information activities

§06.35.01.14 Intercultural cooperation

§06.35.01.15 Fact-finding activities (minor studies)

§06.35.01.17 Seminars, courses, conferences etc.

§06.35.01.18 Evaluation

§06.35.02 International development research

§06.35.02.10 The consultative group on interna-tional agricultural research (CGIAR)

§06.35.02.11 Other international development research

Note: Channels to be covered by the evaluation are listed in bold type.

Of the channels presented in Table 1, §06.35.01.10, §06.35.01.11, §06.35.01.15, §06.35.02.10 and §06.35.02.11, are most directly related to production of research and Table 2 gives an overview of commitments made through each of these channel over the period from 2006 to 2011. Together §06.35.01.10 and §06.35.01.11 constitute the overall frame for support to research on development related topics and capacity building. The purpose of the former channel, §06.35.01.10, is, according to the Finance Act of 2012, to support the use of Danish competencies in development research, e.g. through establishing long-term partnerships between institution in Denmark and institutions in partner countries. Specific modalities over the period to be covered by the present evaluation have included support to research networks and to research centres at the Faculty of Life Sciences at the University of Copenhagen (KU-Life), and from 2011 support to the “Building Stronger Universities” initiative.

Until 2007, six thematic research networks received Danida support (see Figure 2), but in 2007 the research networks for environment (RENED), agriculture (NETARD) and governance, economic policy and public administration (GEPPA) were merged into one network – the Danish Development Research Network (DDRN) – and support to the NPSD Poultry network was discontinued. The last commitments were made to the DDRN, the Danish Water Forum and Research Network for Health in 2009 (totalling DKK 14 million), and covered a final period ending in June 2011.

Support has been provided to three research centres located at Copenhagen University (KU-LIFE) until 2012. These were continuations of support provided to earlier centres, e.g. Danida Forest Seed Centre and the Danish Bilharziasis Laboratory, as these were brought under a new institutional organisation with KU-LIFE. From 2011 this support was replaced by a the new modality, “Building Stronger Universities in Developing Countries”, which cooperates with “Universities Denmark” (an organisation of eight Danish universities). It comprises four thematic “platforms” – ‘environment and climate’, ‘growth and employment’, ‘human health’, and ‘stability, democracy and rights’.

90


Finally, since 2011, DKK 10 million a year has been given to the three-year-long international research and communication programme ReCom (co-financed by Sida and coordinated by UNU-WIDER), with the objective of summing up and communicating existing knowledge about the forms of aid that work within each of the five areas: growth and employment; environment, energy and climate; good governance and conflict and stability; social development; and gender equality. ReCom will not be covered by the present evaluation.

The purpose of funding through §06.35.01.11 is to support the development of new knowledge for the benefit of developing countries and to maintain and expand the capacity of these countries with respect to producing research. It is a requirement that supported research involves cooperation between research environments in developing countries and in Denmark, and that it contributes to strengthening research capacity in the developing country. Grants are awarded on a competitive basis after annual calls for applications within priority themes of relevance to Danish development cooperation and to the needs in partner countries. The Consultative Research Committee on Devel-opment Research (FFU) evaluates the applications according to three equivalent criteria: the quality, relevance, and potential effect of the research. The FFU assesses which applications are worthy of support and forward their recommendation for formal consent by The Danish Council for Strategic Research (DCSR). The Ministry of Foreign Affairs makes the final decision on which applications to be supported among the applications worthy of support. While North-South collaboration is a requirement, the main applicant must generally be affiliated a Danish research organisation. Exempt from this rule are grants financed through the Pilot Research Cooperation Programmes (PRCPs) that were launched in Tanzania and Vietnam in 2008 and in Ghana in 2011, in an effort to make research more South- and demand-driven. In the PRCPs local researchers draw up concept notes (within the framework of Danish development cooperation) and select Danish researchers for cooperation, after which they jointly prepare the definitive research projects. The PRCP is further set apart from other funding through §06.35.01.11 in that priority themes are defined in framework agreements with the programme countries, and thus not on an annual basis. Individual projects under these programmes must otherwise meet the same requirements as other research cooperation and are approved through the same process mentioned before. The Danida Research Portal provides88 a detailed overview of projects supported through §06.35.01.11.

Funding through §06.35.01.15 is for minor studies of a “fact-finding” nature, implemented with the main objective of strengthening the quality of Danish development cooperation.

In preparation for this evaluation, an Issues Paper has been developed, which gives an overview of central issues and challenges of Danida’s research support within agriculture and natural resource management themes. As part of this exercise, an overview of more specific elements, objectives and goals of modalities to be covered under §06.35.01 has been prepared, and this is annexed to these ToR89. Funding through §06.35.01.13, §06.35.01.14, §06.35.01.17 and §06.35.01.18 will not be covered.

Funding through §06.35.02 will also not be covered by the evaluation. It comprises sup-port for international organisations that bring about research result, provide advisory services, and carry out education and capacity development for the benefit of developing countries

88 http://drp.dfcentre.com/ 89 Please find full ToR at http://evaluation.um.dk/

91


within various fields or agriculture, health and social sciences. For some years, there have been efforts to focus this support, with a narrowing down of the number of partner insti-tutions, while awarding each institution multi-year grants. Within the area of agriculture, Danish support to international research is primarily directed at the CGIAR system, which undertakes research focussed on increasing agricultural production (including in view of climate change) in developing countries. “Other international development research” comprises support to institutions that conduct research within health and social sciences.

In addition to the funding through §06.35, Danida is responsible for a series of grants for other types of development research, which will not be covered by the present eval-uation. This includes the Universities, Business and Research in Agricultural Innovation (UniBrain) programme (DKK 30 million in 2010 and DKK 99 million for the period 2012/2015), which aims to foster innovative solutions and products, as well as to strengthen the role of the research communities and of universities in agriculture and agro/industry, and hence economic growth and employment. Further, a series of Danish institutions, including DIIS, Danish Institute for Human Rights and the Rehabilitation and Research Centre for Torture Victims (RCT) receive core funding contributions from Danida, part of which finances research and analysis, as well as participation of researchers in Danish and international debates and academic networking. Likewise, Denmark contributes core funding to a series of UN organisations and to the World Bank, as well as a great deal of earmarked allocations to these organisations’ research activities. And finally, there are research components in certain sector programmes, such as the environmental programme in Bolivia, the business and agricultural sector programmes in Vietnam, the budget support programme in Mozambique, and the transitions support programme in Bhutan.

Figure 2 Commitments by funding instrument, support for development research

Commitments by funding instrument (million DKK); with budget codes of the Finance Act 2006 2007 2008 2009 2010 2011 Sum

1. FFU (§06.35.01.11) 96.7 96 132 167 133 91 715.7

1.1 Competitive Research Grant 96.7 96 112 147 104 71 635.7 **

Operating costs DFC and FFU, total 9

1.2 Pilot Research Projects (Vietnam, Tanzania, Ghana)

20 20 20 20 80 **

2. Projects in Denmark (§06.35.01.10) 47.1 54.7 37.4 52 32.2 106 329.4

2.1 Building Stronger Universities (BSU) 5 3 60 68

2.2 Research Networks 8.6 16.2 14 38.8

2.2.1 Danish Water Forum 1.5 3 4.5 *

2.2.2 Research Network for Health 0.8 3.2 4

2.2.3 GEPPA (Governance Economic Policy, Public Administration)

1.3 1.3

2.2.4 ReNED (Environment) 1.2 1.2 *

2.2.5 NETARD (Agriculture) 1.4 1.4 *

2.2.6 NPSD Poultry Network 2.4 2.4 *

92


2.2.7 Danish Development Research Network (DDRN)

10 10 **

2.3 Centres at Copenhagen University (KU-LIFE)

38.5 38.5 37.4 33 29.2 36 212.6

2.3.1 Seed Health Centre 9.5 9.5 11 8 6,8 44.8 *

2.3.2 Forest, Landscape and Planning 6 6 6 6 5.2 29.2 *

2.3.3 Institute for Health Research and Development

23 23 20.4 19.0 17.2 102.6

2.4 ReCom 10 10

3. Minor Studies (§06.35.01.15) 10 10.2 6.3 6.2 11 11 54.7 **

4. International Agricultural Research (CGIAR) (§06.35.02.10)

38 36 35 35 35 35 214

5. Other International Development Research (§06.35.02.11)

12 16 15 15 25 25 108

Sum 203.8 212.9 225.7 275.2 235.2 268 1422

Sources: Issues Paper prepared for the evaluation (figures for 2010 and 2010 updated according to “Forskningsredegørelsen 2011”. To be included in the evaluation: *All projects (all agriculture and NRM related); **Only projects related to agriculture and NRM. 2. Evaluation Purpose

The dual purpose of this evaluation is to:

• Assess, document and explain the relevance, effectiveness and efficiency – and where possible sustainability and impact – of Danish support to development research within the thematic areas of agriculture and natural resource management. Emphasis should be on identifying core elements of importance for advancing the support and its results, and by implication both support channels, individual projects and important conditions and processes that frame the research (e.g. criteria that supported activities must live up to, follow-up, incentive structure etc.) must be analysed.

And on the basis hereof,

• Provide lessons learned and recommendations which may feed into on-going discussions on how to improve support to development research, and more specifically into the current process of developing an overall strategic framework for support to development research, which is expected to be published in May 2013.

3. Scope of work

Evaluation periodThe evaluation will mainly cover research projects which have been approved and for which commitments have been made between 2006 and 2011 (both years included).

93


Projects approved before 2006 will not be covered, even if implementation extents into the period covered by the evaluation. An exception should be made for projects approved between 2006 and 2011 that have received funding in several phases, where it will be necessary to look also at phases initiated before 2006.

While this time frame has been chosen so as to allow for tracing and assessment of processes involved in supported activities covered by the evaluation, it is clear that pos-sibilities for assessing achieved outputs and outcomes will vary with the time of initiation of the activities. Thus, for example, research projects approved in 2011 can largely not be expected to be achieving outcomes yet, and criteria and methods for assessing them will need to take this into account (e.g. by focussing of relevance and efficiency, and assessing effectiveness primarily through evaluation of the plausibility of assumed causal chains).

Thematic focusThe thematic focus of the evaluation will be on ‘agriculture and natural resource manage-ment’. This should be interpreted broadly to also include, for example, veterinary science, aquaculture, climate change, land use planning, and ‘green’ environment. An overview of the relevant research projects funded between 2006 and 2010 is provided in the Issues Paper developed in preparation for this evaluation, but will need to be updated to cover also 201190 .

CoverageThe evaluation must include analysis and assessment of support to development research within agriculture and natural resource management both overall and at the level of individual projects and modalities. Further, an assessment of the relative merits/com-plementarities/etc. of different modalities is expected. In order to do this, the evaluation should not only look at initial objectives and achieved results, but also on the processes linking and shaping the two, so as to be able to identify forward-looking lessons on what can be done, how and why, to enhance the merits of the development research support.

The modalities to be covered are:

• The competitive research project grants

• PRCP projects

• The research networks relevant to the theme of the evaluation: Danish Water Forum, ReNED (Environment), NETARD (Agriculture), NPSD (Poultry network) and DDRN (Development research).

• The research centres (institutional framework contracts) at KU-Life. Relevant to include are: “Danish Seed Health Centre” and ‘Centre for Forest, Landscape and Planning’.

• The BSU initiative

• Fact-finding activities (Minor studies)

90 Broegaard, R.B. (2012): ’Issues paper for future evaluation of effect of Danida supported research on agriculture and natural resources’ (plus data files).

94


While the entire portfolio of relevant research projects (competitive and PRCP) is to be covered descriptively (the Issues Paper identified 92, but this list will need to be updated to include 2011), more in-depth assessments are expected of a selection of projects. The majority of these projects will be from Burkina Faso and Tanzania, where field visits are to be made, however a fraction of support in other countries will also be subjected to desk-based in-depth analysis. Criteria for selecting these projects have to be developed by the evaluation team.

Establishing the relation between the in-depth analysis of the selected projects and processes on the one hand, and the descriptive analysis of the overall portfolio on the other, will be important, and the inception report should include reflections on how this will be ensured.

Given its novelty, the BSU initiative cannot be covered comprehensively by the evaluation. However, it will be required to assess the set-up of the BSU initiative, including preliminary experiences.

4. Evaluation Questions

Evaluation questions will primarily be centred around the OECD/DAC evaluation crite-ria of relevance, efficiency and effectiveness. Considering the short lifetime of a number of the projects to be covered, it is likely to be too early to assess impact and sustainability in some cases. However, where possible (e.g. in the case of research projects that have undergone several phases and centre contracts) these criteria should also be applied. It will be important to keep in mind, as described in the introduction above, that different support channels have different purposes, and consequently will to some degree need to be assessed against different objectives. Likewise, not all the below listed evaluation questions will be equally relevant for all modalities. At the same time, the evaluation will be required to contain an element of comparison of the relative relevance, effectiveness and efficiency of different categories of support, e.g. North- vs. South-driven, funds subject to competition vs. centre contracts and, for FFU projects, small (e.g. PhD and post doc grants) vs. larger strategic projects. In undertaking such comparisons, it will be important to compare over a sufficient range of issues to be able to highlight the different merits of different categories, and determine whether there are likely to be trade-offs implied in prioritizing certain categories over others. Further, to meet the objective of contributing to learning and in order to ensure a basis for recommendations for future support, the evaluation should strive to explain how and why support has achieved the results it has (in terms of relevance, effectiveness, efficiency and where relevant impact and sustainability).

RelevanceAs point of departure, the criterion of relevance relates to the extent to which the objectives of an intervention are consistent with beneficiaries’ requirements, needs, overall priorities and partners’ and donors’ policies, strategies etc.

In the present case, relevance will need to be assessed at different levels, and questions to be answered will include, but not necessarily be limited to the following:

• Is the composition of the portfolio of Danish support relevant in view of partner country policies and strategies?

95


• Is the composition of the portfolio of Danish support relevant in view of Danida policies and strategies?

• Are the objectives of specific projects/networks/centres/studies relevant in view of knowledge gaps and/or information needs and, where relevant, research capacity needs within the substance area and countries addressed?

• How and to what degree is the relevance of the overall portfolio of projects and mix of modalities continuously ensured (e.g. by responding to changing Danish or partner country priorities etc.)?

• How appropriate is the procedure for formulating research themes guiding FFU annual calls for applications in terms of ensuring relevancy, particularly in terms of usefulness (to partners and/or Danida)?

• How appropriate and relevant are the different channels and modalities, individually and compositely? Where relevant, this assessment should be made in view of the principles set forth inter alia the Paris Declaration on Aid Effectiveness (particularly alignment and ownership), and in terms ensuring that research is demand-driven.

• How appropriate are the different strategies for ensuring and enhancing relevance to end users, both in terms of content, partner involvement, dissemination etc.?

EffectivenessAs point of departure, the criterion of effectiveness relates to the extent to which the intervention’s objectives were achieved, or are expected to be achieved taking into account their relative importance. In the present case, this links to assessing the delivered outputs and, to the degree possible, outcomes – in terms of research/knowledge produced, disseminated, put to use etc. and research capacity built – for individual projects and to the degree possible also at portfolio level.

It will be important to assess effectiveness in terms of the stated objectives of Danida support to development research, including the specific objectives of individual modalities, as well as the specific objectives of individual projects. The assessments are, as far as possible, expected to include analysis of the additionality of Danida support. It should be noted that the assessments of the different types of levels of results must consider the processes and procedures that has defined the achievements, so as to be able to identify the important points and lessons. The evaluation is expected to include the following:

• To what extent have planned outputs and objectives of individual research projects, special studies, centres or networks been achieved?

• What is the quality level of produced research?

• What is the nature of the collaboration established between Danish and developing country research institutions? Has Danida support been decisive in establishing this collaboration? Might different forms of collaboration have emerged in the absence of Danida support?

96


• Has sufficient attention has been given to building research capacity (at the level of individuals/institutions/country where relevant), and have such efforts have been successful?

• Have sufficient efforts been made to disseminate research findings, and have findings been communicated in a way that promotes use in partner countries, internationally and/or in Danida?

• How and to what degree have research findings been put to use – internationally, in Danida programmes and/or by national partners?

• How appropriate/plausible are assumed causal pathways from research and/or capacity building outputs over behavioural outcomes to wider impacts?

• How adequate are the monitoring systems and follow up requirements imposed by Danida?

• How (if at all) is coherence and synergy between different channels and modalities (competitive grants, PRCP project grants, research centres, networks, BSU) ensured?

EfficiencyThe criterion of efficiency can be seen as a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results. It is not expected that a full cost-benefit analysis can be carried out. However, it is expected that the evaluation will assess if resources have been put to good use, as well as the strengths and weaknesses of differ-ent types of activities and also differences between different modalities of support (i.e. the South-driven pilot projects versus the North-driven larger strategic FFU projects, or activities under the Centre-contract). The evaluation is expected to include the following:

• What is the level of resources employed in the process of administering and monitoring support, and is this appropriate? This will include a comparison between different channels / modalities.

• Has division of labour between the FFU, UFT, the Danida Fellowship Centre, the embassies and partner institutions been appropriate – for individual modalities and in terms of the mix of modalities?

• Have projects been undertaken as planned, i.e. followed their time schedule, using the resources planned, delivering outputs as planned.

• Has sufficient attention has been given to ensuring harmonisation with other donors’ support?

Impact and SustainabilityImpacts and sustainability can be seen as being interrelated in the sense that impacts relates to the wider and longer-term effects, and sustainability to whether effects and achievements will be sustained over time. Impacts of support to development research include the positive and negative changes produced by the support (i.e. research and capacity building projects), whether they are direct, indirect, intended or unintended.

97


In relation to support to development research, this could be translated into the following questions:

• What differences has the cooperation, capacity building and/or research projects made to the participating researchers and the participating institutions?

• What differences have the findings and/or the application of the results had with the field of research/the sector/the local end users?

• Who has benefitted from the improved research capacity (direct beneficiaries such as those researchers who have been trained, as well as the indirect beneficiaries, such as their students) and how?

• Who has benefitted from the application of the findings to help solve development problems and how?

It is not expected that the evaluation will be able to address these questions systematically. However, it will be expected to explore the possibility of conducting example-based impact assessments, exploring the questions for a few multiple-phased projects, with due consideration of the limitations of such examples.

5. Methodology

The evaluation will be carried out in accordance with the Danida Guidelines for Eval-uation (Danida, 2012) and the OECD/DAC Evaluation Quality Standards (2010). As part of the preparation of the evaluation, the Evaluation Department has commissioned a study of recent evaluations of likeminded donors’ support to development research, which will provide an overview of objectives and methodological approaches. The contracted evaluators may use this study as well as their expertise on the subject matter and evaluation methodology to develop a suitable methodology during the inception phase. The evaluation is expected to assess different types and levels of results with consideration of the processes leading to them. This will pose substantial requirements on the collection of data, both in terms of quantity and quality. By implication, it is expected that the evaluation will need to build on different analytical methods, and utilise different types of information, separately and in combination, to answer the different questions. As a minimum, the methodology should incorporate the principles, elements and considerations outlined in this section.

It must entail a combination of desk studies, bibliometric analysis and primary data collection in Copenhagen and in the field study countries, as well as distance interviews with stake-holders in other countries, and is expected to be based on both quantitative and qualitative methods of analysis. Triangulation of both data and methods is of core importance, and the inception report must describe how this will be ensured. The following elements will be required in the overall methodology:

• A literature review to place the evaluation into the context of development coop-eration, and of the research institutions in Denmark and in the partner countries. It will include a review of documents related to Danida support to development research, policies and priorities of Danish development cooperation as well as

98


partner countries, and other donors’ experiences and policies regarding research support. The Issues Paper and overview of relevant evaluations developed in preparation for this evaluation may guide part of this review.

• Quantitative description of the outputs of supported research projects, fact finding activities (minor studies), networks and centres, in terms of publications, articles in scientific journals, books, conference papers etc., as well as “second rank outputs” such as patents. Much of this information will be available from monitoring information and is summarized in the Issues Paper prepared for the evaluation (though this does not cover 2011). However, it is expected that it will be necessary to supplement with a survey administered to representatives of the supported projects etc, including in order to identify outputs superseding project completion. A draft format of this survey must be included in the inception report.

• Bibliometric analysis on publication and citation, so as to provide an input to the assessment of knowledge production and pick-up.

• Quantitative description of outputs related to capacity building, including number of master degrees, PhDs and Post-Docs financed, number of local staff trained (e.g. lab-technicians, teachers etc.), course-material developed, upgrade of physical research facilities, etc. Again, this information is available in part through monitoring information and summarized in the Issues Paper, but will need updating and supplementation with a survey (e.g. to assess the degree to which e.g. PhDs and Post-Docs continue their academic careers).

• Survey administered to stakeholders in contact with the supported centres and members of supported networks as well as recipients of competitive research grants (in Denmark and in partner countries).

• Interviews with key informants including Danida staff, partner country authori-ties, FFU, DFC, research institutions and individual researchers (in Denmark and partner countries) as well as the intended users of the research outputs. These interviews will be conducted with the aim of validating and substantiating quantitative findings, as well as to assess effects and issues related to relevance, efficiency, sustainability and impacts that are best addressed qualitatively. Particularly the processes that frame supported activities (selection criteria, follow-up, incentive structures etc.) and issues related to relevance and quality of outputs and outcomes (research and capacity building) will primarily need to be addressed through interviews.

• Field studies in the two selected countries. At least 50% of the supported projects in the field study countries will be assessed in depth, and evaluators should present a well-founded proposal for selection criteria, as well as considerations regarding the implications of the proposed selection. It is important that the selection ensures a certain level of representativeness as well as possibilities for drawing lessons, e.g. regarding successful practices, for future support. In addition to assessing the specific research projects field studies should explore how national institutions are involved and gain from research cooperation through specific projects, networks and centres. In Tanzania, particular attention will be given to the demand-driven pilot research cooperation projects.

99


• In-depth analysis of a further number of research projects (from non-visited countries), exploring issues related to quality and use of research results, as well as management and processes. Proposed selection criteria and projects meeting these criteria should be presented in the inception report. Again, it is important that the selection ensures a certain level of representativeness as well as possibilities for drawing lessons for future support. Further, it will be required that priority is given to larger projects including those that are in their second or third phase in order to gain the longer time perspective relevant for research impacts, and that at least two projects are PRCP projects (from non-visited countries).

As indicated, different mixes of data and methods of analysis will be required to assess activities of different purposes (or combinations of purposes). It is expected that the technical proposal for the evaluation will include proposed strategies and indications of methodologies for assessments with respect to the various previously mentioned purposes of the support. These methodologies will then have to be further developed in the inception report.

As the evaluation is expected to work with a combination of different analytical tools, rendering information with different levels of coverage and depth (i.a. in depth case studies and broader compilations of quantitative information), care must to taken to maximise the mutual benefit and value added of the information gathered. The analytical strategy is thus expected to consider how to use the information so as to enhance validation and triangulation throughout.

In addition to providing precise and internally valid conclusions regarding support as it has been implemented, it will important that the evaluation – through attention to the role of context (thematic, institutional, Danish, partner country etc.) – reflects on the likelihood that “lessons learned” will be applicable more widely to future Danish support to development research, and that recommendations reflect such considerations. In other words, in formulating recommendations, attention must be given to the likely extent of external validity of conclusions (or conditions under which conclusions are likely to carry over), and how this influences the applicability of the different recommendations, from the overall strategic level to the more specific levels, and for different actors, as may be relevant.

Comments to these ToR and a further elaborated proposal for the methodology, approach, work plan and organisation of the evaluation are part of the tender criteria for the selection the evaluation consultant.

6. Outputs of the Evaluation

The outputs of the evaluation include: i) an inception report, ii) an evaluation report and, iii) relevant process documents and data (an evidence data base).

An Inception Report in draft(s) and final version (not exceeding 30 pages excluding annexes), based on desk study and a first round of interviews in Copenhagen. The incep-tion report shall include a comprehensive presentation of the context of the evaluation and a thorough description of the evaluation design. This will include project selection criteria and an outline of the content of projects selected, an evaluation matrix with indication of data sources and coverage, triangulation strategy, analytical methods etc.,

100


as well as implications of choices made. Descriptions of how and to what degree the analysis will allow the evaluation to assess both different areas and levels of support, different modalities, as well as the overall picture of the achievements of the research support, should be included. The inception report will also suggest if any changes to the evaluation questions are appropriate, and present a detailed work plan to facilitate the logistics of field work in advance. Further, an outline of the expected structure of the evaluation report must be included as well. The draft and final version of the inception report must be submitted to EVAL for comments and final approval of the latter.

An Evaluation Report in draft(s) and final version (not exceeding 60 pages excluding annexes and with cover photo proposals). The evaluation report must include an executive summary of maximum 10 pages, introduction and background, presentation and explanation of the methodological approach and its analytical implications, and findings, conclusions and recommendations. Systematic referencing to gathered evidence should be made throughout.

The draft(s) of the evaluation report must be submitted to EVAL for commenting. As part of this process EVAL will invite comments from Evaluation Reference Group and other stakeholders. The evaluation report will be made public once printed by EVAL.

The full text of the ToR can be found on http://evaluation.um.dk/

101

Annex B Evaluation Matrix

Relevance The criterion of Relevance relates to the extent to which the objectives of an intervention are consistent with beneficiaries’ requirements, needs, overall priorities and partners’ and donors’ policies, strategies etc.

Evaluation Question Proposed IndicatorsMeans of Verification (source/method)

1. Is the composition of the portfolio of Danish support relevant in view of partner country policies and strategies?

Level of coherence with country policies and strategies (examples)

Document review (Burkina and Tanzania)

SWOT analysis workshop/focus groups (Burkina and Tanzania)

2. Is the composition of the portfolio of Danish support relevant in view of Danida policies and strategies?

Level of coherence with Danida policies and strategies (examples), at formulation and currently

Document review

3. Are the objectives of specific pro-jects/networks/centres/studies relevant in view of knowledge gaps and/or information needs and, where relevant, research capacity needs within the substance area and countries addressed?

Evidence of no duplication/overlap

Indication that relevant knowledge/information is not already available/existing

Grant Application Forms

Key stakeholder interviews


4. How and to what degree is the relevance of the overall portfolio of projects and mix of modalities continuously ensured (e.g. by responding to changing Danish or partner country priorities etc.)?

Changes in portfolio composition in response to policy changes

Document review

5. How appropriate is the approach for formulating research themes which guide FFU annual calls for applications. Are the mechanisms fit for purpose – in terms of ensur-ing relevancy, particularly in terms of usefulness (to partners and/or Danida)?

Evidence of compliance with criteria for participation, demand-driven, gender sensitive (for example)

Review of procedures and application process

6. How appropriate and relevant are the different channels and modalities, individually and compositely? Where relevant, this assessment should be made in view of the principles set forth inter alia the Paris Declaration on Aid Effectiveness ((particularly alignment and ownership), and in terms ensuring that research is demand-driven.

Share of projects being led by non-Danish organisations

Online survey (all countries)


Document review

102


7. How appropriate are the different strategies for ensuring and enhancing relevance to end users, both in terms of content, partner involvement, dissemination etc.?

Number of projects conducting end-of-project events

Level of interaction with non-research stakeholders/partners

Number of projects relevant to end-users

Inclusiveness of dissemina-tion options

Online survey

Project site visits (Burkina and Tanzania)



Effectiveness The criterion of Effectiveness relates to the extent to which the intervention’s objectives were achieved, or are expected to be achieved taking into account their relative importance.

In the present case, this links to assessing the delivered outputs and, to the degree possible, outcomes – in terms of research/knowledge produced, disseminated, put to use etc. and research capacity built – for individual projects and to the degree possible also at portfolio level.


8. How appropriate/plausible are assumed causal pathways from research and/or capacity building outputs over behavioural outcomes to wider impacts? (Previously Q13)

Deviations in implementa-tion of original plans

Document review

Key stakeholder interviews (Selected projects in Burkina, Tanzania, DK and selected non-field countries)

Online survey

9. To what extent have planned out-puts and objectives of individual research projects, special studies, centres or networks been achieved?

Level of compliance with indicators

Document review (Annual and Completion Reports for selected projects)


Online survey

10. What is the quality level of produced research?

Number of publications and citations in Peer review Journals

Bibliometric analysis

11. What is the nature of the collaboration established between Danish and developing country research institutions? Has Danida support been decisive in establish-ing this collaboration? Might different forms of collaboration have emerged in the absence of Danida support?

Roles and responsibilities of collaborators

Number of collaborations that would potentially have been established without Danida support


Online survey

103


12. Has sufficient attention has been given to building research capacity (at the level of individu-als/institutions/country where relevant), and have such efforts have been successful? (Was capacity built as intended?)

Performance/capacity improved within targeted institutions


Online survey

13. Have sufficient efforts been made to disseminate research findings, and have findings been communicated in a way that promotes use in partner countries, internationally and/or in Danida?

Number of projects with an appropriate dissemina-tion strategy/action plan implemented

Document review

Key stakeholder interviews ((Selected projects in Burkina, Tanzania, DK and selected non-field countries)

Online survey

14. How and to what degree have research findings been put to use – internationally, in Danida programmes and/or by national partners?

Evidence (examples) on use of research findings in follow-up activities (research, production, marketing etc.)

Document review (second and third phase projects)


Danida staff (agriculture)

Online survey

15. How adequate are the monitoring systems and follow up require-ments imposed by Danida?

Performance-based release of funds

Level of compliance with reporting formats (projects)

Document review (progress/completion reports, accounts)

DFC/Danida financial/ monitoring reports


16. How (if at all) is coherence and synergy between different channels and modalities (competitive grants, PRCP project grants, research centres, networks, BSU) ensured?

Existence of mechanisms in place for ensuring interaction, sharing of experiences/information/learning


Efficiency The criterion of Efficiency can be seen as a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.


17. What is the level of resources employed in the process of administering and monitoring support, and is this appropriate? This will include a comparison between different channels/ modalities.

(two questions)

Level of budgets and human resources allocated to administration and monitoring tasks

Timely reporting (monitoring and financial reports)

Timely feed-back and follow up on progress (reports)

DFC/Danida financial and administrative reports


104


18. Has division of labour between the FFU, UFT, the Danida Fellow-ship Centre, the embassies and partner institutions been appro priate – for individual modalities and in terms of the mix of modalities?

Level of duplication/ omission of tasks

Task descriptions/ToR/ guidelines


19. Have projects been undertaken as planned, i.e. followed their time schedule, using the resources planned, delivering outputs as planned?

Level of compliance with budgets/time schedules/indicators

Document review


Online survey

20. Has sufficient attention has been given to ensuring harmonisation with other donors’ support?

Information on multiple funding included in application forms

Number of joint/collabora-tive donor initiatives

Guidelines for grant application

Document review (including annual reports from relevant institutions)

Impact and Sustainability Impacts and sustainability can be seen as being interrelated in the sense that impacts relates to the wider and longer-term effects, and sustainability to whether effects and achievements will be sustained over time.


21. What differences has the cooperation, capacity building and/or research projects made to the participating researchers and the participating institutions?

Changes in rank/position/career development (researchers)

Changes in performance of the institutions (activity levels, partnerships, funding, networks)

Document review (including annual reports from institutions)


Online survey

22. What differences have the findings and/or the application of the results had with the field of research/the sector/the local end users?

Level of changes observed (new initiatives started following the project)

Document review (including annual reports from institutions)


Online survey

Project site visits

23. Who has benefitted from the improved research capacity (direct beneficiaries such as those researchers who have been trained, as well as the indirect beneficiaries, such as their students) and how?

Number of direct beneficiaries

List of indirect beneficiaries

Nature/type of benefits

Document review


Online survey

24. Who has benefitted from the application of the findings to help solve development problems and how?

(linked to Q13)

List/type of beneficiaries and functions

Document review


Online survey

2 Asiatisk Plads

DK-1448 Copenhagen K

Denmark

Tel +45 33 92 00 00

Fax +45 32 54 05 33

[email protected]

www.um.dk

ISBN: 978-87-7087-760-2

e-ISBN: 978-87-7087-761-9

EVALUATION OF DANIDA SUPPORTED RESEARCH ON AGRICULTURE AND NATURAL RESOURCE MANAGEMENT 2006-2011