27
DFID only. Not for distribution or quotation. A Review of Analytical Work Underpinning the Development of CAPs This Review was jointly commissioned by the Drivers of Change Team and the Performance and Effectiveness Department of DFID. Key findings ..................................................................................................... 1 Introduction ...................................................................................................... 3 CAP processes in DFID ................................................................................... 4 Analysis in DFID .............................................................................................. 4 Analytical approaches under review ................................................................ 6 Planning the CAP process ............................................................................... 8 Identifying knowledge gaps.............................................................................. 8 Timing of analysis ............................................................................................ 9 Use of internal and external expertise............................................................ 10 Analytical instruments in CAP processes....................................................... 11 Drivers of Change ............................................................................... 12 Risk Assessment ................................................................................. 13 Change Forecasting ............................................................................ 15 Change Impact Monitoring Tables ...................................................... 16 Scenario planning ............................................................................... 16 Analysis of change in a PRS environment ..................................................... 17 Combining long-, medium-, and short-term perspectives............................... 18 Reorientation of the programme .................................................................... 19 Multi-donor work ............................................................................................ 20 Regional Assistance Plans ............................................................................ 21 Annex 1 - ToR for a joint PED and Drivers of Change Team initiative: “A Review of Analytical Work Underpinning the CAP” ........................................ 23 The Review was carried out by Erin Coyle, with David Booth (ODI) and Alison Evans. Sincere thanks to all of those who gave their time. i

A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

A Review of Analytical Work Underpinning the Development of CAPs

This Review was jointly commissioned by the Drivers of Change Team and the Performance and Effectiveness Department of

DFID.

Key findings .....................................................................................................1 Introduction ......................................................................................................3 CAP processes in DFID ...................................................................................4 Analysis in DFID ..............................................................................................4 Analytical approaches under review ................................................................6 Planning the CAP process ...............................................................................8 Identifying knowledge gaps..............................................................................8 Timing of analysis ............................................................................................9 Use of internal and external expertise............................................................10 Analytical instruments in CAP processes.......................................................11

Drivers of Change ...............................................................................12 Risk Assessment.................................................................................13 Change Forecasting............................................................................15 Change Impact Monitoring Tables ......................................................16 Scenario planning ...............................................................................16

Analysis of change in a PRS environment .....................................................17 Combining long-, medium-, and short-term perspectives...............................18 Reorientation of the programme ....................................................................19 Multi-donor work ............................................................................................20 Regional Assistance Plans ............................................................................21 Annex 1 - ToR for a joint PED and Drivers of Change Team initiative: “A Review of Analytical Work Underpinning the CAP”........................................23

The Review was carried out by Erin Coyle, with David Booth (ODI) and Alison Evans.

Sincere thanks to all of those who gave their time.

i

Page 2: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Key findings DFID faces a challenge in developing Country Assistance Plans (CAPs)

that make a clear link between analysis of the possibilities for pro-poor change, and the design of programme interventions. There are some examples of how analysis has affected decisions at sector or project level, but few more substantial examples. Even where there has been a conscious attempt to use drivers of change analysis to support the prioritisation and streamlining of programmes, in retrospect offices say that they should have been more systematic and “ruthless” in following through the implications of this analysis.

There is a tendency to rely heavily on advisory expertise for analysis of

country context and change processes to feed into the development of the CAP. However, advisors (and others) typically share their understanding only informally. In consequence, working hypotheses and assumptions about how change is likely to come about in the short, medium and long term are sometimes not set out explicitly in writing. This has important consequences for transparency and institutional memory.

In some cases, views about change and country context are shared only

within the office. There is rarely a robust challenge function built in to the CAP process. There are, however, examples of country offices that are consciously seeking to improve understanding of change processes in collaboration with a wider group of national stakeholders.

There is often a weak link between different tools or approaches used in

CAP development. Change forecasting and Change Impact Monitoring Tables (CIMTs) are sometimes developed before there has been adequate analysis of medium and long-term change processes and obstacles to change, acquired through drivers of change, ‘forcefield’, or risk analysis. Without this basis of understanding to build upon, however, change forecasting becomes a less-than-optimal tool.

There is a need for more clarity on the function and audience of CIMTs.

Ideally, the CIMT is the place where long-, medium-, and short-term factors are integrated into a “causal story” about what changes can be expected and how these relate to the achievement of the MDGs. However, most country teams reviewed did not allow enough time to develop a meaningful CIMT.

1

Page 3: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Overall, CAP processes are most effective when there is initial clarity across the office about the purpose of the exercise, and the identification of knowledge gaps. Strong leadership from the Head of Office has proved essential in ensuring coherence and focusing on prioritisation. Dedication of some resources to managing the CAP process has increased efficiency.

Timing of analytical inputs is crucial. Processes have been more effective when offices have set out a work plan that indicates key decision points, and ensured that relevant analytical findings were available at these junctures. Several teams commented that they would have preferred more time to think through the implications of drivers of change work. Understanding the fundamental change processes in a country or region requires some months, and it takes time for the findings of studies such as drivers of change to be properly assimilated by teams.

Regional Assistance Plans have suffered somewhat from a lack of clarity

among key stakeholders as to the audience and purpose of the RAP. Expectations have changed during the course of the RAP, with subsequent difficulties for teams. RAP teams have generally avoided undertaking dedicated analytical work on regional change processes, and have also avoided change forecasting/CIMT exercises.

2

Page 4: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Introduction 1. The Performance Effectiveness Department (PED) and the Drivers of Change Team jointly commissioned this ‘Review of the Analytical Work Underpinning the CAP’. The purpose is to consider what kind of analytical work country offices presently undertake to support the development of their CAP (Country Assistance Plan), and how this feeds into programme design.1 This report will be supported by a second output, an Information Note to be used alongside revised CAP Guidance material. 2. The current CAP Guidance produced by PED outlines how country offices should develop the various components of the CAP. Key elements of the CAP that encompass analytical work include Risk Analysis and Change Forecasting. Within Change Forecasting, “forcefield analysis” should provide an understanding of the forces likely to support or constrain pro-poor change. More recently, there is growing interest within DFID country offices to understand the “drivers” of pro-poor change, in a way that can inform the development and revision of the CAP. 3. To date, there has been no attempt to integrate the discrete analytical components of the CAP. Given this, PED and the Drivers of Change Team decided to work together to consider how the various elements of the CAP fit together, and how these can be integrated with other relevant approaches including Drivers of Change. Underlying this is a desire to strengthen the process whereby country offices acquire a deeper understanding of how to support change that will benefit the poor, and successfully translate that understanding into strategic priorities within the CAP. 4. The Review covers a selection of CAP and Regional Assistance Plan (RAP) processes.2 RAP processes raise particular issues, addressed at the end of the report. Some of the CAP processes reviewed were conducted before the CAP guidance was finalised; others are still in their early stages. The Review does not dwell at length on issues that are primarily about the CAP process in general, nor is it intended as an assessment of the quality of analysis underpinning CAPs or RAPs. Rather, the aim is to consider how analysis has been produced and fed into CAP design, which approaches have been used and how.3

1 Terms of Reference for the Review are attached at Annex 1. 2 Including Bangladesh, Burma, India, Nepal, and Vietnam in Asia; Ghana, Kenya, Malawi, Mozambique, Nigeria, Tanzania and Zambia in Africa; and the Brazil CAP (which has since been superseded by the Latin America RAP), and RAP processes in the Caribbean, Middle East and North Africa Department (MENAD), and the Balkans. 3 The Review draws on existing reviews of CAP processes and the Drivers of Change work in DFID, as well as country specific material and interviews with country offices. The Performance Assessment Resource Centre in particular has produced various reports on the CAP process. Please see www.parcinfo.org for more information.

3

Page 5: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

CAP processes in DFID 5. CAPs are intended to set out how DFID aims to contribute to the achievement of the MDGs in various countries, and how DFID will work as part of the international development effort to support a country's strategy for reducing poverty. They include a framework for annual assessment of DFID's performance in implementing the plan. Offices are encouraged to consult a variety of stakeholders in their CAP process, and the final document should include sections on the country challenge, the risks to achieving the PRS goals, what DFID has learned, DFID’s medium term plans and programme resources, and an annual plan. 6. The Performance Assessment Resource Centre (PARC) recently produced some reflections of CAP processes in DFID, to which the reader is referred for more general information on CAPs.4 PARC notes that country teams experience conflicting messages from the centre about the CAP. On the one hand, there are requests for more comprehensive analysis; on the other hand, there are enjoinders to keep the CAP simple and understandable. This tension was evident in this Review. 7. PARC also notes that CAP processes serve several purposes besides strategic planning.5 This Review found that the view of the office (or the Head of Office) on the primary purpose of the CAP process influences greatly the use and value of different analytical tools. Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical tools and approaches, while a lack of clarity on purpose may lead to inefficiencies and confusion. RAP processes in particular have suffered from changing expectations among DFID stakeholders. 8. The Review found that most Heads of Office are aware that the CAP processes is adaptable, and can be as ‘heavy’ or ‘light’ as required. Programmes with a small spend tend to feel that a ‘heavy’ CAP process would be inappropriate, which influences both the amount of resources dedicated to the CAP process in general and the process of improving understanding in particular.

Analysis in DFID 9. In general, DFID’s advisors are considered responsible for remaining intellectually current in their area of expertise and thus able to contribute to the office’s analytical needs on demand - including for the CAP process.

4 Performance Assessment Resource Centre, ‘CAP lessons to consider in preparing revised guidance’ February 2004; and Mark Keen and John Kawalek for the Performance Assessment Resource Centre, ‘Learning Lessons from DFID’s CAPs as part of DFID's performance system from an “Organisation Development” (OD) perspective’ December 2003 5 For instance team building, establishing or improving relationships with partners, public relations, developing a framework for portfolio cleaning, strategic (or business) planning in support of a PRS, or deep reflection on the role of DFID in a difficult environment.

4

Page 6: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Importantly, however, staff interviewed for the Review found it difficult to articulate how advisors gain the understanding that they bring to bear on the CAP process. Although this informality appears to be relatively common practice, the lack of transparency in such processes has serious implications for institutional memory and for accountability. 10. Where analysis is informal or working assumptions are not explicit, as is often the case in smaller programmes, there may be problems with institutional memory. Decisions may be made without transparent justifications. Staff turnover may result in previous implicit understandings being lost. The Vietnam CSP Review, for instance, noted that previous decisions on DFID’s engagement in Vietnam had a poor paper trail. ‘This has made it hard…to gauge on what basis decisions were made and therefore how to respond to these and move forward.’6 11. The Balkans CAP manager also noted that DFID is presently overly reliant on individual memory and filing systems, which reduces the efficiency of CAP processes – good information management will help to ensure that officers have ready access to existing analysis. 12. In terms of challenging assumptions about how change will come about, the Review found that country teams gain confidence in their analysis mainly through internal consensus building. An explicit challenge function is rare (see Box 1 below). There does not appear to be a formal requirement for the CAP to be challenged by colleagues at headquarters, except at Director level. Public consultation exercises were considered ‘analytical’ by many of the interviewees in that they test the conclusions of the CAP in the public domain. However, while public consultation exercises serve important purposes, they are not always designed to provide a rigorous challenge to the assumptions about change that underpin the CAP. Box 1. Challenging internal analysis Mozambique: Three thematic working groups came up with possibilities for donor support and Change Impact Monitoring Tables (CIMTs), which were then sifted according to prioritisation criteria established by a fourth group. Although there was little in the way of formal analysis conducted for the Mozambique CAP process, there was (and is) a challenge function built into the working group system. Each working group is led by advisors who work particularly on that area, and each is challenged by leaders of other working groups. 13. Several interviewees commented that DFID advisors tend to be intellectually hungry and thrive on analysis – but that this is not always helpful for the purpose of a CAP or RAP process where decisions need to be taken. There is a need for an evidence base on which to build policy, but this evidence base should be added to strategically and in proportion to the requirements of the CAP or RAP process. 6 Vietnam CSP Review, Julian Gayfer, Alison Lochhead, Adam Fforde, September 2002, p ii

5

Page 7: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Box 2. Focusing on efficiency Balkans: The advisory team had engaged in a lengthy and unstructured process of reflection when a CAP manager was brought in to produce the CAP to deadline. India: The Head of Office aimed for a focused and prioritised programme in India, which meant creating a tightly managed and efficient decision-making process without room for long periods of loose reflection. There was room for substantial formal analysis in this process but it was part of a coherent structure. 14. Another implication of the reliance on advisors is that it poses dangers of compartmentalisation. This Review found that analytical work which falls outside the remit of advisory groups is particularly important and has proved significant in some cases for ‘breaking the advisory silos’.7 The internal tensions that can be generated through such an approach may require the strong leadership of the Head of Office, rather than the CAP manager alone. Box 3. Cross-advisory analysis DFID India: Early in the CAP process an open brainstorming identified knowledge gaps and identified a process and calendar for taking decisions. Despite the enormous amount of literature on India, DFID identified key questions to be answered for policy making. The CAP Steering Group decided that DFID India would be best served by a small number of studies that looked at the overall development of India through a number of non-sectoral lenses.

Analytical approaches under review 15. Although CAPs are expected to address wide range of questions (on aid efficiency, economic development, the poverty profile, and more), this Review focuses on a limited range of analytical approaches, including: drivers of change, risk assessment, change forecasting, change impact monitoring tables, and scenario planning. These approaches all relate to the need to understand what kind of change is likely to happen in a given context, and why, and to identify those changes that DFID will enable or support, as a means to achieve progress towards the MDGs. 16. At present, there is a lack of agreement or understanding within DFID about the optimum role for each of these analytical approaches, and how they might fit together in a CAP process. Ideally, different analytical approaches will be used in such a way as to enable the country team to fulfil the basic requirements of the CAP, including the need to: a) understand the context (or make explicit the existing understanding of the context) in terms of what kind of change is possible and why, b) decide how DFID will intervene, and c) monitor DFID’s interventions. Table 1 below outlines one way of linking the analytical approaches together to facilitate this.

7 Note that although there are no Drivers of Change Advisors, drivers of change work is often managed by Governance Advisors.

6

Page 8: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Table 1. Linking analytical approaches

Need Possible approaches Detail

Drivers of change Identifies those structures, institutions and agents that bring about change in a given situation

Stakeholder analysis

Maps the interests of stakeholders in a given situation

Forcefield analysis Maps the forces that promote and resist change in a given situation

Risk assessment Identifies the risks to poverty reduction/implementation of the PRS

Strategic conflict assessment

Identifies the causes, actors and dynamics in a conflict situation, and maps existing responses to the conflict

Drivers of change/forcefield analysis

Identifies areas where interventions may be ineffective because change is unlikely

Change forecasting Identifies the steps necessary between the current situation and a ‘desirable future’ such as that set out in a PRS, and shows how DFID’s interventions fit with these steps

Scenario planning Forces DFID to consider how its actions would look in various possible futures

Understanding the context These approaches can all usefully inform each other. In particular, drivers of change work should inform the risk assessment. Deciding what DFID can do effectively These should build on the work done to understand the context. In particular, change forecasting should build on risk assessments (and therefore drivers of change work). Monitoring what DFID does This should build on change forecasting (and thus on risk assessment and thus on drivers of change).

Change impact monitoring table

Identifies indicators that allow DFID to tell whether the country is heading towards the ‘desirable future’ and whether DFID’s actions are contributing to a positive direction of change

17. Country offices weight the different elements of the CAP differently, according to context. In some contexts, DFID’s actions may be heavily circumscribed by the existing policy of other actors or agencies; in such circumstances, DFID offices tend to use their existing understanding of the context and focus on the decision-making aspect of the CAP process. In other contexts, high-level frustration with a lack of progress on the MDGs has prompted deep reflective work on change processes, with CAPs leaving room for further work on this theme.

7

Page 9: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

8

Planning the CAP process 18. This Review found that an early planning event is important to establish the purpose and scope of the CAP. Most offices begin by agreeing some kind of work plan for their CAP, although this may shift according to circumstances or even be abandoned. A few CAP processes are strongly structured with plans for findings to be made available at key junctures in order to feed into decision making processes, with a clear CAP deadline in sight (Bangladesh, India, Mozambique, Nigeria – see India’s timetable below). Offices can then ensure that relevant analysis is made available for key decisions, which is good practice. Such a plan has helped to focus the efforts of the team on priority areas, and momentum has been maintained by strong leadership from both the Head of Office and the CAP manager.

Dec Jan Feb Mar Apr M ay Jun Jul Aug SepIn ternal d is cus s ions o f p roces s x x

Preparation of Part 3 x x x

Terms of Reference x x xStud ies x x x xFacilitated s es s ions x

Cons u ltations on p roces s with GoI ---- ---- ---- ------GoI d is cus s ions ---- ---- ---- ---- ---- ------Other external cons u ltations x x x x

CA P Steering Group cons u ltations with HQ ---- ------ xIden tification of +ve and -ve p rio rities x xTheme working groups x x xDrafting of parts 1 and 2 x x x x

India C A P 2003 : P lanner

Diagram 1. India’s CAP Planner 19. Many offices noted the importance of dedicating sufficient resources to managing the CAP process. All those offices that adhered to the initial plan had dedicated resources that managed the CAP process and prevented it from ‘wandering’. Those who did not have an allocated CAP manager have noted in retrospect that it would have improved efficiency and would have freed up the Head of Office or equivalent.

Identifying knowledge gaps 20. Formal stocktakes of existing analytical literature are relatively rare, in large part because advisors are expected to be up to date on relevant literature. Teams are more often asked to brainstorm about what they know and what they need to know. Some (but surprisingly not all) offices used the CAP guidance as a checklist to see if they could provide all the relevant

Page 10: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

information and analysis. Teams noted that the early identification of knowledge gaps is a crucial part of the CAP process. Box 4. Identifying knowledge gaps Burma: Staff members were asked to identify critical questions that needed to be answered as part of the CAP process. The team could answer most of them internally, but in hindsight consider this exercise a crucial part of the process. Zambia: Staff went through CAP guidance early on in the process. The office’s assessment of the PRS identified a number of critical areas needing further work, so they commissioned scoping studies to see if more in-depth work was needed on each area.

Timing of analysis 21. Some teams conducted most of the analysis internally as part of their decision making process and thus did not have a discrete stage for formal analysis (Burma, Ghana, Malawi, Mozambique). Several of these countries have PRSs, and the CAP was deliberately produced after the PRS was completed to ensure that the CAP is informed by the analysis that underpins the PRS. Note however that the PRS does not usually include analysis of change processes. 22. The Review found that where formal analysis was commissioned, the timing of analytical work has often been decided on the basis of the CAP deadline, with formal analytical work set in motion early in the process. Several offices chose to conduct drivers of change studies over relatively short periods of time (less than 3 months) in order to maintain the momentum of the CAP process, while others have allowed studies to take longer in order to capture certain relevant events in country (Bangladesh) or to allow time to build ownership of analysis amongst country stakeholders (Nigeria). 23. Even where timing was structured in advance, several teams commented that drivers of change work fed into their CAP process too late to be sufficiently influential. The drivers of change work constitutes a relatively new way of thinking about country contexts and officers mentioned that they did not have time to fully assimilate the findings of the drivers of change work before the CAP was finalised.8 Box 5. Timing drivers of change work with the CAP process Nigeria: One component of the drivers of change work is specifically aimed at linking the drivers of change work to the CAP process. In addition, the drivers of change work was designed to go some way to satisfy the analytical requirements outlined in the CAP guidance and therefore addresses much of the work needed for change forecasting and risk analysis.

8 This concurs with the findings of a review of country experience with drivers of change (September 2003).

9

Page 11: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

24. Nearly all CAPs and draft CAPs (and RAPs) mention the need for deeper understanding of the political economy, incentives for reform and agents of change. Many offices that did not conduct drivers of change work as part of their CAP or RAP have explicitly mentioned in their final or draft CAPs that they intend to pursue this agenda in the future (Burma, Malawi, MENAD). 25. The Review found that only a few offices have allowed sufficient time for developing CIMTs, while most have rushed this part of the process and found it difficult.

Use of internal and external expertise 26. Offices improve their understanding of change processes in a number of ways ranging from the formal to the informal (see Table 2 below). This applies to both internal (DFID) and external expertise. Table 2. Use of expertise in CAP and RAP processes Internal External

Informal Burma: Starting from low knowledge base, so building team knowledge of Burma through visits, knowledge sharing events, dividing up key readings Mozambique: Advisory staff already well-acquainted with Mozambique compiled ‘libraries’ of key readings for CAP working groups

Zambia: Conducted mini-workshops where both DFID and consultants presented views on exploratory areas India: drivers of change consultant facilitates discussion in office and writes up

Formal MENAD: Used a consultant to draw together the views of internal DFID staff. Process a priority: one-to-one interviews, field visit, workshop, and report. Kenya: Commissioned 3 internal ‘thinkpieces’ from advisory staff on key areas

Nigeria: drivers of change work involves commissioning both an initial broad study from UK based consultants and around 25 position papers from (primarily) Nigerian researchers Ghana: CAP Reference Group. Included GoG, CSOs, DPs who looked at 3-4 drafts of CAP. Considered useful.

27. Some offices interviewed drew heavily on external sources of expertise (consultants) in their CAP processes (India, Bangladesh, Nigeria, Nepal, Zambia). In these instances, offices note that the interaction with consultants must be carefully managed to ensure that the needs of the offices are satisfied. In several cases, DFID attributes the success (or otherwise) of a study to the careful selection of the consultancy team. 28. Several offices identified the use of non-country office DFID staff as invaluable: not only assistance from Policy Division teams (which is felt to be very important and sometimes inadequate) but also assistance from regional advisors, staff who have relevant experiences in other country offices, former staff of the country office itself, and so on.

10

Page 12: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

29. Some offices drew on very little external expertise in their CAP and RAP processes, but instead relied on expertise within the country team (Malawi, Mozambique, MENAD). In some of these cases, teams feel that the reliance on in-house expertise has strengthened ownership of the CAP and prevented information overload in the context of an analytically sound PRS (Malawi). However, as mentioned earlier, relying on advisory expertise sometimes means that assumptions, hypotheses, and analysis of change processes are not transparently documented. This has consequences for institutional memory. The absence of formal challenge functions is also of concern in this regard. 30. Participation in improving understanding of change processes is more limited than participation in the CAP, which appears to be inclusive. Drivers of change studies are often managed by one or two advisors who then feed the findings to other members of the team. Late entry of the drivers of change work into the CAP process means that findings are not always easily transferred to a wider group. However, DFID India has stated that their drivers of change study has seen lasting process benefits (see Box 6 below). This Review found that change forecast/CIMT exercises are not usually open to the whole office; in fact there have been several cases of the CIMT being largely drafted by one person. Box 6. DFID India: Drivers of Change ‘We managed the Drivers of Change study in line with our overall approach to the CAP process; we kept it short and asked the consultants to focus and structure ideas rather than generate them in quantity. DFIDI staff, similarly, were encouraged to think in political economy terms and identify a few key hypotheses rather than to analyse huge quantities of propositions.’9 31. There are usually efforts to conduct public consultation exercises on the CAP. Sectoral change forecasting exercises are sometimes conducted with country stakeholders (Bangladesh, Nepal, Zambia). To date, work on understanding drivers of pro-poor change has tended not be widely shared with governments, although some country offices have shared the analysis with national stakeholders. DFID Nigeria and DFID Tanzania, however, have undertaken their drivers of change work in a way that will build up or be led by national stakeholder constituencies. DFID Tanzania in particular felt that it was important for analysis to be made to serve a wider function than DFID’s planning needs.

Analytical instruments in CAP processes 32. Some country teams rely heavily on internal informal processes in developing their CAPs and RAPs. In some cases, such as Nepal, teams did not use any of the tools set out in the CAP guidance10 and instead developed their own programme of work to satisfy analytical needs. Country teams 9 ‘“Drivers of Change” – how DFIDI is using the concept’, May 2003, DFID India 10 The Nepal team did conduct one Change Forecasting exercise on the conflict

11

Page 13: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

interviewed use a variety of tools besides those under review here. Some common approaches are brainstormings, facilitated discussions, shared readings and SWOT11 analysis. The CSP Review is also important; some offices used it to answer a broad set of questions about the changing country context (Vietnam), while others used it primarily for accountability purposes (Bangladesh). Stakeholder analysis and forcefield analysis were used as stand alone tools in some cases, although they did not appear to be popular and were more common at the sectoral level than the macro level. 33. In general the different approaches to understanding change processes are only weakly linked in CAP processes. Ideally, drivers of change analysis should inform the risk assessment, and both of these should inform the change forecast and the development of the CIMT. At present, however, change forecasting/CIMT exercises do not draw sufficiently on the drivers of change analysis and risk assessment, and risk assessments are often conducted as a somewhat separate exercise.

Drivers of Change

34. Five offices undertook explicit, formal ‘drivers of change’ exercises as part of their CAP process (Bangladesh, India, Kenya, Nigeria and Zambia). In addition, DFID Vietnam was a case study in a multi-country study on pro-poor policy change, driven by ASREP and the Governance Department. In other countries and regions (Ghana, Caribbean), work was undertaken on the political environment (related to the drivers of change agenda although not as broad). There is considerable internal literature on the take up of the drivers of change approach in DFID to which the reader is referred for more information.12 It is worth repeating the point that many offices did not allow sufficient time in their CAP process for assimilating the findings of work on drivers of change. As such, it is often weakly linked to other approaches such as risk assessment or change forecasting. Box 7. Addressing the drivers of change agenda internally Mozambique: DFID addressed the drivers of change agenda internally in CAP working groups. Groups were asked to answer the following questions in their thematic areas: what are the key areas where progress is needed? what are the processes that will drive this progress? what can donors do to facilitate these processes?

11 Strengths – Weaknesses – Opportunities – Threats 12 See ‘What does DoC mean for DFID? A draft approach paper’ August 2003, ‘Drivers of Change: a selected review of country experience’ September 2003 (both DoC team), and also ‘Drivers of Change: reflections on experience to date’ June 2003 (OPM).

12

Page 14: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Risk Assessment

35. All offices have some assessment of risk in their (draft) CAP. Most considered the question in internal workshops, though not all conducted formal risk assessment exercises. According to the CAP guidance, risk assessments address the question of the risks to poverty reduction (or development, or reform, or implementation of the PRS) rather than the question of the risks associated with DFID’s activities (e.g. creating a risk balanced portfolio).13 As such, the assessment of risk is closely related to the understanding of the country context, rather than to the question of DFID’s activities. 36. The Review found that risk assessments do not always link sufficiently closely with other approaches – they are seen as a slightly separate exercise. Risk assessments are occasionally formally based upon drivers of change work and other pieces of work on understanding the country context (Kenya, India, Bangladesh). There are a few good examples of country offices linking the risk analysis and the CIMT (Kenya, Nepal, Ghana – see Box 9 on the following page). 37. Many country programmes identify lack of political will (or similar) as a major or ‘killer’ risk to poverty reduction,14 and most identify some steps that DFID can take towards mitigating this risk. Some offices explicitly address the question of mitigating activities open to DFID (or the international community) as part of their risk assessment – this is good practice and assists in linking the assessment to the CIMT. This may mean that offices are explicit about the fact that DFID has low influence over certain key areas of risk.

In the Middle East and North Africa, the killer risk to development progress is the failure of the Middle East Peace Process, over which DFID has a very limited amount of influence.

In Kenya, the risk assessment highlights three killer risks, and identifies new and ongoing interventions to address all these risks with the explicit caveat that DFID does not have a strong ability to influence these areas.

Box 8. Malawi: killer risks and change forecasting The Malawi CAP process involved a debate about the links between change forecasting/CIMT and the risk assessment. Opinion was divided between those who thought that the change forecast/CIMT should cover DFID’s entire programme in Malawi (because the CIMT is at least partially about DFID’s internal accountability and corporate governance) and those who felt that the change forecast/CIMT should focus on the killer risks in Malawi, particularly ‘lack of political will’ (because of a recognition that the CIMT cannot provide a complete picture of all DFIDM’s activities and should focus on the key areas).

13 Note that scenario planning is more closely related to the question of risks associated with DFID’s activities. 14 ‘Killer risks’ are defined as those with high impact and high probability

13

Page 15: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Box 9. Ghana’s Change Forecast The Change Forecast Table in Ghana’s CAP is based on their Risk Assessment, which in turn draws on a multi-donor risk assessment.15 Using the key risks to set plan change forecast steps Summary of risk analysis Addressing key risks: objectives (or plan change

forecast steps) for future support16 GoG expenditure does not benefit the poor

⇒ GPRS implementation with pro-poor, sustainable development focus

Failure in macro-economic management

⇒ Economic management and pro-poor growth

Fiduciary risk ⇒ Effective PFM strengthening Weak public sector ⇒ Public sector reform Lack of government accountability to citizens

⇒ GoG accountability and responsiveness

MDBS may not deliver ⇒ Donor harmonisation & predictable funding

Marginalisation of vulnerable groups ⇒ Mainstreaming action on vulnerable & excluded HIV/AIDS undermines development ⇒ Strengthened HIV/AIDS response Conflict & regional instability ⇒ Conflict prevention & sub-regional strengthening

“Once the key plan change forecast steps had been identified, the next step was to select the annual change forecast indicators. To enhance convergence with the GPRS support and MDBS processes, indicators and milestones jointly agreed with GoG and other development partners were incorporated wherever possible.” Ghana’s CAP part III has the following CIMT headings as a result:

Key risk identified in the CAP

Plan Change Forecast Step

Annual Change Forecast Indicators for 2003(updated annually)

DFID supported interventions (updated annually)

Annual Assessment of progress

15 This information is derived from ‘DFID CAPs: Learning from the Experience of DFID Ghana’, DFID Ghana, June 2003 16 The full wording of the plan change forecast steps is contained in the CAP Part III which can be viewed on the DFID Ghana Insight page.

Key steps in Ghana’s Change Forecasting: Key poverty strategy outcome ⇒ Key risks ⇒ Plan Change Forecast Steps ⇒ Annual Change Forecast Indicators ⇒ DFID-supported interventions ⇒ Annual review of progress.

14

Page 16: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Change Forecasting

38. Most country offices interviewed have attempted a change forecasting exercise on some level, even if in some cases it was not taken forward very far (see Table 3 below). DFID Nepal noted that their Strategic Conflict Assessment contains many of the same elements as change forecasting. Regional offices have not undertaken change forecasting exercises. Table 3. Change Forecasting Exercises CAPs RAPs

No CF exercise to date17 Burma, Kenya, India MENAD, Balkans, Caribbean

Sectoral level CF done Bangladesh, Nepal, Zambia

Country/regional level CF done

Brazil, Ghana, Malawi, Mozambique, Vietnam

39. The change forecasting guidance was criticised by several teams as being very difficult to understand. Indeed, some of the teams that did conduct a change forecasting exercise noted that it was extremely resource intensive as teams familiarised themselves with this approach (Ghana, Malawi). Other teams did not have particular difficulties with it and seemed to find it a useful way to work given enough time (Mozambique). 40. This Review found that change forecasting as set out in the current guidance was perceived to be less useful in countries without a PRS or PRS-like initiative (Burma) or where the PRS is criticised as failing to develop a (viable) vision of the country’s future. As one officer stated, ‘If the country does not have a viable, nationally-owned long- or medium-term vision of where it is going, it is not helpful for DFID to create this vision for its own purposes.’ 41. Some teams decided not to attempt to forecast change in a difficult and complex environment: in Bangladesh, this decision was made on the basis of the drivers of change study which said that it would be difficult to forecast change in this environment. However, it seems the best use of the change-forecasting tool is not to attempt to predict the future, but to map the steps to a ‘desirable’ future such as that stated in the PRS. It is possible that this is not clear from the guidance, in which case this message could be strengthened. 42. In most offices, change forecasting exercises are only weakly linked to other aspects of the CAP process such as drivers of change studies or risk assessments. Ghana has explicitly linked its Change Forecasting exercise to the Risk Assessment (see Box 9 above). Ensuring a tighter link between the risk assessment and the change forecasting exercise will help DFID to focus its activities appropriately. 17 Nigeria and Tanzania have not yet conducted CF/CIMT exercises as it is early in their process. India and Kenya have both produced CIMTs although India’s was not integrated into the final CAP. Kenya did not conduct a country-level change forecasting exercise.

15

Page 17: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Change Impact Monitoring Tables

43. The development of indicators to monitor DFID’s support to change processes has been varied. In part, it seems that many staff are not clear about the purpose of the CIMT, either as part of the decision-making process of the CAP or as a reporting and monitoring instrument within the country office and corporately. There is a need for more clarity on expectations around this issue. 44. The Review found that in a few cases, teams have developed one overarching CIMT to cover their programme in summary (Ghana). It is more common for teams to develop three or four CIMTs based on the priority areas of their CAP (Malawi, Mozambique, Nepal). In some countries with PRSs and strong donor coordination, DFID has drawn heavily or exclusively on jointly agreed indicators for change (Ghana, Tanzania, Vietnam). Some teams have not produced CIMTs (Bangladesh) although they have provided relevant monitoring information in less unified form. Regional programmes have not yet developed indicators. 45. Most teams did not plan to devote a significant proportion of time to the CIMT (although they may have ended up spending longer than they had planned). Mozambique is unique in that it specified early on in its CAP process that it would focus on the CIMT through its monitoring and evaluation working group. This seems to have helped them in that they did not find it as troublesome as some other country offices. 46. As noted earlier, there are sometimes quite weak links between the risk assessment and the CIMT. This affects the way that ‘killer risks’ are answered in programming. Because change forecasts and CIMTs are not always based explicitly on risk assessments, a killer risk that is identified in the risk assessment may not find its way through into the CIMT (see Box 8 earlier). The CIMT also provides an opportunity to draw together the long-term view of change (10 – 20 years) with short- to medium-term views of change (annual to 5 years). However, this potential is only realised in a few cases (see Tables 4 and 5 later).

Scenario planning

47. This Review found that few offices have undertaken scenario-planning-type exercises to date, although there is growing interest. Some consultants have included scenarios as part of their drivers of change studies, which has been useful primarily for presentation and pedagogical purposes (Bangladesh, Kenya). In Nigeria, scenario planning is being planned as a way of involving stakeholders in country in a discussion of Nigeria’s direction of change.

16

Page 18: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

48. There is little evidence to date of scenario planning fulfilling its primary function of assisting with risk management.18 Offices may routinely plan for base-case and high-case scenarios but this is broad-brush. The Burma team intends to conduct scenario-planning exercises to work out contingency plans for sudden changes (especially improvements) in the context. Box 10. Scenario Planning Zambia: DFID developed three possible scenarios based on whether or not Zambia goes off-track with the IMF, and shared these with the WB and IMF. The scenarios assisted DFID in thinking clearly about the implications of possible off-trackedness for DFID’s work in Zambia. In this case scenarios were used as an influencing tool as well as a planning tool. Palestine: The Palestine team conducted a brief scenario planning exercise to consider DFID’s options against four possible futures for Palestine in 2010. The final step in the workshop was to “assess the viability of our strategic options in the light of the different scenarios.”19

Analysis of change in a PRS environment 49. The Review found that many offices take seriously the idea that the CAP is a business plan for the DFID’s role in delivering the national PRS. The principle of support to the national PRS is enshrined as a fundamental principle in paragraph 1 of the CAP guidance, and offices have been clear that they interpret the guidance as telling them to ‘start from the PRS’ where one exists. 50. This being the case, several offices have drawn their strategic objectives primarily or wholly from the PRS itself (Bangladesh, Ghana, Malawi, Mozambique, Tanzania, Vietnam).20 Indeed, the Head of DFID Mozambique stressed that the existence of national PRS processes means that DFID no longer has licence to decide its activities independently of the PRS. 51. The Drivers of Change Approach Paper sets out the rationale for engaging with the drivers of change agenda in a PRS country, noting that, “…the existence of PRSP processes only increases the urgency of understanding and responding to the underlying incentive structures that govern policy in a country.”21 A few offices have been exercised by possible tensions between the PRS approach, with its emphasis on ownership and support for nationally-led strategies, and drivers of change work, which implies DFID developing its own analysis of change dynamics in a country. 18 Interview with Tamsyn Barton, contact point on futures and scenario planning. 19 Palestine CAP scenarios, June 2001 20 Note however that PRSs are often poorly prioritised themselves. The Head of DFID Bangladesh noted that the PRS does not really help DFID to prioritise its actions, but it is useful to be able to present work as supporting the PRS. 21 Drivers of Change Approach Paper, p 2

17

Page 19: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Increasingly, however, drivers of change is seen as enabling DFID to better understand the factors that could inhibit the successful implementation of the PRS, and helping DFID to focus its efforts in support of PRSs. 52. The Mozambique office, for example, is clear that while DFID’s support to the PRS is a fundamental principle, the CAP needs to be underpinned by an analysis of which processes will influence the outcomes – and notes that this analysis is not present in the PRS. The Nepal CAP presents a clear statement of its opinion on this question (see below). Box 11. Nepal: uncertain implementation of the PRS ‘There is great uncertainty about the extent to which it will be possible to implement the PRSP. There will be difficulties due to capacity constraints and resistance to reform, but the most serious risks related to the conflict and the political uncertainties. … Notwithstanding these risks and uncertainties, we believe that the case for supporting the PRSP is a strong one. …The main risk and uncertainties relate to the current circumstances in Nepal and cannot be avoided by working outside the PRSP framework.’

Combining long-, medium-, and short-term perspectives 53. The CAP is a short-term (annual) and medium-term (3 to 5 years) planning instrument, while some of the analytical approaches reviewed rely on, or imply, a long-term vision of change in the country (change occurring over 10 to 20 years). Equally, some of the pre-existing analysis or national PRSs that offices draw on have a long-term vision of the country’s development. This Review found that country offices have had varying degrees of success in drawing these approaches together into a coherent short-to-medium-term approach. 54. Some offices have been quite clear about the way that these different ‘lenses’ are nested or fit together. In Mozambique’s draft CAP, for instance, the CIMT has the following headings: Table 4. Mozambique’s CIMT headings Medium term change (10-15 years)

Short term change (3-4 years)

Annual Change Forecast indicators

DFID supporting interventions within the next year

Annual assessment of progress

55. In other countries, the perspective beyond 10 years is not given such visibility. In Kenya, for instance, although the long-term view of change over 10 or 20 years is implicit in the drivers of change analysis, the CIMT does not directly identify change over the long term. The CIMT does link the short and medium term (annual changes over the next four years) but there is no explicit vision of change beyond the lifetime of the CAP. This is not uncommon, and indeed there are questions about the validity of tabling indicators beyond the lifetime of the CAP.

18

Page 20: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Table 5. Kenya’s CIMT headings

2003 Situation 2007 Forecasted Change

Annual Change Forecast

DFID Supporting Activities

56. Some country offices were clear that they consciously chose not to put effort into developing a long-term vision of change for the country.22 Some teams felt uncomfortable attempting to forecast change (and develop indicators) beyond the lifetime of the CAP. As mentioned earlier, some felt that there is no point in DFID developing a vision of a country’s development in isolation. Also, in conflict countries such as Nepal, the planning horizon is very short; developing a long-term vision in such a situation would not necessarily be the best use of DFID’s resources.

Reorientation of the programme 57. The CAP is seen to give an overarching framework for assistance that establishes priorities and will guide decision making in future.23 Some DFID programmes have tended to suffer from an unfocused plan with a wide range of possible interventions. There is considerable inertia in some country programmes that is difficult to shift even with high-level strategic analysis and with substantially improved understandings of change processes. There are several factors that influence the design of a new programme, including the history of the programme, the work of other donors, and the advisory staff available. 58. Analysis of change process is not always a key factor in programme decisions, and this Review found that offices face a challenge in linking analysis of the possibilities for pro-poor change with the design of interventions. A recent scoping review on ‘Incentives for Adopting a Drivers of Change Approach’ also noted that the drivers of change approach is not yet making a tangible impact on programme selectivity.24 In Nigeria, for instance, the drivers of change work has introduced a new model of how change happens in countries, with potentially wide ramifications; the implications have yet to be worked through into programme design. 59. The CAP process itself does not guarantee greater focus or selectivity. This Review found that where the CAP process has been used to bring greater focus to the programme, this has been driven by the Head of Office or equivalent. In the Balkans, for instance, the RAP was specifically undertaken to radically increase the focus of activities to adhere to objectives and purpose.

22 Other officers noted failed attempts to get DFID to think over the long term and acknowledged that some teams find it very difficult. 23 Country offices work through the implications of the priorities set out in the CAP over the entire period of the CAP, and decisions about priorities are worked through into practice on an ongoing basis. 24 Alison Evans, ‘Incentives for Adopting a DoC Approach: Results of the Scoping Review’, presentation February 2004

19

Page 21: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

60. This Review found that some offices have explicitly set out to improve their focus as a result of analysis of change processes, but have struggled. In India, for instance, the Head of Office suggests that it would have been helpful to be more systematic in realising the implications of the drivers of change work in the portfolio cleaning process. Although this requires considerable effort and discipline, she recommends: ‘Be ruthless. The importance of an area for poverty reduction is not enough if DFID can't be effective in the area or make a difference to poverty reduction.’ 61. Most offices struggled to think of activities that they had decided not to engage in as a result of their understanding of change processes. Strong examples given include:

In Mozambique, the analysis of the likelihood of change in the

education sector led DFID to decide that its support should primarily be through other agencies.

In Bangladesh, the Head of Office noted that there are powerful arguments for engaging in urban development work but DFID could identify no strong drivers of change. Instead they discovered a complex and dysfunctional institutional structure that meant the chances of successful intervention would be relatively small.

62. Teams have also noted examples where DFID will continue to work despite the slim possibility of reform. In Zambia, DFID will not withdraw its support to public-sector reform, despite the recognition that change is unlikely in the medium term, because public sector reform is a key potential driver of change in other areas.

Multi-donor work 63. A few CAP processes have been consciously developed with an eye to the planning processes of other donors, although the extent of actual read-across is not always apparent. The extent of joint working has an impact on the use of joint analysis and sharing of analytical work. Box 12. CAP links with other donors Ghana: The CAP Advisory Group included representation from other development partners (as well as government and civil society). Tanzania: Donors are working towards a joint assistance strategy in support of the second iteration PRS. 64. This Review found that some CAP processes have used jointly prepared risk assessments (Ghana, Vietnam) and built on common assessments of the PRS where one existed (Malawi, Tanzania). In some countries, jointly prepared Strategic Conflict Assessments are also used (Nigeria).

20

Page 22: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

65. In many cases, offices felt that they had shared their understandings with a wide variety of partners (including other donors) through public consultation exercises on draft CAPs. However, there are only isolated examples of offices that shared DFID’s underlying analysis of change processes. In Bangladesh, DFID’s drivers of change work included a short summary piece that has been widely circulated amongst stakeholders in country. Nigeria’s drivers of change work will feed into the World Bank ‘Policy Notes’ initiative and will feed into USAID and EC medium term planning processes. Box 13. Shared analysis in a Joint Assistance Strategy Tanzania: There are plans to create a joint assistance plan for the like-minded donor community, in support of the second iteration of the PRS. A key part of this vision is the use of joint analytical work that supports Tanzanian analytical capacity. Donors are considering whether and how World Bank standard analytical tools can be turned to the needs of other agencies. The donor community is also working to assist the government in building a sound analytical base for the PRS, the foundation of the joint assistance strategy. Drivers of change work is also being conducted under a multi-donor steering group.

Regional Assistance Plans 66. This Review found that RAP teams have all chosen to be ‘guided but not bound’ by the CAP guidance. RAP processes reviewed had strong elements of team building or public relations, as well as strategic planning. In the case of DFID Caribbean, for instance, the RAP process was seen as a significant way for DFID to engage with regional partners. In the case of MENAD, the RAP was seen as a way of pulling a team together and signalling the existence of a new department corporately and more widely. 67. Regional teams have usually chosen to undertake RAPs for purposes other than corporate governance and accounting requirements, although in some cases expectations are changing as the process evolves. This has led to confusion in teams as to what expectations they should be trying to meet. An early planning meeting involving all stakeholders may have gone some way to avoiding this problem. 68. RAP processes have not involved formal analytic studies of change processes. As a rule, they have either built on the experience of the regional advisory team (Balkans, Caribbean) or have acknowledged explicitly that the team is starting from a low knowledge base and that one of the RAP’s aims is to improve this base (MENAD). 69. RAPs do include risk assessments, but have avoided change forecasting or the development of CIMTs. Offices gave several reasons for this:

21

Page 23: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Problems of attribution where DFID is a small player There are no regional strategies or visions on which to base the

forecast – and country strategies in only a few cases Regional changes are very hard to identify and may not be visible for

several years RAP fatigue in a long process

70. It is worth noting that many country programme face similar difficulties, particularly with the latter two problems. It is important for all plans to address directly and clearly the question of long-term change and short-term change, and to be clear what can be monitored over the short term. Equally, all teams need to allow sufficient time for developing indicators, rather than treating this exercise as an afterthought.

22

Page 24: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

Annex 1 - ToR for a joint PED and Drivers of Change Team initiative: “A Review of Analytical Work Underpinning the CAP”

Background 1. There has been growing interest in improving the quality of the analytical work DFID does to support its planning, policy development and implementation, as reflected in the Country Assistance Plan (CAP). There is also growing interest within DFID country offices to undertake work on understanding the “drivers” of pro-poor change, in a way that can inform the development and revision of the CAP. 2. CAP Guidance produced by Performance and Effectiveness Department (PED) provides details about how country offices should develop the various components of the CAP. In terms of analytical work and understanding, key elements of the CAP include Risk Analysis, and Change Forecasting. Within Change Forecasting, “forcefield analysis” should provide an understanding of the forces likely to support or constrain pro-poor change. 3. Recently, country offices have expressed interest in further thinking on how the various analytical components of the CAP fit together. In addition, other analytical tools have been developed within DFID including Scenario Planning, which adopts a long term (20 to 25 years later) perspective, and work on understand Drivers of Change, which considers underlying systemic and institutional factors that affect the possibilities for pro-poor change in the short, medium and longer-term. Scenario Planning, Drivers of Change, Risk Analysis and Change Forecasting/Change Impact Monitoring each imply slightly different timeframes and angles of view on change processes, but are all fundamentally concerned with improving the effectiveness of DFID’s support to change that will benefit the poor. 4. To date, however, there has been no attempt to build bridges between these various analytical components and tools, or to consider how they can be used in a more integrated way to inform the development of Country Assistance Plans. 5. Given the coincidence of interests in the CAP process between the Drivers of Change Team and PED, it is proposed that we work together to consider how to strengthen the analytical components of the CAP process, with particular focus on how the various elements of the CAP fit together, and how these can be integrated with other relevant tools and approaches that are being developed, including Drivers of Change. Underlying this collaboration is a desire to strengthen the process whereby country offices acquire a deeper understanding of how to support change that will benefit the poor, and

23

Page 25: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

successfully translate that understanding into a set of focused, strategic priorities within the CAP. 6. In order to do this, it is proposed to jointly commission a review of experiences of country offices to date in undertaking different kinds of analytical work to feed into the development and revision of their CAPs, and to use this review to identify emerging good practice and key issues for consideration.

Purpose and outputs 7. The purpose of the Review is to strengthen the quality and coherence of the analytical work and understanding of change processes that feeds into the development of DFID’s Country Assistance Plans, and to support country offices by identifying good practice and lessons learned to date in undertaking such work. 8. Outputs of the project will include:

A Report of Review Findings of not more than 25 pages including an Executive Summary and recommendations. The Review Report should identify challenges and potential gaps in current practice and suggest recommendations for how to fill these.

An Information Note to accompany the present CAP guidance, aimed

at DFID country office staff (length to be agreed but not more than 5 pages), and based on the Review Report. The Information Note will support country offices by providing them with information, lessons learned, and emerging good practice in undertaking and using analytical work and understanding of change processes to inform the CAP. It is expected that, pending approval by PED, the Information Note will be posted on the CAP Guidance webpage on Insight.

Scope of Work 9. The Review will be carried out through desk-based work, primarily in the form of key informant interviews with country office staff. The Review will cover the experience of a diverse range of country offices, at different stages in the CAP process. Country offices selected to participate in the Review – and staff within those offices to be interviewed - will be agreed between the consultant(s) carrying out the Review and PED/Drivers of Change. The consultant(s) will also draw on the existing review material compiled by PED and relevant documents and information from country offices. Drivers of Change Team and PED will facilitate contacts with country offices as far as possible; however, it is envisaged that considerable “leg work” will be needed to follow-up country office contacts, obtain documents and information, and set appointments for substantive interviews (by telephone, video conference, or e-mail, or the appropriate combination of all three).

24

Page 26: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

10. The review will investigate a range of issues in relation to country office experience of developing and using analytical work and understanding of change processes to feed into the CAP, including: 10.1. The process by which analytical work and understanding of change processes and underlying issues has been set in motion, including timing, including:

What is the view of country office staff and the Head of Office about

the importance of undertaking new analysis or refreshing understanding of change processes to feed into CAP development/revision?

Has there been a review of analysis already available, for example from other donors or institutions, and has this been taken account of and used? If not, why not?

Has there been a consciously directed and managed process of achieving understanding and/or doing analysis? How have decisions been reached on what analytical work is needed?

How have country offices identified their needs for analysis and understanding to feed into the CAP?

How far upstream before the CAP deadline has work been set in motion? How has timing been decided?

What element of the CAP are being given particular emphasis (e.g. Risk Analysis, Change Forecasting, “forcefield analysis”) and why?

What other analytical work outside the CAP framework is being planned to specifically feed into the CAP process (e.g. scenario analysis, Drivers of Change, other)? Is other analytical work being undertaken that is not feeding into the CAP? Why is this the case?

Who in the office is leading the work, and how much cross-office participation and engagement has there been?

How is the work being done and what mechanisms are being/have been used (i.e. specific commissioned studies; a structured learning and reflection process including outside consultants/researchers; other mechanisms or a combination of these)

Has the existing knowledge of staff been tapped? Has existing knowledge of researchers and written material been tapped?

Has there been a process established for sharing the analysis and understanding with other donors? Why or why not?

10.2 The translation of analysis and understanding into programme design, including:

What process has been established to manage the translation of

analysis into a strategic programme design? Who has led this process? If there has not been a clear process for translation, why not?

How have linkages between different analytical components in the CAP and use of other approaches such as Drivers of Change been made? What has enabled or constrained these linkages from being made?

25

Page 27: A Review of Analytical Work Underpinning the CAP · Strong leadership from the Head of Office ensures coherence to the CAP process, and a more effective and appropriate use of analytical

DFID only. Not for distribution or quotation.

How have country offices connected a short-term view of change reflected in the CAP with a longer-term perspective or view about how change will occur over a 10-20 year span. If the office has not developed a longer-term perspective, why not?

How has a link been made between Risk Analysis and programme design. Are risks identified reflected in terms of strategies and activities in the CAP? If “lack of political will” is identified as a risk, is it addressed programmatically? Why or why not?

How has Change Forecasting been derived from the analytical work undertaken, and how do the indicators of change identified reflect the understanding that has been achieved?

What is the relationship between the development/revision of the CAP and existing programme activities? How has analysis and understanding been used to revise, re-sequence or change existing activities?

10.3 Challenges and lessons learned for the future including:

What have been the key strengths, weaknesses and challenges of the country office’s experience to date?

How can the process of deepening analysis and knowledge of pro-poor change feeding into the CAP be strengthened for the future?

How can various components of the CAP and approaches such as Drivers of Change be better integrated? What is emerging good practice in this?

How can the process of developing the CAP be improved to ensure a CAP that is fully informed by analysis and understanding, and that focuses on a smaller number of strategic priorities arising out of this analysis/understanding?

Inputs, Management and Timing 11. The Review will be carried out by a consultant or consultants familiar with DFID’s work in general and the CAP process in particular. 12. The work of the consultant(s) will be overseen on a day-to-day basis by the Drivers of Change Team, on behalf of PED and the Drivers Team.

13. Inputs will include up to 40 person-days. No overseas travel is envisaged. Limited UK travel will be included in the budget. Communications expenses will be budgeted at £500. 14. A draft Review Report should be completed by no later than March 1, 2004. A draft Information Note, based on the Review Report, should be completed no later than March 20, 2004.

26