Outcome Mapping Reflecting and learning from applications in eastern Africa INTERNATIONAL INSTITUTE OF RURAL RECONSTRUCTION A webinar on the book’s content

Outcome MappingReflecting and learning fromapplications in eastern Africa

INTERNATIONAL INSTITUTE OF RURAL RECONSTRUCTION

A webinar on the book’s content and some feedback

Julius Nyangaga Edward Ontita

Janet NyaoroCharles Muchunguzi

1

Contents

Introduction

The OM book process: survey, cases, cross-case analysis

Key issues/messages

Way forward

Advantages reported; Demand for OM in the region?Challenges to learning and application

2

Panelists’ comments on IIRR, the write-shop Process and some feedback on content

Introduction

IIRR Book: … Produced at the close of a four year period during which IIRR led a series of training workshops –The Evaluation Hub Project

… Contains some info from a survey on the understanding and use of OM and a review of a few selected application experiences in the region

3

Introduction

Outcome Mapping : a project management design, implementation, monitoring and evaluation focusing on the outcomes of intended results.

Interest in Outcome Mapping by R & D processes because it provides a system for more effective support to outcomes that enhance the possibility of intended impacts

Outcomes = defined as the immediate, intermediate effects of a program’s on society (behavioral changes in individuals, communities, organizations, and institutions).

4

IIRR and the OM Write Shop

International Institute of Rural Reconstruction (IIRR) aim: … improving the quality of lives of the rural poor in through sustainable integrated, people-centered development through capacity development and practical field applications.

In 2011 and at closure of the 4-year evaluation capacity project, with focus on Outcome Mapping, IIRR, ILRI and Measure Africa facilitated a write shop process for Outcome Mapping experiences in eastern Africa

5

… 99 participants of past OM training sessions by IIRR, ILRI and Measure Africa

The OM book process: survey, cases, cross-case analysis

The book is not a manual or a demonstration of perfect application experiences but a presentation of key lessons from the cases and a brief analytical observation on challenges and successes of OM use in the region.

Started with survey of OM users and applications in the region

OM Case selection and write-ups

In book there is a Cross-Application Analysis

6Target readers: Project Managers and M & E Professionals

The OM cases in the book

Referencename

Full project title Implemented by

Napier Grass Napier grass smut and stunt diseases in East Africa

Kenya Agricultural ResearchInstitute, Kenya

Zoonosis Reducing the risk pathways andvulnerabilities of wildlife-related diseases using an eco-system approach to health in the cattle corridor of Uganda

Department of Biological Sciences, Makerere University, Uganda

Climatevariability

Adaptation to the impact of climatevariability on food and health security in the cattle corridor of Uganda

Africa Innovation Institute in collaboration with Makerere University, Uganda

DrumNet Building a GSM enabled Information,Communication and Transaction System for Small Holder Farmers in Kenya

Pride Africa, Kenya

7

The OM cases in the book

Referencename

Full project title Implemented by

Reto-o-Reto Better policy and management optionsfor pastoral lands: Assessing trade-offsbetween poverty alleviation and Wildlife Conservation

International Livestock Research Institute (ILRI), Kenya

Safe Water Safe Water Kiosks Project International Institute of Rural Reconstruction (IIRR), Kenya

FAO Evaluation Regional Support Programme for the Coordination and Capacity Strengthening for Disaster and Drought Preparedness in the Horn of Africa – an external review using the Outcome Mapping approach

FAO East Africa Regional Office,Kenya

8

Key issues/messages from OM application in the region

“ … during evaluation ….selected Boundary Partners developed their own performance indicators and (assessed) their own performance.”

“Outcome mapping ‘enhanced’ project implementation ..”

“… By identifying and involving Boundary Partners in project activities, ownership of the process was cultivated …”

Observations on WORKING WITH PARTNERS from the write-shop discussions:

9

OM Advantage(s) …. attracting a demand for Capacity


..in advocacy,… “the project uses Focus Group Discussions among the Boundary Partners to enable people to define their own destiny.”

“In … stakeholder analysis at the inception of projects, key Boundary Partners are identified and involved in priority setting, identification of key constraints and possible strategies.

(They) agree on the roles, work plans and methodologies.”

“… Several respondents used the concept of “Boundary Partners” to serve various objectives..

Survey findings:

10

OM Advantage(s) …. attracting a demand for Capacity


Training opportunities, … observation via on-line feedback

Is there is demand for OM Capacity …?

How can OM training may be scaled out in Eastern Africa?

How can more actors be reached with more training and more effectively?

11


Understanding and using OM concepts and terminology

Term Definition/ Explanation

Penned local Terms/phrases

BoundaryPartners

Change PartnersTarget Beneficiaries

IntentionalDesign

Program/Project PlanProgram/Project Design

ProgressMarkers

-Qualitative, Outcome indicators/milestones-Moving indicators (not static)-Living measures

Strategy Map

Program/project Implementation plan/guide (activities/outputs)

… use of more easily understandable terms

12



On-line discussion on understanding OM concepts, language

What are the most appropriate ways for agreeing on more acceptable, local

(Kiswahili?) equivalents of OM concepts?

- Is there value in translations of the OM concepts or the entire OM manual?

- Who should be involved? Why?13



Use of Organizational Practices

This is the least used of OM practices “… only four of 24 survey respondents.

In write-shop discussions, the trainers … “present OM organizational practices in their training”, but … there is a lack of follow-up application.

One participant comment at the write-shop “ignoring organizational practices can lead to paternalism - you are only focused on influencing others, not considering that you need to (be sensitive to demands), grow and develop yourself.”

14


Integrating OM with other M & E approaches

38% of the survey respondent indicated that they had adapted Outcome Mapping to their contexts

Book content, participants finding it necessary to integrate

50% have integrated OM with other monitoring and evaluation frameworks.

17% combined OM with Logical Framework Analysis (LFA),13% users integrated it with Result Based Management, and4% integrate it with a Theory-of-Change method.

One (survey) respondent said, “Logical Framework Analysis still remains the main design & monitoring tool, but results and indicators are formulated in different ways.” .. Including Outcome Mapping.

15



… Views on integration: …

Respondents tried to balance between their own needs and donor requirements. For example, a project may have used the Outcome Mapping approach for data gathering and developed reports using the LFA approach.

In the Napier grass diseases case, Outcome Mapping was specifically brought in to understand and track interim changes in behavior that affected the control of diseased grass, … while

the scientific investigation into a disease resistant variety (the desired impact in the project’s log frame) was the subject of continued scientific investigation.

16


LFA outputs LFA activities Related Boundary Partners

OM Progress markers

1. Disease inaffectedAreas Mappedand clones collected for evaluation

-Surveys on distribution of diseases-Collect disease resistant clones-Evaluate for resistance

Researchers -Involve other stakeholders in site selection and surveys-Establish clone plots-Screen and selected tolerant varieties

Farmers -Support on farm collections, and demonstrations-Self/community based screening

Integrating OM with LFA; Napier case example

Narrative Verifiable indicator

LFA Goal Changes in farms or household nutritional, health and economic status

LFA Purpose Characterize disease, develop community participation … , build local capacity to manage disease, seek and share tolerant varieties; ..resulting Napier grass productivity and dairy Production

17


LFA outputs LFA activities Related Boundary Partners

OM Progress markers

2. Information on geneticdiversity, managementStrategies available to stakeholders

Improved resistant clones, training manuals and information tools developed and made available

Researchers - .. sharing information, tolerant materials and receiving feedback from other stakeholders (identification of tolerant materials…

Extension -Participating in the selection of tolerant materials-Helping farmers access information and clean planting materials-Organizing information on management strategies of stunt and smut

Farmers -.. Reporting incidences of forage disease -.. Using public forums to raise awareness (among themselves)

Integrating OM with other M & E approaches; Napier case

18



Lessons:- By integrating, it is possible to demonstrate how desired changes in behavior and relationships … facilitate solutions to challenges and better demonstrate progress (impact).

Integrating OM with other M & E approaches; Napier case

Should we integrate OM principles and information in other operational project

frameworks?

Do we have adequate capacity to integrate?

Describe such integration. What models? 19


Institutional alignment to OM: accepting and utilizing …

From book, survey …

“(OM) is a relatively recent approach, it has been difficult to convince colleagues, donors, partners, and community on its value”

One respondent said, “I have a challenge of getting colleagues to change and see Outcome Mapping as an option.”Another stated that “the challenge is reaching out to policy makers and donors to explain the value of Outcome Mapping and getting them put aside resources … or allow a budget for sharing information or tracking change.”

One added that they had to combine Outcome Mapping with other methods because of “stakeholders’ resistance.”

20



How do we support organizational and institutional acceptance of Outcome

Mapping as a Program Design,and useful M & E System?

Institutional alignment to OM: accepting and utilizing OM

21


Use of Outcome Mapping Journals

Adaptation example: The Climate Variability Project in Uganda

Progress Markers (indicators)(Number and description)

Evaluate achievement; achieved? How well?

Evaluative comment: why the evaluation? What evidence

Expect to see…

1. Attending platform meetings and participate byraising questions and concerns on food and healthsecurity

10% One (1) platform of representatives of 4 farmer categories established- Plans are underway to establish 3 levels of platforms; a) community-based, b) sub-county level (at resource centers),and c) district level

2. Sharing information in platform on past and currenttrends and impacts of climate variability on theirpractices and production

10% During FGDs and household surveys, farmers presented information on the evidence of climate variability; climate change trends; and impacts on their practices and production

22



From book, survey The Climate Variability Project …

Progress Markers (indicators)(Number and description)

Evaluate achievement; achieved? How well?

Evaluative comment: why the evaluation? What evidence

Like to see…

4. Developing implementation strategies and plans, allocating …. roles and assignments that will ensure implementation of the suggested practices

0% These will happen when the platforms become functional

5. Implementing suggested strategies in their farms, pastoral and fishing sites, even if on pilot basis with targeted areas of farm sites

0% The documented strategies will be shared in the platforms and piloted in second year with other candidate options, in participatory manner with communities at the resource centres and designated demonstration sites

23



“ … other projects found collecting info from boundary partners cumbersome or difficult … so project staff gathered what was required during face to face meetings)”

24


How do we support monitoring processes in Outcome Mapping?

Which Systems, Structures and Tools would work? Any useful adaptations?


25

Way forward

Further training-“.. 63% of survey respondents

- Technical follow-up with focus should be on how to select Progress Markers and facilitate the collection of information.

-Local and national partners further afield should also be trained so that all project members are on the same level while implementing Outcome Mapping.”

Follow up support-.. “17% of survey respondents.

-An Outcome Mapping expert(s) to visit and give advice, encouragement and facilitate sharing of ideas, challenges, lessons and best practices.”

Suggestions from survey respondents, from write shop

26

Way forward

Reading materials-“ …8% of survey respondents: … inadequacy of available reading material with actual examples and success (application) stories.

-Request for templates that can be used during monitoring and evaluation, and “friendlier” methods for those who cannot write to make them feel more engaged in the monitoring and evaluation.”

27

What are YOUR(other)

views on the way forward?

Way forward

28

End, … asante sana

29

For a copy of the IIRR Om book, write an email to [email protected], or [email protected]

mailto:[email protected]

mailto:[email protected]

Documents

Outcome Mapping Reflecting and learning from applications in eastern Africa INTERNATIONAL INSTITUTE OF RURAL RECONSTRUCTION A webinar on the book’s content