7/31/2019 ~9281390

1/24

Evaluation of

World Vision UK Programme

Partnership Arrangement

with DFID

November 2010

Jill Edbrooke, Ella Haruna,

Richard Nyirenda, Sarah Thomas

7/31/2019 ~9281390

2/24

Acronyms

ADD Action on Disability and Development

ADP Area Development Programme (World Vision)

CABA Children affected by HIV and AIDS

CCC Community Care CoalitionsCBPM Community Based Programme Monitoring

CBO Community Based Organisation

CIDT Centre for International Development and Training

CoH Channels of Hope

CSO Civil Society Organisation

CVA Citizen Voice and Action (a process of citizen demand led governance)

DAN Disability Awareness Network

DPO Disabled Persons Organisation

FBO Faith Based Organisation

FTI Fast Track Initiative

GSF Global Schools Forum

IATT Inter Agency Task Team on Children Affected by HIV & AIDS

IDDC International Disability and Development Consortium

INGO International Non-Governmental Organisation

IPM Integrated programme Model

LEAP Learning through Evaluation, Accountability and Planning

MDGs Millennium Development Goals

MP Member of Parliament

NGO Non-Governmental OrganisationNPA National Plan of Action

PPA Programme Partnership Arrangement

PPRC Programmes and Projects Risk Committee

ODI Overseas Development Institute

OVC Orphans and Vulnerable Children

PLWHIV People Living with HIV

PMTCT Preventing Mother-to-Child Transmission

PRS Poverty Reduction Strategy

SSI Semi Structured Interview

SHG Self Help Groups

UKAN UK Aid Network

UNESCO UN Educational, Cultural and Scientific Organisation

UNGEI UN Girls Education Initiative

UNICEF UN International Childrens Emergency Fund

VCA Vulnerable Child Advocacy

WB World Bank

WV World Vision

WVI World Vision International

WVUK World Vision UKYA Youth Ambassador

7/31/2019 ~9281390

3/24

3

Contents

Acronyms........................................................................................... 2

Contents............................................................................................. 3

1. Executive Summary....................................................................... 42. INTRODUCTION............................................................................. 62.1 The evaluation team and methodology .......................................... 63. PPA Results ................................................................................... 73.1 Impacts on the lives of beneficiaries .............................................. 7

Strategic Objective 1 ............................................................................. 7Strategic Objective 2 ............................................................................. 9Strategic Objective 3 ............................................................................. 9Strategic Objective 4 ........................................................................... 10

3.2 Relevant policy issues and progress towards achieving policyoutcomes........................................................................................... 113.3 Organisational development of World Vision UK and the wider WVpartnership ........................................................................................ 123.4 Overall health of World Vision UK monitoring and evaluationsystems. ............................................................................................ 134. Innovation .................................................................................... 145. Value for Money........................................................................... 155.1 Financial and programme management, accountability and audit of

PPA funding....................................................................................... 155.2 Adding value through achieving sustainable programme impacts 166. Lesson Learning.......................................................................... 176.1 Introducing evidence based advocacy approaches...................... 176.2 Making child protection more effective ......................................... 186.3 Lessons learned about programming and knowledge management.......................................................................................................... 187. Building Support for Development............................................. 208. Issues to be addressed ............................................................... 22

7/31/2019 ~9281390

4/24

4

1. Executive SummaryThis evaluation assesses the extent to which World Vision UK has been able to progressthe Objectives set out in its Programme Partnership Arrangement with DFID since 2008(continuing until March 2011). The purpose of the PPA was to contribute to poverty

reduction, social inclusion and support for development. The funding supported WorldVision, to accelerate advocacy at the local and national levels as an integral part of itsprogramming including strengthening the ability of poor people to hold service providersand governments accountable. The four strategic objectives reviewed here focused on:

1. Strengthening development effectiveness through improved citizen engagement;2. Mitigating the impact of HIV & AIDS on OVC through improved citizen participation;3. Strengthening the engagement of disabled people in poverty reduction;4. Enabling young people to participate in and influence poverty reduction.

The final objective was added in 2008 and focused on Building Support for Developmentin the UK with young people (a subset of a much wider area of involvement WV UK hasthrough sponsors and churches etc) and received a small proportion of the PPA grant.

During the evaluation, a team of 4 people from CIDT reviewed documentation; interviewedkey stakeholders within WV UK1 and its partner WV organisations, external partners andadvocacy targets; visited programmes in India, Kenya, Uganda and Zambia; and fed backinitial findings through a workshop for WV UK staff.

At programme level, work in the first three objectives is having a direct impact on the livesof children and poor people in a wide variety of ways. Targets have been exceeded onmany of the indicators across the logframe, however it is too soon to quantify sustainableimpacts. World Visions addresses some social inclusion issues (gender, age, ethnicity)through the initial assessment of communities and programme methodology. Through thisPPA, disability and HIV/AIDS are tackled directly and explicitly, complementing wider workon vulnerability and demonstrating how analysis and advocacy efforts on diverse issues

can overlap. The work has highlighted the need for an overarching framework throughwhich to understand vulnerability and identify programming priorities. It has also supporteddevelopment of an Integrated Programme Model (IPM) across the World Visionpartnership and a focus on enabling marginalised people to understand the reasons fortheir exclusion and take measures to hold governments and service providers to account.

Strategic Objective 1 The primary vehicle for this objective is the Citizen Voice and Actiontool (CVA), a process that, crucially is community-owned, with WV offering initial facilitationsupport. This, combined with the establishment of dialogue and relationship betweenservice user and duty bearer, presents a powerful mechanism for sustainability. Progressin developing links from the grass roots to national advocacy initiatives has so far beenhighly context specific strong in some countries and nascent in others - however many

different examples of improved services and access to entitlements are evident and theapproach has supported the shift of focus towards advocacy among programme staff.

Strategic Objective 2 the HIV & AIDS OVC advocacy objective builds on previous workincluding, Channels of Hope (CoH) training and Community Care Coalitions (CCCs). 291CCCs have been initiated and a model of Vulnerable Child Advocacy (VCA) has evolved.There have been achievements in each domain during the PPA, and some innovative workhas been developed and scaled up e.g. Child Parliaments. With careful planning to ensuresynergy rather than confusion with initiatives such as CVA, these approaches arechanging community attitudes and empowering community members, PLWHIV and OVCsto engage in advocacy.

1World Vision UK is referred to in this evaluation as WV UK; WV partners are the country partners belonging

to the WV Global Partnership; and WVI is World Vision International

7/31/2019 ~9281390

5/24

5

Strategic Objective 3 has taken an experimental approach to mainstreamingdisabilityacross selected WV partners. Priority was given to developing initial training (reaching over2,500 staff); using external expertise to support pilot work and learning in areas of servicedelivery such as education; development of external partnerships (a new approach) withnational DPOs; and evaluation. Much has been learned however more time is needed toembed it sustainably and to ensure it is cross cutting in programme design and evaluation.

International advocacy led by WV UK PPA funded staff explicitly supports the work in all3 strategic objectives (tracked through logframe indicators), maximising the potential thatWV programme reach offers for informing policy. Field based evidence and carefullycommissioned research has been used to good effect and WV is a key member of differentgroupings in the UK, including UKAN and the UK Consortium on AIDS and InternationalDevelopment. WV UK has influenced advocacy content and strategies of these and otherkey international partners, targets and donors such as WB, UNGEI, FTI, Global Fund toFight AIDS, TB & Malaria, UN IATT, UNICEF and DFID and larger WV partners.

Innovative work, in the context of the WV partnership, has given staff the knowledge, toolsand skills to facilitate and support direct engagement between the beneficiaries and

governments/ service providers, supporting a strategic shift in approach from direct servicedelivery towards advocacy and empowerment.

Value for money appears good but is difficult to quantify exactly in terms of programmeimpacts and has potential to increase if the work is scaled up and embedded sustainably.Management of funding has been rigorous and transparent and although not influenced bythe PPA, the ratio of fundraising and governance to programme costs has improved overthe evaluation period. Further thought is needed to address high staff turn-over, whichappears to occur across all partners, affecting sustainability and training effectiveness aswell as value for money.

Monitoring and evaluation of the PPA has been intentionally more rigorous than thestandards expected through WVs Learning through Evaluation with Accountability &

Planning (LEAP2

) in order to manage the amount of learning generated and fast pace ofproject development.

Lessons have been learned and applied across the objectives, but particularly aboutintroducing evidence based approaches to advocacy work in different contexts, how childprotection can be improved through more effective analysis of vulnerability, and ways ofmanaging learning and knowledge management to improve programme impacts.

Building support for development in the UK through the PPA focuses on youthawareness and activism, working mainly through partnerships. A small scale, dynamicprogramme of activities has been developed, however at this stage it is difficult to judgethe likely impact. A recent decision to refocus on a new target demographic suggests thatsustainability is likely to be limited.

Overall, progress towards the PPA purpose has been good. Impact is visible andcontinues to develop across the three main objectives, and WV UK has worked hard withWV partners to share learning from this, particularly in terms of advocacy content andusing lessons to strengthen and better align programming. Reach and sustainability couldbe increased further by strengthening links to national advocacy from both internationaland field level. Through the PPA WV UK has been successful in influencing the strategicapproach being developed by the wider WV partnership, thus reaching a massive potentialaudience for scaling-up. Taken together, the flexibility of PPA funding and the innovation itallows has enabled WV UK to punch above its weight both across the WV partnershipand with key external stakeholders, adding value to the partnership strategically.

2 LEAP is the design, monitoring and evaluation framework used by World Vision partners acrossprogramming. The LEAP handbook recognises that processes outlined may need adaptation for use inadvocacy programmes and projects.

7/31/2019 ~9281390

6/24

6

2. INTRODUCTIONWorld Vision works through childfocused, community development, humanitarian andadvocacy programmes. This PPA was intended to enable the organisation to strengthenwork in the areas of citizen engagement with service providers and government; mitigating

the impact of HIV and AIDS on children; mainstreaming disability; and Building Support forDevelopment in the UK3. Overall the first two objectives received around two-thirds of thebudget, although this proportion fluctuated and was not a pre-defined allocation.

Programme Partnership Agreements4 (PPA) were set up by DFID in 2000. However WorldVision UK has only held a PPA with DFID since 2006 making it one of the newestrecipients of this strategic funding. Since 2008, funding has been about 2.5 million/year,around 4% of WVUKs income, which is a low level but none the less critical unrestrictedfunding compared to many other PPA holders.

WV UK has programmes in around 26 of the 97 countries where WV works and the PPAfunded work in 11 of these. Much of the PPA work was carried out within the AreaDevelopment Programme (ADP) framework used across WV, through which WV maintains

a community presence for around 10-15 years through funding from sponsors who receiveupdates on specific children. However, some aspects of the HIV/AIDS strategic objectivewere managed through the WV health teams, which do not always link geographically withADPs and have shorter time frames. In total, 12-13 UK staff (full time equivalent) havebeen funded to lead on and co-ordinate the PPA work, placed within the programme,policy and advocacy teams.

2.1 The evaluation team and methodologyThe evaluation was undertaken during September and October of 2010 by a team of fourconsultants from the Centre for International Development and Training, CIDT, Universityof Wolverhampton.

The methodology included review of key documentation (Annex 3). Semi structuredinterviews were undertaken face to face and by telephone, skype, and email with WV UK,WV country partner and WVI staff, external partners and key stakeholders (beneficiariesand non-beneficiaries)5. One workshop was held with UK staff to review the preliminaryfindings and allow for clarification/ triangulation with field visit results.

The field visits included SSIs, focus groups, workshops and visits to communities inUganda (5 days), Kenya (1 day), Zambia (3 days) and India (5 days). Selection of thecountries to visit was led by WV UK based on the spread of PPA objectives and fundingacross these countries. The WV partner then proposed the communities/ stakeholders thatevaluators met with input and feedback from the Evaluators. The time scale for theevaluation and short lead-in time meant that these choices were pragmatic. Overall the

evaluation team felt able to contextualise the results reasonably well, although it provedmore difficult to meet with or interview external stakeholders in country programmes.

At field level in some countries it is clear that there is both overlap and synergy betweenthe different strategic objectives. In addition, the range of terminology used in practice bydifferent WV partners made it difficult at times for the evaluation team to establish clearattribution between the initiatives funded through this PPA and related initiatives beingdeveloped by WVI or WV national partners.

3A fifth strategic objective centred on work in Latin America was added later in the PPA, but this

work was not included within the current evaluation.4

Subsequently changed to Programme Partnership Arrangements521 face to face/ telephone interviews with WV UK staff, 14 with WVI staff and 12 with partners and

stakeholders (excluding those undertaken on field visits)

7/31/2019 ~9281390

7/24

7

3. PPA ResultsThe ToR ask for an assessment of the results of World Vision PPA activities, includingimpact, on three areas the lives of poor people (including gender relations and socialexclusion issues); relevant policy issues; and overall organisational development of WorldVision itself. This section takes each in turn. Much of strategic objective 4 targets young

people in the UK through development education, discussed in Section 7. Tablessummarise progress in relation to the logframe indicators after each objective is set out6.

3.1 Impacts on the lives of beneficiariesGender and social exclusion are cross-cutting through the first three strategic objectiveswhere WV engages directly with poor and excluded people, particularly children, atprogramme level. Disability and HIV & AIDS are tackled directly and explicitly, while issuessuch as age, gender and ethnicity are already largely embedded through the initialassessment of communities prior to setting up ADPs, as well as through the programmemethodology e.g. setting up Self Help Groups (SHGs) for women, childrens clubs and soon. The explicit focus on disability and HIV & AIDS complements and endorses wider work

on vulnerability, by highlighting the many forms of exclusion that WV is trying to tackle atonce, and demonstrating the ways that analysis and advocacy efforts can overlap. TheIntegrated Programme Model (IPM) that WVI and WV national partners will be introducingin their new strategic plans offers WV an overarching framework through which tounderstand vulnerability, identify its programming priorities and thus staff training needs.

Strategic Objective 1: Development and aid effectiveness at local and nationallevels in at least 9 countries7 are influenced and improved through citizenengagement with poverty reduction strategies and their implementation

Indicator 1 civil society involvement in local government development planning processes (particularly PRSs)in 6 countries2006 Limited/ no influence in 6

i

selected countries and certainvulnerable groups excluded

2010 Civil Society Organisations (CSOs) have demonstrated

increased capacity to participate in and influence national and localpolicy and planning processes

Indicator 2 Citizens in at least 100 new countries are engaged in partnerships for demand led governance atlocal, regional and national levels.06 Limited/ no demand ledgovernance in a 100 communities

2010 Citizens in over 100 communities across 5 countries areengaged in partnerships for demand led governance

Indicator 3 Sustainable Community Based Organisations (CBOs) engaged in local evidence based advocacycampaigns and policy influence.06 Weak CBOs with little/noengagement in policy and localadvocacy

2010 139 CBOs in 69 new communities are engaged in localevidence-based advocacy campaigns and policy influence

At field level, Strategic Objective 1 has been led by the Citizen Voice and Action tool

(CVA), which is the primary vehicle for this objective, a means to mobilise and energisecommunities and raise their awareness of concepts of citizenship, rights and advocacy.CVA is variously seen as a tool, an approach, a model and a value base and is used indifferent ways in different national contexts. It builds on World Visions version of CBPM, apre-existing model developed outside WV8.

A key facet of CVA is that the process is community-owned, with WV offering initialfacilitation support. This combined with the establishment of dialogue and relationship

6The summaries of change from 2006 2010 given for the first 3 Strategic Objectives are selected from the annual self

assessment reports submitted by WV UK. No specific baselines were provided for 2008, except for objective 4, which onlystarted in 2008. Although the details have not been independently verified by the evaluation team, it was possible to confirmelements through samples seen during the field visits and overall, they appear to be reasonable indicators intended to

capture outcomes as well as outputs, and also different levels of change local national and international.7Including Armenia, Bolivia, Brazil, Cambodia, Honduras, India, Kenya, Uganda and Zambia.8In India, although training has been delivered on CVA to update the training already given on CBPM, the ADP staff continue

to use CBPM as the name of the tool they use.

7/31/2019 ~9281390

8/24

8

between service user and duty bearer presents a powerful mechanism for sustainability.The ethos of CVA is as a partnership to bridge gaps in provision; the approach calls forbuy-in from all sides, transparency and a non-confrontational stance. Thus local politiciansand service providers can see the process as an opportunity rather than a threat and beempowered as community champions, whilst communities also make a commitment toplay a specific role in addressing the problems identified. WV has been trainingCommunity Based Organisations (CBOs) and Self Help Groups (SHGs) and somegovernment actors/ service providers to use and understand the CVA methodology andsee the benefits it can bring. Issues to tackle through advocacy are selected by thecommunity, but many countries have focused in on health and education sectors.

One measure of this form of citizen engagement was to build the capacity of, and workthrough CBOs. Whilst the capability and confidence of CBOs is varied, this is an effectivestrategy for generating ownership, sustainability and efficiency. CVA is seen as offeringgood value for money due to the quick wins which can be obtained at local level withrelatively low programming inputs.

During the field visits, it was apparent that there has been a very effective absorption of

CVA training with participants talking confidently about an advocacy process. Much of thetraining and engagement requires functional literacy and participants in some communitieswhere CBOs were not well established or levels of literacy are low, engagement with theservice providers can be intimidating. It was also reported that CVA effectiveness isinfluenced by community culture; in some places advocacy is already part of the culturehowever in more conservative communities not used to questioning authority, the methodcan take longer to bed in.

In Uganda CBOs own the CVA process, replicate it in other neighbouring areas and trainother CBOs in the methodology. There is improved delivery in health services; morehealth workers recruited, removal of ghost workers, more drugs supplied, proper use ofresources of health centres, health centres built and sub-county resource allocation. InKenya CVA has been used to increase transparency and community voice in relation tothe Constitutional Development Fund, devolved funding distributed by local MPs. Inspecific ADPs in Senegal, female representation on rural councils has increased as aresult of advocacy work and the President of Rural Councils became aware that womencan legally own land, which he did not know before. In Armenia CVA community activistsare being elected to government posts. Right to Information legislation passed recently inIndia has led to a step change in what can be achieved using CVA as it represents a realstick to complement the carrot of the benefits of community involvement in decisionmaking and many examples of improved service delivery were mentioned by beneficiariesin some ADPs.

Not surprisingly for a bottom-up initiative, gains at regional and national level are notnecessarily linked to the immediate needs prioritised by local communities. They can also

be more challenging and take more time to achieve because a different level ofrepresentation/ advocacy is needed. In some countries, CBOs are being invited torepresent their communities at higher levels and specific interest groups such as PLWHIVand DPOs are starting to network at county/ regional/state levels. However, in most of thepilot countries, there is some way to go before advocacy initiatives generated from grassroots can influence at national levels. The capacity of WV to undertake national advocacyalso varies considerably and is discussed in the next part of this section 3.2.

Although the methodology is spreading organically in some areas at community level, dueto high demand, there has been a bottle-neck in delivery of the formal CVA Training ofTrainers within WV, two WV national offices reporting a year long delay.

7/31/2019 ~9281390

9/24

9

Strategic Objective 2: The impact of HIV & AIDS on children is mitigated and theirrights upheld in at least 7 countries

9through expanded and strengthened civil society

participation in developing and implementing policies and programmes at community,national and international levels.

Indicator 1 increase the number of community coalitions and faith group providing protection and support to

children affected by HIV/AIDS, and reducing stigma.06 120 community coalitions and 150faith groups providing protection

2010 a further 151 have been established through the PPA bringingthe total number to 291

Indicator 2 Increase, from 28,000 to 84,000, the number of boys and girls affected by HIV & AIDS reachedthrough WV supported projects, are more aware of their rights and access services/support06 28,000 affected are more aware oftheir rights

2010 124,434 children are receiving comprehensive care from over6,000 home visitors

Indicator 3 - at least 50% of the 350 community coalitions are actively engaged in advocacy at their local level

06 no more than 100 coalitionsengaged in advocacy

2010 evidence that the majority of community coalitions are acengaged in advocacy at local level and impacting for example onmarriages and education in African countries

This area of work was well developed prior to the PPA with established partnerships andactivities. 33% of PPA funding has been invested in this objective.

The PPA funded a critical assessment of advocacy related to reducing the vulnerability oforphans and vulnerable children in 6 countries, and designed an approach for tackling theroot causes of these vulnerabilities. A model of Vulnerable Child Advocacy (VCA) hasevolved since 2008 incorporating 5 domains; government influencing, socio-culturalchange in communities, community ownership, child participation and networking. Peereducation processes to transfer learning and promote sharing of experience across theregion were built into the pilot from the outset.

Achievements through this strategic objective have included the development of by-lawson child labour that have been adopted by local government in Uganda, increased schoolattendance and retention, stopping the use of children in ritual cleansing, reducing thenumber of child marriages, enhanced the level of awareness of child rights, child protection

and reduced stigma and discrimination in communities of children infected/affected by HIVand AIDS. Civil society participation has included establishing mechanisms such asAdvocacy Councils whose members recognise the need to work through coalitions andhave identified people to network with including the police, probation and faith leaders.

In WV Mozambique child-led advocacy has been piloted through Child Parliaments in theprovince of Zambezi, using child participation to influence stakeholders work and policy.In Mumbwa ADP in Zambia the chieftain made a pronouncement never to condone earlymarriages in her chiefdom after participating in Child rights and OVC Advocacy training.

Strategic Objective 3: The inclusion and empowerment of disabled people inpoverty reduction is strengthened at local, national and international level.

Indicator 1 - WV progressing disability mainstreaming in development, humanitarian and advocacyprogrammes in at least 20 countries (including Cambodia, Ethiopia, India and Sierra Leone06 Nascent global commitment tomainstream disability through DisabilityAwareness Network (DAN), policydecisions and specialist staff. Initiatives tomainstream disability in 2 countries

2010 WV has progressed disability mainstreaming in 14countries. Technical support and inputs on disabilitymainstreaming given to 34 further countries and partners e.g.disability awareness and inclusion training course delivered to2,500 staff. Publications include Travelling Together andPractical lessons from 4 projects in 3 continents on disabilityinclusive programming

Indicator 3 - Collaborative relationships strengthened and joint work implemented with Action on Disability andDevelopment (ADD) to mutually strengthen and extend programme/ policy reach06 Commitment to develop collaborativeprogramming and advocacy with ADD

2010 WV has developed its relationship with ADD throughdisability awareness training, technical support and evaluations in6 countries. Progress limited by senior staff turn over in ADD

9Including Ethiopia, India, Kenya, Malawi, Mozambique, Uganda and Zambia.

7/31/2019 ~9281390

10/24

10

WV UK had been developing work on disability since 2000, but prior to the PPA no workhad been done to mainstream disability across the WV partnership. Traditionally WV hasnot prioritised partnership working and this approach prioritised involvement of DPOs anddisabled people Funding was therefore used flexibly to ensure that training involved DPOsand disabled people, encouraging partnership working. Recognition of the conceptualapproach taken to address disability in each partner country was important many still usemedical and charity models and influencing attitudes through training was key to theearly stages of the work. Other key aspects included identification and use of champions tolead change processes, and an investment in learning processes/ external evaluations.

At programme level, following the initial trainings, partners have prioritised different areasof disability, including inclusive education, development of work with DPOs andstrengthening the DPOs, staff development and integrating disability into design,monitoring and evaluation.

This was an immensely ambitious objective for the 5 year time frame it was given, andalthough good progress has been made in some countries, further effort is required ifimpact is to be sustainable. In common with other areas of PPA work, senior staff buy-in

has been needed and progress has been affected by staff turn-over and staffing issuesdelaying work). A rich evidence base will come from the work already undertaken (much ofwhich will be evaluated in the remaining 6 months of the PPA) and to achieve the value formoney sought by WV UK and by DFID it will be important to continue developing theapproach in any future PPA, in order to strengthen the IPM.

The coherent linkage between WV UK and WVI approaches, apparent in StrategicObjective1, is not as clear in this area of the PPA, despite the fact that key staff haveworked closely together in the past, so it appears to the evaluation team that WV UK stillneeds to remain a focus point if the momentum generated is not to be lost. Evidence ofinteresting and positive progress can be found in most of the partner countries: in Armeniaa DPO took the training materials, translated them and now delivers disability training toother organisations; in India an impressive range of training and dissemination tools tosupport the work have been developed; in Kenya, work is low key but considerableprogress has been made; overall, huge numbers of staff have already been trained (2,500) however, challenges remain and further work is needed to develop ways of scaling up,rolling out and embedding work based on the social model of disability across countryprogrammes.

Strategic Objective 4: Young people in the UK and the rest of the world are ableto participate and influence others in global poverty reduction both with and onbehalf of children in the worlds poorest communities

Indicator 3 Increase in number of young people who are mobilising to hold governments to account in at least10 countries in both north and south08 WV mobilizing young people in 9

countries (not directly attributable to WVUK)

2010 WV UK PPA supports mobilisation of young people in at

least 6 countries, including Mozambique, India, Armenia andCambodia and is analysing links across several countries to seehow this can be developed further

Different models are being used to mobilise children and young people including ChildrensParliaments in Mozambique and countries in East Africa; in Armenia, Student Councilshave been representing their peers in CVA work. More broadly, children and youngpeoples clubs are involved in CVA and VCA work across several programmes at differentlevels. Additionally, WVUK is supporting a significant piece of work being developed byWVI and external partners to support youth participation in governance processes, due tobe published in 2012, which (for WV purposes) will contribute to delivery of the IPM.

7/31/2019 ~9281390

11/24

11

3.2 Relevant policy issues and progress towards achieving policyoutcomes (Indicators 1.4, 1.5, 2.4, 2.5, 3.2, 3.4)

Indicator 1.4 - 50 CSOs demonstrably increasing their contribution to influencing national government anddonor aid policies through submission to at least 10 aid consultations06 CSOs have limited/no engagement

with government

2010 partnerships and networks in the north and south10

the PPA

is supporting at least 50 CSOs to contribute to and influencenational government and donor aid policiesIndicator 1.5 - Bi/multilateral donor practices on aid effectiveness are changed (e.g. predictability of aid) byCSOs and coalitions in the north and south, including CIVICUS and UK Aid Network06 Increased air flows are unlikely tomake a serious dent into global poverty

2010 working with partners and in networks towards not only thedelivery of promised aid, but critically to ensure that this aid isdelivered effectively.

Indicator 2.4 - 6 sub Saharan countries are increasingly lobbying their governments for the development andimplementation of policies and plans for children affected by HIV and AIDS and their inclusion in developmentinstruments06 limited and variable quality of civilinvolvement in lobbying of governmentsacross 6 countries

2010 community based evidence brought to bear as part of CSOconsortia and advocacy work to lobby governments for policiesthat include children and address the effects of HIV/AIDS on theirlives in 6 countries e.g. through Childrens Councils and CCCs

and multi-sectoral groupsIndicator 2.5 - WV working with civil society have influenced donor policy and funding commitments for childrenaffected by HIV & AIDS of at least 5 bi/multilateral donors06 one bilateral donorwas influenced throughUK Consortium/10%HIV fundingcommitment for OVC

2010 WVUK continues to co-chair the Children Affected by AIDS (CABA) WorkingGroup of the UK AIDS Consortium which succeeded in ensuring that the UKGovernment AIDS Strategy included significant policy commitments for children. TheGlobal Fund Framework on Community Systems Strengthening revised to includeCABA. The Patent Pool was established to reduce costs of paediatric antiretrovirals.UN policies influenced including 2006 HIV Political Declaration and communiqu ofGlobal Partners Forum on children affected by AIDS. Irish Aid programme guidancefor CABA/OVC strengthened

Indicator 3.2 - Policies and practices of 15 UK, European and global NGOs and bi/multi lateral donors showevidenced change through the sharing of best practice in disability mainstreaming06 Limited mention of disability inSPHERE guidelines, mainstreaming indonors, multilaterals and INGOs verylimited, no formal resolution on disabilityin the Global Campaign for Education,

2010 Policies and practices of 12 UK, European and globalNGOs, bi/multi lateral donors and 4 governments show evidencedchange through the sharing of best practice in disabilitymainstreaming

Indicator 3.4 - Disabled children and youth included in an increased number of UN member and donorcommitments to universal primary education by 201506 2005 limited mention of disability inmajor education initiatives FTI,Education for All and in donor inclusionstrategies

DFID education strategy recognises the need for disabilityinclusion, mention of disability in G8 communiqu on education(2009). Educations Missing Millions (WV) used in FTI draftpolicy statement on inclusion. Policy brief requested by UNESCOfor disability inclusion in Early Childhood Education

Policy change is contemplated at several different levels within the PPA work. Local leveladvocacy has been discussed in the first part of this Section, so the focus here is onnational/ regional and global advocacy and policy influencing.

At national levels wider advocacy partnerships are being developed, raising the profile ofWVs work. Where local CVA is difficult or risky, working as part of an umbrella group hasbeen a successful strategy e.g. in Cambodia where MPs have been influenced to analysebudgets for the first time and awareness of the governments responsibilities regarding theMillennium Development Goals (MDGs) has been increased. Efforts are being made toensure that data collected on the MIS is relevant to campaigns/ advocacy priorities andimportantly, to information that the government concerned wants and will use (or evencommission as one Indian State government has done!). PPA work has successfully

7/31/2019 ~9281390

12/24

12

influenced policy in relation to OVCs in several countries e.g. in Zambia, an OVC policyhas been included within the new Strategic Framework for the National AIDS Council(NAC). PPA work has changed the way that many WV national partners have engagedwith DPOs, federations and governments lobbying for disability inclusion.

As an FBO WV UK also contributes to advocacy work from a faith perspective at different

levels. Notable in this PPA is the Channels of Hope (CoH)11

work in the area of HIV andAIDS has developed into a global, grassroots inter-faith response to the pandemic. Itdemonstrates how effective the combination of evidence based advocacy (to churchleaders in this case), training and partnership working can be undertaken in differentnational and faith contexts.

International and UK focused advocacy strategies developed by WV UK have been clearand very effective. There are attributable elements of policy in all of the strategic objectivesseeking to influence policy and the external advocacy partners interviewed value WVhighly as a partner. Aid Effectiveness is a relatively new policy theme, so programmesupport has been prioritised (although WV UK is a founder member of UKAN and PPAfunding supports the work). Contributions made by WV partners to some advocacy

processes e.g. recent inputs by WV Zambia and Uganda to meetings held by the WorldBank on Aid Effectiveness, have demonstrated how their perspectives can change theagenda and create knock on effects in subsequent dialogues in-country and globally.

In contrast PPA funded advocacy on HIV and AIDS builds on many years of work. WV UKhas close working partnerships with high level UN HIV/AIDS policy making bodies,contributing to, and successfully influencing, key decisions as the table above shows(indicator 2.5) e.g. co-funding research with UNICEF HQ on behalf of the Inter Agency taskteam on Children Affected by HIV & AIDS (IATT); as a steering committee member forthe Global Partners Forum on Children Affected by HIV & AIDS;; as co-chair of the CABAworking group the UK AIDS Consortium co-hosted the PMTCT Technical meeting withDFID; participating in the Care and Support WG of the Consortium, it has contributed torecently agreed UNGASS the indicators national governments will have to report against.

In disability, global advocacy has been particularly effective in relation to inclusiveeducation supported by the WV report Educations Missing Millions and now beingstrengthened by interesting research results emerging in India at a grass roots level.Inclusion is often not a focus at local level, so WV UK has been able to influence the FastTrack Initiative (FTI) through reference to good case study material, influencinginternational actors to see cross cutting issues as intensifiers in relation to inclusion, ratherthan stand-alone priorities. WV UK is seen as a leader in this area of work among UKINGOs and has provided useful input for DFIDs education work a focus on girls, disabilityand education resonates.

Internally it has sometimes been challenging for staff with a policy brief to co-ordinate with

programmes and experience of work for the PPA has influenced a decision to merge policyand programmes to facilitate the development of a more solid evidence base fromprogrammes in the new Strategic Plan. None-the-less, the achievements in StrategicObjectives 2 and 3 show WV UK is able to make use of its wide reach to develop verysolid evidence for policy change at international levels.

3.3 Organisational development of World Vision UK and the widerWV partnershipWithin the international WV partnership, a federation of southern and northern CSOs, WVUK is not the biggest office but it can claim to add value beyond its size and partner offices

11Channels of Hope training was developed prior to 2008, but much subsequent training has been funded

through the current PPA

7/31/2019 ~9281390

13/24

13

recognise the strong technical support it can provide. The PPA funds have supported thisvalue added through the experimental programming that has been put in place.

The application submitted under the new round of PPA funding will be further aligned withWVUK core strategy but it is clear that restructuring of organisational strategy has alsobeen influenced by this PPA. The PPA programming has helped WV UK to fully

understand and prioritise a vulnerability emphasis in its new strategy, through an analysisof barriers and focus on child protection. PPA funding has accelerated a shift away fromservice delivery towards integrated programmes with advocacy as a key approach for WVglobally. For example the recent Learning Forum produced a communiqu on the need foradvocacy to be included as a competency in job descriptions.

Within the PPA portfolio, WV contributes a faith based perspective. Its status as an FBO isclearly reflected in the Christian identity of many of their staff and the corporatisation of theChristian ethos through regular devotional meetings where context appropriate. In thisPPA, faith has been most explicit in work on Strategic Objective 2, for example CoH whichworks with religious leaders and congregations on the issue of HIV/AIDS through trainingand developing inter-faith partnerships at a local level. The CoH methodology challenges

faith-based leadership (tailored to different faiths) and mobilizes the infrastructure,organizational capacity, volunteers, and moral authority of local churches and faithcommunities. WVs Christian identity can be an asset or a drawback depending on context.In the liberal culture of Senegal Muslim/Christian dialogue is very open and WV SenegalsChristian identity assists in programming around human rights and sharing of human rightsmessages. In Ethiopia it took 2 years for WV Ethiopia staff to overcome suspicion, beingviewed as a Protestant organisation in strong Orthodox areas.

3.4 World Vision UK monitoring and evaluation systems.Each PPA strategic objective has its own logframe nested within the overall PPAobjectives. The indicators targeting national level policy engagement were perhapsunrealistic given the timeframe and variable advocacy capacity of country offices; however

other targets have been exceeded. The definition of community in the indicators hasbeen hotly debated within WV UK and the national partners making monitoring difficult.

WV UK and the national partners are aware that advocacy is more difficult to monitor thantraditional development projects. One suggestion to address this is the use of milestoneindicators in the logframe to reflect progress, especially on policy influencing. It has alsobeen difficult to capture the changes resulting from the work in strategic objective 3 in anydepth, partly exacerbated perhaps by the widely differing approaches/ entry points thathave (deliberately) been used. Overall there is a proactive attitude to the challenges thathave arisen staff clearly see the value of monitoring, have attempted to share this withCBOs and want to make it as effective as they can. As a result different approaches havebeen explored to strengthen processes/ amend the information gathered.

WVUK monitors and manages the results of its programmes through regular field visits byprogramme staff combined with appropriate technical support and use of WVs globalproject management cycle known as LEAP (Learning through Evaluation, Accountabilityand Planning) which requires the review of biannual programme and financial reports fromall WV offices. This is complemented by qualitative methods including Most SignificantChange, focus groups and surveys. There is some feedback to suggest that PPAreporting and modality could be better integrated with wider WV reporting guidelines asDFID reporting requirements do not fully align with WV internal reporting requirements.

The challenge of community level monitoring was reported by several national partners,including Senegal and Ethiopia. For example management of information collectedthrough CVA has been weak in many countries, taking a long time to aggregate scores

from community level. There were calls for a simplified CVA database which can be moreeasily maintained by community, perhaps only collecting data on key indicators.

7/31/2019 ~9281390

14/24

14

4. InnovationWe have assessed innovation in this section using the context of the World Visionpartnership and programming as a framework. Thus we judge many aspects of thedeveloping work on advocacy as innovative, although to other INGOs they may not appear

to be new. Some innovation is also country specific: what is normal practice in communitymay be completely new to communities in other cultural, social or political context.

There are specific examples of innovative practice emerging from the PPA work in differentareas for example the use of local proverbs in Senegal to promote messages abouthuman rights, the use of score cards in Zambia CVA work (innovative in this countrycontext) and employing transformational development facilitators for work on disability12in Cambodia (responding to needs identified by staff for advocacy skills at this level).

However, it is the wider scale of innovation generated by the PPA, accelerating the WVpartnerships emphasis from service delivery to an integrated programme and advocacyapproach which is strategically so important to the WV partnership. WV Armenia describedan emerging process of empowerment from the grass roots which is freeing up WV staff

at ADP level to engage in regional and national advocacy.The development of advocacy has led to more and wider external partnerships beingdeveloped at all levels, another area of innovation for WV - as one respondent put it thePPA has given us the courage to partner. WV is such a large organisation, it has beenaccustomed to relying on in-house resources to develop its work, but the work involved inthe PPA leads CBOs to develop partnerships with other groups of CBOs, SHGs and singleissue groups, as well as the media and at times local government itself and WV staff havealso developed and worked through wider networks and forums.

For WV, sponsorship tends to be tied to practical programme work, so obtaining the PPAfunding represented a chance to innovate and experiment. In the context of WVmainstream programmes, which use a sponsorship approach structured through ADPs,

the empowerment model that the CVA and VCA represent have been very innovatory,building on existing advocacy work. The opportunity to focus on HIV/AIDS and disabilityhas also created space to explore the extent to which demand-led advocacy and the socialmodel of disability can fit within the standard sponsorship/ADP framework.

Work on disability mainstreaming in particular was designed quite deliberately to exploredifferent approaches to mainstreaming top down, through thematic work e.g. inclusiveeducation, cascading training, using external consultants to mentor are some of theapproaches used. A clear focus on external evaluation of these processes has ensuredstrong reporting on this objective and translation of lessons into tools that are being widelydisseminated to external actors as well as the internal WV audience.

As intended, PPA funding allows the flexibility to influence as an ongoing process unfolds

e.g. in India it was used to scale up the CVA work and then develop a Learning Dialoguewith other countries. Whilst not highly innovative, the use of learning laboratories (periodicmeetings for staff involved in particular areas of work) has been a very effective way ofenabling WV partners to capture and contextualise local learning. The OVC AdvocacyLearning Forum, involving WV staff from the 6 countries involved in VCA, has alsoconsolidated learning regionally and supported wider scale-up. These processes have alsohelped to generate ownership and enthusiasm for new areas of work which mightotherwise be seen as an additional burden.

Taken together, the flexibility of PPA funding and the innovation it allows has enabled WVUK to punch above its weight both across the WV partnership and with key externalstakeholders, adding value to the partnership strategically.

126-8 community mobilisers in each ADP work in 2-3 villages each, staying overnight in villages

7/31/2019 ~9281390

15/24

15

5. Value for MoneyWe begin this section with a caveat about the difficulty of retrospective assessment. Overthe course of this PPA, DFID has become increasingly interested in ensuring that itsinvestment represents value for money to the UK tax payer. This aim is completely

justifiable, but has not been matched by adjustments in the monitoring and evaluationframeworks so there is no easy way to read across from impact to value for money.

Two aspects contributing to value for money are addressed. Firstly, how well WV UK hasmanaged the money it has received and how it has attempted to ensure that the cost baseis minimised. A more qualitative approach is taken to the second part of this section,looking at how successful programme work i.e. sustainable change which impactspositively on the lives of poor people, policies and WV as an organisation, represents gooduse of the money spent and may result in changes on a wider scale than anticipated by theinitial investment. In common with logframes for other PPAs, the logframe agreed betweenWV UK and DFID had no baseline or indicators at purpose level, making evaluation of theoverall impact speculative at best. Although the ToR does not require a detailedassessment of impact at this level, this is also an important consideration in relation to

assessing the value for money offered by this PPA.

Before discussing fiscal and programme management of the PPA, it is important also tohighlight one of the most notable features of WV as an organisation in the context of valuefor money its size. With over 40,000 staff and programmes in 97 countries, WV is by anymeasure a massivebureaucracy. This gives the potential for huge leverage to scale (andthus added value for money) from the small scale beginnings of PPA work, which appearslikely in strategic objective 1, but can also present significant challenges for creating acritical mass/ momentum for change and organisational learning/ knowledge management,as strategic objective 3 clearly shows.

The vast majority of staff are local and not a cost borne by the PPA, so arguably buildingcapacity of the WV national partners in effect the local partners represents sustainablechange adding value to development initiatives led through the PPA.

5.1 Financial and programme management, accountability andaudit of PPA fundingA basic element of achieving good value for money is robust financial management.Overall, financial management of the PPA reflects the rigorous and comprehensivesystems and procedures that are in place across WV.

WV has strong governance structures and procedures to meet the needs of such a largeorganisation. It has a federated structure in which each WV organisation is a separatelegal entity.. All WV national organisations are subject to an annual external audit and WVI

conducts internal audits on a 3-year cycle.The size of WV gives clear benefits of scale in relation to negotiating power andprocurement costs and the procurement procedures are subject to strict protocols to avoidcorruption. A further advantage of scale can be seen when the GBP fell against the US$ in2009 and currency was hedged on behalf of WV offices to maximise favourable exchangerates.

Although not directly attributable to the PPA funding, in common with other organisationsfacing the challenges of the current economic climate, WV has made efforts during theevaluation period to ensure that efficiency is maximised and that funds are well used. Oneexample of this is the reduction of the ratio of fundraising and governance to programmingcosts by over 20% in the evaluation period.

The PPA funds have been subject to quarterly reporting and management through WVUKs Programmes and Resources Information System (PARIS) which incorporates sign-off

7/31/2019 ~9281390

16/24

16

procedures for project approvals and funding commitments and interfaces with WVsglobal financial database. To counter fiduciary risk there are strong management systems,including spreadsheets which track spend and achievement. WV UK set up a SteeringCommittee of relevant staff and manages PPA funding as if it is restricted. The SteeringCommittee informs the Programme and Projects Risk Committee (PPRC) of aspects ofprogramme management that might represent a risk to achievement of programmeoutcomes. In turn, the remit of the PPRC includes scrutiny of WVIs internal audit reportsfor those offices where WV UK has invested funds.

Variances have been closely managed and overall there is no under-spend of PPAbudgets allocated to WV national partners, however some parts of the PPA have beenaffected by under-spending on the activities originally proposed in India for example,staff shortages have led to delays in project implementation and although the funding hasbeen reallocated, the projects initially allocated funds have not achieved the anticipatedoutcomes in the timescale envisaged. In Ethiopia there has been a project history of underspending over each consecutive year since implementation began. Much of the under-spending had resulted from the integrated nature of the project, which meant that activitiesat an NO level were dependent on the National Government.

5.2 Adding value through achieving sustainable programmeimpactsWV UK will ensure that it delivers value for money by achieving results - sustainablechanges that reduce levels of poverty and enable marginalised people to holdgovernments and service providers to account in order to access their entitlements. Muchof Section 3 considers whether this has happened and overall progress seems satisfactoryalthough patchy. For reasons already mentioned, relevant information is not available tovalue this in monetary terms, however for some areas of the PPA it is possible to look atthe direction of change and extrapolate from this. The rest of this section looks at eachstrategic objective in turn.

The work done to introduce CVA has clearly resulted in some early wins which havemade immediate practical (and quantifiable) impacts on the lives of many individuals e.g.improved quality of school meals, access to pensions and other entitlements related todisability and HIV status. At this stage it is not possible to tell whether more strategicchanges will result from the work on CVA e.g. government responses at national level,which would also leverage impact at a wider scale. However, wider uptake of the CVA toolboth across ADPs and between communities is providing clear evidence of how useful andrelevant it is for CBOs with sufficient capacity to use it effectively.

Strategic Objective 2 has also included both local level advocacy and work to influencenational and international spending and policy approaches on HIV/AIDS. Where attributioncan be claimed for changes that sustainably impact on the quality of life for PLWHIV, and

the ability they and their families have to develop their livelihoods, the PPA funding willhave offered good value for money. Some synergy is evident across the PPA objectives asthe use of CVA by DPOs and communities involved in advocacy with OVCs enables WV totrain community advocates who will address a variety of issues prioritised in theircommunity. At the same time the tool also creates a mechanism for linking local levelrealities to regional and national advocacy work, creating the possibility of developing morecoherent bottom-up approaches to accountability and higher level advocacy.

Work on disability mainstreaming has been deliberately exploratory, with the intention ofsupporting learning about different approaches and has not attempted to give good valuein the sense of sustainable impacts. However, the priority given to working in partnershipwith (or supporting the creation of) local DPOs is giving a direction of change predictive of

sustainable increase in capacity and empowerment, which will be an important element inlong term value for money. The training that has taken place has yet to create a critical

7/31/2019 ~9281390

17/24

17

mass in the countries visited during field visits, and to shift focus now would reduce thevalue of the investment made during this PPA. At the same time, the interest that hasalready been shown in the new training manual Travelling Together suggests that thePPA investment will support scaling up of disability mainstreaming at a wider scale acrossWV and indeed, other organisations.

6. Lesson LearningThere has been a huge amount of learning generated by this PPA that will enable WV UKto strengthen its practice across the partnership, particularly in the first three strategicobjectives. Lessons are also being learned by WV partners, civil society and externalpartners. Much of this relates to a shift from direct service delivery towards empowermentand advocacy and given the size and reach of WV programmes and the scale of theissues involved, the learning has potential to reach and influence a very wide audienceand influence both poor people, government actors, service providers and WV staff.

Looked at broadly, programmatic learning has been about the introduction of evidencebased advocacy approaches and developing the ability to reach and work with the most

vulnerable groups of children, while management of the PPA and learning has enabledWV UK to look at strategies for capturing lessons learned, knowledge management andapproaches to capacity building for staff across the partnership.

6.1 Introducing evidence based advocacy approachesThe support that WV gives to communities within ADPs to set up SHGs, CBOs andChildrens Councils/ Youth Groups provides an effective and necessary organisationalframework through which to introduce a range of advocacy tools and approaches. ThePPA has shown that the specific tool or way of organising may vary and community uptakewill also vary in different contexts. However once communities and governmentstakeholders understand their roles and responsibilities within the particular governancesetting, they are generally willing to assume these and often keen to share the knowledge.

Capacity levels of the CBOs involved and the wider political context are very important.Shared learning events have been used among CBOs in India following basic training atcommunity level. In Uganda the need to build organisational and technical capacity of CBOpartners is being addressed through the Civil Society Empowerment project. In morechallenging political contexts alternative activities have taken place. In Cambodia anAdvocacy Capacity Building project has aimed to further understanding of advocacyamongst WV Cambodias staff, who face the fear of being labelled political whenadvocating for a balance of power or addressing rights issues and research findings areused to create an entry point for starting discussions on taboo subjects. Further work willbe needed to pilot the CVA tool and VCA approach in fragile states where governmentsare weak and accountability may be almost non-existent.

One of the main lessons learned in WV Kenya and other countries is that working with thedemand-side alone does not work in 2009 WV Kenya changed its approach and beganworking with the supply-side i.e. Government, engaging with and working on a number ofnational policy and legislative processes including the National Disaster ManagementPolicy, the Social Health Insurance Bill and policy. WV Armenian is developing solidworking relationships with the central and regional government different health andeducation issues being raised through CVA. In other countries, engagement has beenmore local, but the importance of creating a dialogue has emerged at every level.

Learning also indicates that local demand-led governance results in greater citizenawareness and engagement in service quality and policy dialogue at local, provincial andnational levels and is more effective in achieving real change in sustainable practice andpolicy than many direct interventions. As a result, WV has incorporated demand ledgovernance into its new programme model, which is being rolled out across all its partners.

7/31/2019 ~9281390

18/24

18

6.2 Making child protection more effectiveTraining developed for the disability mainstreaming activities has provided a conceptualframework from which to develop programming on vulnerability and child protection in thisPPA. Learning about different conceptual frameworks for disability the medical,charitable and social models in use across different country programmes (and dominant in

different cultural/ political contexts) has helped many staff to develop a greaterunderstanding of vulnerability and inclusion within the context of IPM.

Supporting learning about the importance of changing attitudes among non-disabled inthe community and at the same time supporting disabled people to come together as agroup has been challenging but effective for many staff and community members in ADPs.

Working across the different strategic objectives in the same communities has shown thatvulnerable groups, once organised and able to identify shared needs/ priorities, are alsopowerful advocates for change.

Building on the mainstreaming activities by supporting pilot programmes in specificprogramme areas such as inclusive education or water and sanitation, and backing this up

with action research on the barriers to inclusive service delivery offers WV staff,beneficiaries and partners a practical way of further developing their knowledge, skills andunderstanding. At the same time, this generates evidence for advocacy and developinggood practice case studies.

Commissioning thoughtful, well focused research and choosing researchers whounderstand the particular advocacy context is very effective, as the work associated withEducations Missing Millions Report13 has shown -this provided enough strategic andrelevant, high quality advocacy material to keep WV going for almost 4 years. Similarlyresearch conducted on OVC, based on the UNGASS indicators, provided evidence, in theMore Than Words14 report that was used to advocate with several donors to get them toendorse the internationally agreed OVC Framework.

6.3 Lessons learned about programming and knowledgemanagementMainstreaming an issue across an organisation the size of WV could require a massiveinvestment of resources. The approach taken in this PPA has shown that careful targetingof key staff and choosing country programmes where there is interest already, togetherwith an offer of technical support where necessary can generate real progress on bothmainstreaming and service provision (demand and supply). Disability awareness has beencentral to this approach and was prioritised in the early stages.

Learning processes which enable the beneficiaries to interact directly with the widercommunity, as happens in VCA and CoH and the disability awareness training can

generate change in attitudes more effectively than working in separate groups if sensitivelyfacilitated. Many staff whose attitudes have been changed by these trainings, realise thataddressing the issues will require a personal journey involving further learning andnetworking to specialist groups who can give technical support.

However, training alone is not sufficient, nor is the production of resources it is crucial toget Senior Management buy-in, to have champions who will continue to lead, reinforce anddevelop new learning opportunities and give ongoing support for staff teams/ communitiesto reduce recidivism (a tendency to revert to old practices because they still dominate thewider context) and increase knowledge transfer across programme areas.

13Educations Missing Millions http://www.worldvision.org.uk/server.php?show=nav.174414

More Than Words? Action for Orphans and Vulnerable Children in Africa: Monitorjng progress towards theUN Declaration of Commitment on HIV/AIDS.

7/31/2019 ~9281390

19/24

19

Many respondents during this evaluation pointed to a universal lesson that is clear in thisPPA influencing attitudinal change and embedding organisational change takes time, asdoes policy influencing work and many aspects of the PPA work have only made the firststeps. Further time and investment will be needed to make the changes sustainable.

In terms of policy influencing nationally and internationally, careful selection of research,

researchers and consultants has been key to the successful disability and inclusionadvocacy: there have been positive and negative experiences showing how important it isthat they understand the donor approaches and how policy influencing happens.Experience also suggests that the impact of the Educations Missing Millions work wouldhave been even greater if it had been grounded in programming (other INGOs did this andadded credibility to the research) and if there had been a clear advocacy strategy for usingit from the start. Recent research by ODI on health and aid effectiveness, funded throughthe PPA, has also provided excellent evidence based advocacy material linking local tonational and international policy issues and evidence from PPA funded work on HIV andAIDS has been used to good effect in different international advocacy initiatives.

The development of a database for monitoring CVA outputs has shown that the data has

to be meaningful to governments if it is to be used in advocacy, and that this can be achallenging area for countries where access to ICT is limited. However, the potential forproviding a meaningful link from grass roots issues identified by communities, to nationaland international policy campaigns (as well as data information and management needs ofgovernments) has also been demonstrated.

The PPA has contributed to the decision by WV to adopt IPM and lessons speak to thisalso. WV is rolling this out across different programme modalities but at present, mostprogramme work funded through this PPA is through child sponsorship within an AreaDevelopment Programme. This model was outlined earlier on page 6. Although theevaluation team are not familiar with the detailed criteria set for sponsorship, there areindications from this PPA, particularly strategic objective 2, that a focus on the mostvulnerable children will require greater flexibility/ re-alignment of some criteria. A specificand powerful example was seen in India where longstanding work with communities of sexworkers was continued through sponsorship after the original (DFID) funding finished. WVIsponsorship funds were used to allow somewhat greater flexibility, but even so, when girlsare sent to join their mothers for extended periods (part of grooming for their futureemployment as sex workers) they have been dropped from sponsorship. It is importantthat this lesson is captured and considered more widely in relation to the new IPM if WV isgoing to focus on vulnerable groups which might include travelling/ nomadic communities,displaced and refugee communities and so on.

CVA work which targets specific services such as schools or health facilities can alsosometimes miss the target in some ways: if marginalised people are not able to accessthe service in the first place, improvements will not affect the quality of their lives directly.

WV staff and the community advocates have to remain alert to this when identifying theirpriorities for CVA work and at times have successfully removed barriers preventing somepeople from accessing services.

A final area of learning that has emerged in relation to the PPA work is how best learningcan be captured, contextualised and disseminated. Knowledge management is achallenging area for WV as the issues section highlights, but some important lessons havealso been generated about effective practice.

Although WV has a portal where documents can be placed, and various communities ofpractice are operated through the portal, none of the WV staff interviewed considered thisto be a very effective way of disseminating practice at present. Meetings betweenpractitioners have been a more successful approach: annual learning events to support

development of the VCA model have led to wide recognition across the WV partnership;learning forums (regional) and global learning laboratories (thematic meetings to share

7/31/2019 ~9281390

20/24

20

progress on CVA approaches) share learning, generate materials and develop ownershipof the approaches/ objectives.

At country level, methodologies and results have also been shared across the differentobjectives with external partners and neighbouring APDs at times e.g. results of a study onOVC resource tracking in Zambia were shared and used within the Child Budget Network.

In Mozambique ADP staff from areas sponsored by WV Canada and WV Australia havealso been to share PPA work and then replicated it in the ADPs where they are based.

7. Building Support for DevelopmentStrategic Objective 4: Young people in the UK and the rest of the world are able toparticipate and influence others in global poverty reduction, both with and on behalfof children in the worlds poorest communities.WV UK has addressed building support for development through a fourth Strategic

Objective targeting young people specifically: Young people in the UK and the rest of theworld are able to participate and influence others in global poverty reduction, both with andon behalf of children in the worlds poorest communities. The target audience includesformal and non-formal education sectors in the North and South, with the aim of holdingdecision makers to account on global justice issues.

The mechanisms employed for building support for development include partnerships onspecific initiatives such as Global Schools Forum (GSF) and Giving Nation, deployment ofYouth Ambassadors (YAs) and Visiting Speakers to schools. The core of this awarenessraising has been the design and distribution of educational resources.

WVs Echo supporter database holds all the information concerning subscription to youthresources. As of August 2010 over 13,000 requests (by individuals, families, groups and

organisations) had been received for educational resources, A recent email survey onresources that have been developed shows that most people using them feel that theyraise awareness among young people and that some engage in action as a result.

WVUK has explored a number of innovative approaches to take key developmentmessages to young people and engage them in development awareness and advocacycampaigns. These include a 'Flashmob' performance at Legoland (now on U-tube), toraise awareness of the WVs Child Health Now campaign and promote a prompt response- 120 children were involved in an impromptu song and dance routine from the musicalAnnie, to draw attention to children under five who die of preventable diseases likediarrhoea, pneumonia and malaria. The theatre company Riding Lights toured interactive

drama workshops to schools during the summer term in 2010 to help students understandand explore complex issues around poverty and health. A Petition wall was used at theSoul Survivor Christian Rock Festival, where 4000 messages from young people topolitical decision-makers about their commitments on MDG4 and child health wererecorded.

WV UK partnered with Damaris Education on Global School Forums - engaging nationalconferences on international development for up to 600 sixth formers in the UK. Thestudents learn about a key aspect of international development on the day and take awaya toolkit of multimedia material which they use to teach other students in their own schoolsand in their local primary schools. WV UK offered opportunities for students to givesupport to national conferences and create content for resources.

The GSF programme would certainly seem to have provided the impetus to students toengage with global issues in a real way. The number of new schools attending GSF

7/31/2019 ~9281390

21/24

21

conferences every year is on average around 42% which suggests that the GSF initiativeis being successfully taken to scale. Whilst a good working relationship was maintained,WV UK has now ended involvement in this initiative due to a change in strategic focus.The GSF feel WV UK is being short-sighted to pull out of this work with young people15.

Giving Nation is a project set up by the Citizenship Foundation, which nurtures charitable

philanthropic work in schools. World Vision was the charity partner for 2010 and organiseda trip to India for a group of students as the prize in the Giving Nation AwardsProgramme, an annual celebration event allowing one school to see how charity funds arebeing spent on the ground. Unfortunately refocusing WV UK strategy (away from youngpeople16) is perceived as having affected WVUK commitment and energy leading tomissed opportunities. Indeed there was disappointment with return on financial investment,as this change of strategic direction had a direct effect on the impact of Giving Nationswork.

A localised programme of school visits was held, aiming to deploy voluntary specialistspeakers to effectively represent WV and raise awareness of key global justice issues andfacilitate educational resources in schools. The number of volunteer speakers involved

grew from 9 in year 1 to 17 in year 2 and to 21 in year 3. In 2008 over 90% of the schoolsvisited said that they would like to receive another visit within the academic year. Thenumber of campaign pledges each school achieved was the main impact indicator and in2009 with 9 schools participating, 761 pledges were signed for the Stop the child killerscampaign, increasing to 1048 in 2010 for the Child Health Now campaign. Additionalimpact indicators included number of children sponsored and funds raised for WValongside qualitative feedback from teachers. In some instances the response fromschools was to fundraise rather than the intended primary objective of campaigning.

Since 2009 WV UK has supported a small pool of Youth Ambassadors - 21 in the initialyear, dropping to 14 in the current year to be the voice of WVs quest to send povertypacking WV advocacy expert and spokesperson extraordinaire. Seeking to increasetheir awareness of and build their support for international development, and give them theopportunity to participate and lead in advocacy and awareness raising initiatives, the YouthAmbassadors have undertaken a series of assignments, attended a Survival trainingweekend, recorded an anti-poverty song, engaged in WV campaigns, and are operating aspeer educators. The YAs embarked on activities including running school assemblies,fund-raising for WV UK, petitioning and increasing WV child sponsorship.

In August 2010 Youth Ambassadors were invited to engage in the WV Finish the Jobcampaign, resulting in an extra 1337 signatures to a total of 10,500 signatures from WVUK. Three Youth Ambassadors attended the Bond MDG Event on 15th September, whereWV and other NGOs submitted the signatures to Nick Clegg and Andrew Mitchell.

Beyond campaiging and fund-raising it is hard to measure the real impact of the activitiesunder this strategic objective. However there may be longer-term impact, which is notpossible to determine at this stage.

Transformation Research conducted with staff revealed that on many occasions relativelysimple but engaging experiences in childhood have made such a significant impact thatthey and their message are remembered 20 or 30 years later. In reality, however, anindividuals perspective and understanding is determined by a whole host of experiencesthroughout their life. Specialist Speakers Pilot Evaluation Report 2008

15 In its new strategy for FY 2011-15 WVUK will move its focus away from young people as a stand-alone

audience. The rationale for this is that WVUK believes it cannot make a unique contribution compared to otheragencies who have a strong foothold with this demographic but to target other demographics. WVUK alsoclaim not to have found any evidence of strong brand loyalty from the youth demographic.

7/31/2019 ~9281390

22/24

22

8. Issues to be addressedThere are a range of issues to be addressed that are quite specific to the strategicobjectives as well as some more generic, overarching issues.

Issues specific to Strategic Objective 1 include how to cope with the volume of activity,

meet demand for training and offer effective support whilst ensuring that quality ismaintained. There is already a bottleneck in training for two countries, and WV plans tomainstream use of the CVA as an advocacy tool across the partnership. The trainingneeds are also organically developing as communities engage more and engage at widerlevels of governance. Staff capacity and skills (both managers and field based) need tokeep one step ahead if they are to provide meaningful support, and this is alreadyleading into a range of new challenges, including use of the media and legal literacy.

This PPA has also highlighted the challenges of vertical linking. The advocacy capacity ofdifferent WV partners varies tremendously and some offices have devolved responsibilityfor advocacy across their teams rather than employing specialist staff. The CVA has begunto generate considerable amounts of data and has the potential to support a truly bottom-up approach to WV advocacy, but more work is needed to achieve this. A good start has

been made developing a management information system, but the data base is still quitedifficult to use. More work on the indicators, building on initial experiences, should ensurethat carefully focusing them will reduce the number needed.

If CVA is to be used by WV UK more consistently as a research tool to feed national andglobal campaigns such as Child Health Now, then there should be a recognition that CVAmonitoring centres on service delivery providers and facilities rather than on those morevulnerable sections of the communities who do not access the facilities. There would needto be some complementary tools to fill the gap and reach out, for example to the 51% ofthe Ugandan population who do not access health clinics.

Field visits highlighted the challenge that staff face moving from a service delivery focus tosupporting advocacy and empowerment. By definition, vulnerable people have limitedlivelihoods and coping strategies, so it is tempting to focus on immediate practical issuesrather than the strategic policy changes that would create sustainable change. Servicedelivery fits with a charitable approach to Christian Ministry and both beneficiaries and staffhave been comfortable with it for many years; moving the focus shifts responsibilities fromstaff towards governments/ service providers and to the beneficiaries to hold them toaccount. This can be an uncomfortable transition for all stakeholders!

In Strategic Objective 3, there are areas of disability which remain below the radar fewcultures address issues of mental impairment in any way, let alone inclusively and thework to date has reinforced how far attitudes and understanding still need to move to makeinclusive practice a reality at community level.

There appears to be more recognition of how important the disability issues are across WVthan within WV UK where the consensus is that the disability box has been ticked. Somestaff feel that they are now mainstreaming disability, but feedback from others raisesconcern that this is not universal and more importantly, that there is still a need across WVfor a disability champion among the partners. This evaluation has found that disabilitymainstreaming is by no means embedded across the partners who have been directlyinvolved in the PPA yet and there is considerable concern that moving towards aninclusive/ integrated approach will result in disability issues being lost. Although WVI has adisability specialist, there does not appear to be a clear hand-over strategy in the sameway that there is for work on accountability and advocacy.

Although less PPA funding has been directed to work in this area than to the work on CVAand HIV/AIDS, solid groundwork has been undertaken, which should be continued in orderto realise the intended impacts and achieve good value for the money invested. The workhas not yet reached take off if support is not continued, the achievements within WV as

7/31/2019 ~9281390

23/24

23

a whole are likely to remain limited and disability may return to the periphery ofprogramming. We feel that this work should continue to be prioritised by WV, albeit withinthe overarching focus on vulnerability and child well being.

At a more general programming level, this PPA has shown clearly how communities willbreak down barriers created when training initiatives are not coordinated, if the tool or

approach offered by WV meets their needs: in many communities the nuances of CVA,VCA and disability mainstreaming have merged into rights based advocacy aroundinclusion and entitlements. However, on occasions there appear to be a plethora ofinitiatives and care is needed to ensure that existing structures and processes areacknowledged and strengthened e.g. there is concern that CCCs in Zambia areundermining the role of the Area Development Committees.

The PPA funding is a drop in the ocean of global funding managed by WV partners andhas to be used very strategically to maximise its impact. Selecting a few countries from theportfolio funded by WV UK will not necessarily represent the best mix. This PPA hasshown that where countries are working on all three programme objectives cross learningand a degree of synergy has developed. Additionally, work on Strategic Objective 2 has

shown how a Regional approach can add value. Country selection for this PPA tookcapacity and interest into account but other considerations are also important. Forexample, although the PPA work in Armenia has developed very effectively in each of theobjectives, should WV UK seek experience in more fragile contexts, or does the pace ofchange and effectiveness of uptake in Armenia enable more lessons to be learned in ashort time? The income gaps and vulnerability of marginalised people can be moreprofound in some middle income countries than in the poorest ones, so judging on thesecriteria is not straightforward either. In a future PPA, careful thought would again beneeded about how the selection of countries can maximise the learning potential andimpacts.

The structure and modus operandi of WV brings several issues that WV UK cannotnecessarily address, but which need to be acknowledged.

Coping with the effects of staff turnover was referred to in almost every interviewconducted. Whatever the underlying reasons reorganisation, relocation, moving toanother job being just some of them planning for future work has to take account of thefact that this is a feature of WV, so for example training is likely to have an impact for alimited time before refresher/ repeat training is needed.

The building blocks of WV programmes are the ADPs. They (or something similar) areessential, so the challenges they present for dissemination and knowledge managementwill always exist: different WV partners may sponsor neighbouring ADPs and reporting isnot always shared across ADPs. Recent analysis by the India programme design teamidentified how important it is to understand the approach that key senior managers havebecause they influence how programmes develop across each region. These are factorsthat are beyond the control of WV UK and can have more or less impact on learning, theflow of information for advocacy and so on, but they are features of the way that WV worksand need to be considered when developing PPA work. One useful strategy to addresssome of these issues has been to use external consultants (in this case for the disabilitymainstreaming work) who not only bring fresh technical expertise, but because they areexternal can bring fresh insights on organisational dynamics and have usefully drawnlearning together at country level to increase the sustainability of work being developed.They are also able to help national staff to identify good practice and innovation, whichmay not be apparent when it has developed organically in response to particular contexts.

7/31/2019 ~9281390

24/24