Embed Size (px)
Citation preview
June 2017
navigating ChangeAn action learning project about funding outcomes
navigating Change. June 2017.
This work is licensed under the Creative Commons Attribution-Non-commercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.
Please reference as follows:
Inspiring Impact NI. (2017). Navigating Change. Belfast. Building Change Trust
Contents
Background to the project 4
executive Summary 8
1 Funding Outcomes Action Learning Project 11
2 Case studies 16
3 Lessons for good impact practice and outcomes focused funding 22
4 next steps 28
Appendices 30
Appendix One The Code of Good Impact Practice:
Key principles
Appendix Two Funders’ Principles and Drivers of
Good Impact Practice: Summary version
Appendix Three Participants and strands in the Funding Outcomes
Action Learning Project
List of figures
Figure 1 Inspiring Impact’s Impact Practice Cycle 4
Figure 2 The path to this project 6
Figure 3 Benefits of the approach 9
Figure 4 Outcomes table for the action learning project 12
Figure 5 Menu of action learning project activities 13
Figure 6 An example ripple-map 23
3
Background to the project Inspiring Impact northern Ireland
Inspiring Impact Northern Ireland (IINI) is an initiative of the Building Change Trust (BCT) that aims to transform how the voluntary community and social enterprise (VCSE) sector and its funders think and go about demonstrating the difference they make. It is funded by the Department for Communities (DfC) and the Big Lottery Fund.
IINI is delivered by Community Evaluation Northern Ireland (CENI) and is part of Inspiring Impact UK - a UK wide collaboration that is supporting VCSE organisations and their funders to put impact at the heart of the VCSE sector.
Inspiring Impact encourages VCSE organisations and funders to use Good Impact Practice and has developed a range of freely accessible resources to help them do this.
4
• Communicate information about your impact
• Learn from your findings and use them to improve your work
• Make sense of the information you’ve collected about impact
• Draw conclusions about the impact you’ve had and compare findings
• Identify your desired impact and how to deliver it
• Understand what to measure about your impact, identify the information you need and how to collect it
• Deliver the work and impact
• Collect the information you need about impact
1Plan
4Review
2Do
3Assess
There are four main areas of activity that make up impact practice:
Figure 1 Inspiring Impact’s Good Impact Practice cycle
Good Impact Practice
Good impact practice means ensuring that all elements of an organisation’s work have an impact focus. It covers everything from planning and delivery to collecting and analysing data, through to how information is used to share learning and create improvements.
Inspiring Impact UK developed the Code of Good Impact Practice (Appendix One) and a logical cycle to guide organisations in implementing good impact practice, which uses learning from each cycle to inform the next (Figure 1).
Inspiring Impact also created Measuring Up! - a diagnostic tool, or health-check, to help organisations take stock and improve their approach to demonstrating impact. You can find this at www.inspiringimpact.org.
The route to this report
Putting outcomes at the heart of government, public service and funding of the VCSE sector is an important step. And it is not a small one. It involves a seismic shift in accountability, adding evidencing effect to the need to ensure compliance and probity.
So in 2015, when the Executive announced Outcomes Based Accountability (OBA™) as its chosen methodology for the draft Programme for
Government, (PfG) there was a clear challenge to funders and the sector around readiness.
CENI responded by commissioning research to assess how well prepared funders, and particularly public funders, were to understand and implement outcomes focused approaches to funding of the community and voluntary sector.
The report, Embracing Changei, was published in June 2016 and set out three recommendations:
5
Help people embrace the concepts, develop a shared understanding and language about outcomes, and gain skills to ready themselves, their department, unit or section.
Examine the systems used to allocate and account for public expenditure and give guidance to ensure they fit to outcomes based approaches.
Test new approaches and systems through demonstration projects and share learning.
1Prepare people
2Adjust
Systems
3Test And
Learn
The report was well received and IINI agreed with the Voluntary and Community Division (VCD) at the Department for Communities that next steps should focus on supporting a sample of funding programmes to address the first and third of these recommendations.
IINI and CENI created the Funding Outcomes Action Learning Project; a fixed-term initiative to increase capacity for and test approaches to outcomes based funding design set in the real world.
i CENI. (2016). Embracing Change. Belfast: IINI. https://inspiringimpactni.files.wordpress.com/2016/04/embracing-change-IINI-scoping-study-june-2016.pdf
Figure 2 The path to this project
Community Evaluation Northern Ireland reviews
outcomes practice and policies relating to the voluntary, community
social enterprise sector. The finding is that
outcomes approaches are being constrained by
compliance and probity processes and a radical overhaul of culture and
practice is recommended ii.
Inspiring Impact UK established to support charities and their funders to improve how they think and go about demonstrating impact.
In the VCSE, IINI selects 13 organisations to become impact
champions. The champions help their member organisations to use the
Code of Good Impact Practice and the Measuring Up! self-health check.
In the public sector, IINI supports a number of demonstration projects to apply impact practice to public sector
funding programmes.
Funders’ Insightsiii and First Steps to Impactiv learning papers produced
and disseminated.
2014-2015
2010
2012
The path to this project
6
ii CENI (2010) Measuring Up. Belfast. CENI. http://www.ceni.org/sites/default/files/Measuringup%5B1%5D.pdf
iii IINI (June 2015) Insights and lessons from five statutory sector demonstration projects. Belfast. IINI. https://inspiringimpactni.files.wordpress.com/2014/02/inspiring-impact-ni-funder-shared-learning-paper2.pdf
iv IINI (December 2015) First Steps to Impact. Belfast. IINI. https://inspiringimpactni.files.wordpress.com/2014/02/first-steps-to-impact-cover.jpg
The Building Change Trust appoints CENI as their strategic partner to deliver the Inspiring Impact Northern Ireland programme. The Department for Social Development contributes significant support for a programme designed to promote and support good impact practice across the VCSE sector and its funders.
2016-2021 draft PfG is published showing 14
outcomes and 48 indicators.
IINI researches readiness for outcomes and publishes
Embracing Change.
IINI rolls out the Funding outcomes action learning
project in response to the Embracing Change
recommendations.NI Executive announces that the new Programme for Government will embrace the Outcomes Based Accountability (OBA™) methodology.
2013
2015
2016
7
executive Summary
The project
This action learning project was a response to earlier research indicating a need to prepare for outcome focused funding and share learning across funders and sectors.
Through the project we sought to:
help public and independent funders to strengthen their capacity to fund for outcomes;
test and adapt existing outcomes support tools in the new public funding context; and
identify and respond to gaps in the outcomes and impact support available to funders.
As an action learning project, we applied impact tools and methodologies in response to participants’ particular needs and reflected on the results to hone and adapt our resources to funders’ needs and context.
In the six months to March 2017, five funders engaged with the project and applied various impact tools to their own programmes.
8
Key Findings
Benefits of this approach:
Lessons for good impact practice and outcomes focused funding
People need prepared
Good impact planning and co-design processes work best if everyone starts with a basic understanding of three things:
what outcomes focused funding means;
what outcomes mean in the context of the Programme for Government and OBA™ methodology; and
the design process and their role in it.
Co-design is a valuable element of an outcomes focused approach
Co-design helps focus on outcomes because it closes the gap between the many actors involved in creating effective funding programmes. This means planning, action and monitoring are all aligned and centred on creating the desired outcomes. Co-design works well when it encourages knowledge sharing and respect between all co-designers.
There are a range of practical ways to support good design for outcomes
Have written remits and roles Use sub-groups Prepare draftsShow leadership Create collaborative buy-inUse group workBe flexibleDo not oversimplify
9
CongruencyPutting outcomes at the centre of the design process creates a clear line from the funding stream to wider organisational, policy or Programme for Government outcomes.
ConsistencyUsing a well-tested and consistent approach to explaining how change is caused and captured provides a bed rock for understanding outcomes across departments, sectors and methodologies.
CompetencyBringing together delivery agents, policy makers, programme designers, statisticians and outcomes practitioners during planning creates a continuum of expertise that ensures grass roots activities contribute to population outcomes.
CreativityWorking backwards from the changes you want, and involving new people in the discussion about how to create them, stimulates thought and creativity.
1
2
3
4
Figure 3 Benefits of the approach
next steps
To support effective outcomes focused funding we need to work with funders to:
1 enable more funders to explore the full range of the effects of their funding and funding processes across direct and indirect beneficiaries;
2 grow understanding of the general principles of outcomes focused funding;
3 foster skills in using co-design to develop outcomes focused funding;
4 enhance experience of using co-design to shape service delivery;
5 deepen the skills and experience needed to lead outcomes planning and co-design; and
6 more widely share learning about outcomes focused funding across independent funders, NICS and government.
To support the VCSE sector in the move to outcomes focused funding we need to:
7 increase the capacity of the VCSE sector to take part in co-design by increasing awareness, understanding and skills; and
8 deepen the sector’s understanding of how best to respond to the requirements of outcomes focused funding.
To enhance capacity for co-design and learn from the application of outcomes focused design we need to work across sectors to:
9 gather and share learning about the use of co-design across sectors; and
10 monitor the wider impact of outcomes focused funding on the VCSE sector, public and independent funders.
Publishing this action learning paper is, in itself, part of good impact practice. It reflects the imperative to move out of the silos that artificially divide and corral clients, social issues, solutions and, of course, budgets, towards a place where planning better reflects the external, interconnected world. Sharing and co-design help recognise synergies between funders and service deliverers, departments and divisions, policy and programming, assumptions and actuality; and in so doing creates conduits through which further learning can flow.
Action
In taking this work forward we need and welcome your help so do get in touch if you can:
share ideas, information and insights to help us shape responses;
run your own demonstration project to test new approaches;
link us into wider networks; or
resource the development of the tools or events necessary to success.
10
Project goals and outcomes
Action learning is an approach to solving real problems. It involves taking action and reflecting upon the results so as to improve both the problem-solving process and the solutions developed.
Through the Funding Outcomes Action Learning Project (FOAL) we sought to:
help public and independent funders to strengthen their capacity to fund for outcomes;
test and adapt existing outcomes support tools in the new public funding context; and
identify and respond to gaps in the outcomes and impact support available to funders.
Funding Outcomes
Action Learning
Project
11
Real world relevance
While we were clear about the purpose of the project, we did not set a specified curriculum, feeling it better to start from the needs of participating funders, be they to review impact practice, plan a specific project or gain particular skills. The intent was for us and the funders to learn together as we addressed each new task. And we planned to share learning internally and across other funders once the lessons had crystallised. This report is part of that process.
Outcomes: the difference we want to make Impact: long term effect GoalSystems and relationships
1. Shared experience co-design for outcomes, both inter-department and inter-sector
2. Enhance tools to facilitate outcomes based funding design and co-design
Enhanced interaction between funders and the VCSE sector
Funding is effective in creating positive social outcomes
Funders
3. Enhanced awareness of impact practice and better understanding of impact planning
4. Greater motivation to apply impact practice within specific areas of operation
5. Greater capacity to apply outcomes focused approaches to planning and design of selected programmes or areas of work
6. Better understanding of how to invest in and support public service delivery in the VCSE sector in the context an OBA™ based public funding system
7. Good practice models appropriately shared with key government stakeholders and the independent funding sector
8. Wider interest in applying impact practice across other funders and departments
Greater funder capacity to fund outcomes
The VCSe sector
9. Better knowledge of what the VCSE sector will need to engage in outcomes focused co-design with funders
Improved VCSE sector capacity to contribute to outcome focused funding
Figure 4 Outcomes table for the action learning project
12
Figure 5 Menu of action learning project activities
Project activities
Our planned activities centred on applying impact tools and methodologies in response to participants’ particular needs and reflecting on the results so as to hone and adapt our resources to funders’ needs and context.
We arranged our supports into a menu of activities so participants could select the elements that had most relevance to them (Figure 5).
Because the project involved piloting co-design for funding programmes, we expanded our tool kit (Figure 6) by drafting a process chart for good practice in co-design for funding programmes.
We refined this Co:De chart by trialling it across the strands of the project to check that it:
a maximised the input of all parties without compromising their independence or public accountability; and
b created robust performance measures within the context of OBA™.
Project delivery team
The work was funded through and managed by Inspiring Impact NI at CENI. The delivery team was made up of IINI and CENI managers, trainers and consultants.
Project participants
In the six months to March 2017, five funders applied various project activities to a range of projects and programmes as best suited their needs. Appendix Three gives details about which elements were used where.
• Department for Communities VCD:
- Women’s Early Intervention Programme Team
- Programme for Women Involved in Community Transformation Team
- Policy and Innovation Team
- Programme Team
- Volunteer Infrastructure Support Programme
- Social Innovation Working Group
- Fermanagh Trust
- Resurgam Trust
• Arts Council of Northern Ireland
• Causeway Coast and Glens Borough Council
• Derry City and Strabane District Council
• Big Lottery Fund Northern Ireland
13
Awareness and understanding Preparing for outcomes focused funding
Presentation to familiarise funders with the ideas and principles of Good Impact Practice when funding outcomes
Presentation explaining how traditional outcomes approaches and terminology sit within the method and language of OBA™
Presentation explaining how OBA™ alters the way in which funders and those funded will be held accountable for work relating to the Programme for Government
Assessment and action planningTaking stock
Facilitation in the use of Measuring Up! for Funders, a tool which assesses current good impact practice and areas for improvement
Designing outcomes focused funding programmesAligning funding with organisational or PfG outcomes
Facilitation in developing a theory of change to identify the link between funding actions, results, impacts and goals, including PfG outcomes
Facilitation in using the theory of change to identify indicators, measures, data collection methods and report cards as relevant
Facilitation in using the Co:De chart to support good impact practice by enabling funders to use co-design between departments or sectors or both
Sharing Disseminating learning and models
Sharing knowledge and learning within and across funders during the project through group training, meetings and networking
14
Co:De chart We put together an outline process chart for outcomes co-design, indicating the stages and tasks involved in facilitating the co-design of an outcomes focused funding programme. To-date the first two stages have been developed and tested via the supported projects with initial work started on the third stage for one project.
Funders’ principles and drivers of good impact practice The Inspiring Impact UK Code of Good Impact Practice adapted for funders. This gives a set of four principles, and a key driver for each of the phases of the impact cycle (See Appendix Two).
Measuring up! for funders The Inspiring Impact health check tool adapted for use by funders. Self-assessment creates an action plan for building good impact practice into funders’ DNA.
Revie
w Plan
Assess
Do
PurposeTo understand what difference
we make, directly and indirectly.
To learn from what we, and the organisations we
support, do.
To ensure best use and value from our assets.
Driver 1 To be clear about
the difference you and the people
and organisations you support want
to make
Driver 4 Share and act on learning and seek to improve our impact practice
Driver 2 Support grantees
in their impact practice and
resource your own imact practice
Driver 3 Identify difference made and assess how and why it was made
PRINCIPLESA. Apply proportional and approriate rigour and resourcesB. Be flexible, open and transparentC. Acknowledge our respective independent valuesD. Recognise that everyone can contribute to impact practice
• Facilitators - programme familiarisation and documentary review• Develop work plan & schedule with senior programme staff• Identify key stakeholders to involve in outcomes co-design group• Establish outcomes co-design group with agreed term of reference
• Facilitated sessions with outcomes co-design group to:- draft theory of change for programme aligned with PfG outcomes- develop indicators for identified programme outcomes- identify appropriate measures and develop report cards
• Public consultation with potential suppliers to promote/explain outcomes approach• Incorporate outcomes into commissioning procedures, protocols• Inform tender briefing• Inform design of application & assessment processes
• Agree evidence required for monitoring & performance measures• Review existing monitoring, evaluation mechanism to reflet above• Inform ToR for programme evaluation to capture outcomes & learning
ProgrammeInitiation
Co-Design Programme Outcomes
Programme Monitoring &
evaluation
CommissioningOutcomes
Project Toolkit
15
Theory of Change A way to organise thoughts about outcomes to show why what you do will create changes that contribute to your goal. It helps you examine the evidence and assumptions behind your reasoning and consider what conditions and resources are needed to enable effects to occur. By helping you select activities that align with the difference you want to make, it also points to measures that will tell you when outcomes are starting to happen.
Ripple-mapA deceptively simple power-tool for helping to decide what you are and are not accountable for and what sort of indicator you may therefore need to look out for.
What you do
Inputs Activities Outputs
Resources Courses (eg) numbers
Monitoring
Effort
The difference you make
Outcomes Impacts Pre-conds Goal
Achievement Difference Contribution
evaluation
Effect
Reasoning
enablers
Funder
Fund recipients
Individuals
Community
Practice
PfG
16
Case studies
Context
In November 2015, the NI Executive and the UK and Irish Governments agreed a plan to build peace, stability, progress and hope by tackling some of the most difficult issues in Northern Ireland. The Fresh Start Agreement included a commitment to ‘a programme to increase the participation and influence of women in community development’1.
Public sector funder: Our work with the Department for Communities
Programme for Women Involved in Community Transformation and Women’s early Intervention Programme: The Department for Communities needed to co-design a programme to increase women’s engagement in community development in the context of the Fresh Start Agreement. Our delivery team devised and tested a chart for the co-design process and facilitated the creation of a PfG aligned outcomes framework and data collection tools.
1Fresh Start Agreement Section ‘A’, Paragraph 3.9
17
Co-design
The Department for Communities has been working with other departments, statutory agencies and community and women’s organisations to co-design the Programme for Women Involved in Community Transformation. As part of this, it established a co-design group of these key stakeholders.
CENI’s delivery team supported this co-design group in developing an outcomes focused approach to the programme. We created a chart to guide people through each step (Co:De) and set up a sub-group to work through drafts of the outcomes framework before presenting it to the full group.
Our role was to guide, prompt and challenge the outcomes sub-group to focus on the changes they wanted to make rather than the activities they normally delivered. In this way, co-design increased the outcomes focused nature of the programme because it meant delivery agents could shape outcomes and delivery mechanisms around what they knew might work.
Creating the outcomes framework was an iterative process of working through different structures and phraseology and exploring the purpose of the programme in more depth.
Outcomes framework aligned to PfG
Theory of changeWe worked with the sub-group to develop a theory of change for Women Involved in Community Transformation. This set out:
the effects the programme was expected to achieve at individual, organisational and strategic levels;
the subsequent impacts and relevance to the Tackling Paramilitarism and Organised Crime Programme; and
the potential contribution to PfG outcomes (Outcome 10: indicator 28).
Indicators of changeThe outline theory of change was extended to a more detailed framework through the addition of indicators of change for each of the proposed programme outcomes.
Measures and report cards
We subsequently worked with NISRA statisticians to incorporate relevant measures into an OBA™ style report card. Not all of the change indicators in the outcomes framework were reducible to quantitative performance measures and those omitted will be captured and assessed using mostly qualitative methods.
Data collection toolsThe co-design process indicated the need for a Women’s Early Intervention Programme (WEIP) as a precursor that would inform the main Programme for Women Involved in Community Transformation. This WEIP this would run from January to March 2017. Working with the sub-group, CENI used the original theory of change to develop an outcomes framework for WEIP to capture change over the shorter timeframe.
CENI designed and developed monitoring tools based on the WEIP outcomes framework and refined them with the co-design group. The data collected has been analysed and written up into a review of the early intervention programme.
This was a valuable exercise with stakeholders
from various sectors and different perspectives
coming together to agree an outcomes framework
for a complex and politically sensitive programme.
Context
Resurgam Community Development Trust Ltd was established in 2011 by community groups and social enterprise organisations following fifteen years of experience of working in communities. Based in the Old Warren Estate, the Trust works with 26 community-based projects across Lisburn.
The Department for Communities has had an ongoing relationship with the Trust and was keen to support the Trust to adopt a more outcomes focused approach to their organisational planning and management. The Trust had applied various outcome methodologies and tools to individual projects within the organisation, but the structure (including a number of social enterprises) and sheer diversity of projects it managed (from early years interventions to support to community network members) had militated against the use of any universal measurement tool to capture and evidence the overall outcomes of its work.
Co-design
Our delivery team supported the Trust to develop an outcome based approach for the organisation. To do this we set up an outcomes group consisting of senior management and project leaders within the Trust. This group shaped up drafts and used these to report to and consult with the wider staff team at each stage of the process.
Constructing the final indicator framework was an iterative process, with the structure and wording going through numerous amendments before arriving at an agreed final document. This was greatly eased by having a sub-group with both delegated authority to develop the work and the commitment to update and consult with the wider Trust membership. As a result, there is now a broad understanding of, and buy-in to, the exercise.
VCSe sector fundee: Our work with Resurgam Community Development Trust
Capturing evidence across many funded strands: Resurgam Community Development Trust Ltd had used a variety of methods to assess the effect of many of its strands of work and was now keen to look at impact on an organisation-wide basis. We worked with a sub-group of Trust personnel to create a theory of change and indicators of change.
18
Indicators of changeThe outline theory of change informed the creation of a detailed framework which included indicators and associated measures or evidence for each outcome across the proposed categories.
Measures and data collection
The Trust is currently finalising the indicator framework before it moves on to firming up measures and data collection tools and plans. This will include identifying performance measures through which to demonstrate the contribution it makes to PfG outcomes.
19
Outcomes framework aligned to PfG
Theory of changeWe worked with the sub-group to develop a theory of change for the Trust. This drew on a review of existing strategic and operational plans, and involved:
Clarifying the different effects of the organisation’s work.
- On the community: individuals; groups; environment; and relationships.
- On its operations: representative; quality; and innovative.
- On its strategic position: partnerships; policy and practice influence.
Identifying and locating specific outcomes within each category.
Anticipating the impacts that achieving these outcomes would have on the community and on the Trust itself.
Linking all elements to the overall organisational goals and (potential) contribution to PfG outcomes.
This was a valuable exercise in organisation-wide
co-design. It saw stakeholders (including funders) from
various projects within the Trust coming together to
deliberate and negotiate an agreed theory of change for
what is a complex and diverse organisation.
Context
The Big Lottery Fund is best known for distributing National Lottery players’ money to support projects in health, education, environment and charitable purposes. Our work is about creating opportunities for people and communities to do great things – whether that’s through an award of £500 up to our larger grants of £500,000. People in the Lead, our strategic framework 2015 to 2021, sets out a vision for our funding and is driven by a belief that people should be at the forefront of improving their lives and communities.
As we funded Inspiring Impact we wanted to use the approach ourselves, to help us consider our impact and develop messages about our work beyond the National Lottery funding that we award and the number of grants made.
Measuring up! including theory of change
We used the Measuring Up! funders’ version and tested its applicability and usefulness for us as a funder. This involved two facilitated workshops with the Inspiring Impact NI team at CENI, as well as a number of internal meetings to take things forward.
Co-design and buy-in
Staff from all our Northern Ireland teams are involved - this ensured good, honest discussion from the beginning. There were challenges early on but as we’ve continued, more colleagues have begun to see the value in the approach. It’s important that our work on impact belongs to everyone within our team, whether they assess or manage grants or highlight the difference our funding is making to people and communities.
An independent funder: A review of Measuring Up! by Big Lottery Fund NI
Looking at impact beyond the funding awarded: The Big Lottery Fund NI wanted to take an overarching look at its impact practice across all strands of its work. Personnel from across the NI portfolio came together and, with our support, used Measuring Up! for funders and explored a funder’s theory of change. Here they reflect on the experience.
20
Benefits of the process
This approach helped us collate key facts and information for our NI programmes; reflect on progress over the last year; and consider where we want to develop how we work into 2018.
Defining and explaining our work The process provided an opportunity to (re)start a conversation on what we want to say about the National Lottery funding that we deliver, our role as a funder and specific funding programmes. This conversation is open, honest and ongoing, and has helped us to explain other changes in the way we work, for example our data coding and analysis, and our grant management and reporting.
Measures and data collectionThis approach enabled us to consider what information we want to collect for our People and Communities programme which has been open for just over a year. We are also in the early stages of using the approach to consider how we can evaluate our Empowering Young People programme.
Planning and reviewingWe have moved on from static reporting and, as we progress, we will integrate actions informed by Measuring Up! into how we work. We have created a living document that will, and has been, updated.
Communicating and sharingWe developed one-pagers with a mix of data and text to tell the story about our funding programmes in NI. These helped anchor our discussions, take stock and agree how we progress. We used them to update our UK Board on the impact of our NI programmes and feedback has been very positive. We are sharing our learning on the use of Measuring Up! including how it is enabling us to support and influence change in our ways of working.
Challenges of the process Changing how we think about reflecting our
impact as a funder more broadly, rather than just the impact of our funding.
Moving from perceiving Inspiring Impact and Measuring Up! as named and set processes, to understanding them as more fluid supports in reality.
Working through any confusion around differing interpretations of Measuring Up! questions and using it to facilitate discussion.
Not appreciating the amount of time it would take at the start.
21
“Having applied this process we can recommend it to other funders as being beneficial.
We found that more people are involved in discussion around impact than had
been the case with previous approaches to measures of success and programme effectiveness. Using Measuring Up! has
started a useful conversation on what we want to say about our impact as a
funder. I am confident that this is only the start.”
Norrie Breslin, Big Lottery Fund NI
22
We used structured interviews, informal feedback conversations and surveys to collect reflections from participating funders, fund recipients and our delivery team.
Having identified common comments across the different strands we grouped them into themes and distilled the following insights on outcomes focused funding:
People need to be prepared with basic competences and understanding
Co-design is a powerful aid to outcome delivery; and
There are a number of practical things you can do to support the process
Lessons for good impact practice and outcomes focused funding
23
People need preparation
Participants told us that the good impact planning and co-design processes would work best if everyone began with a basic understanding of three things:
what outcomes focused funding means;
what outcomes mean in the context of the Programme for Government and OBA™ methodology; and
the design process and their role in it.
The extent to which these basics were clearly explained and understood from the start had a significant impact on managing the expectations, roles, time commitments and overall progress of the projects in this study.
understanding outcomes focused processOutcomes focused means building everything around the beneficial changes to be created across individuals, communities or systems.
This project showed the value of using a theory of change to pinpoint the desired changes and define the exact nature of the beneficiaries.
Completing their own ripple-map helped programme designers to clarify the different types of beneficiaries (or targets) of the changes to be made, and the varying degrees to which they are accountable for the benefits accruing to them.
The ripples of influence exercise also revealed that funders tend to look more at the effects that funded activities have on individuals, and often overlook the impact they have on organisations, communities, practices, sectors and policies.
understanding the context of public funding
The Programme for Government places outcomes at the centre of all public investment. We found confusion around the language of the OBA™ methodology to be one of the key design stumbling blocks.
OBA™ is based on a distinction between two types of accountability for delivering wellbeing: population accountability and performance accountability. Population accountability refers to the changes made to an entire population, for instance, ‘all children’ or ‘all adults’ in Northern Ireland; whereas performance accountability relates to the specific programmes or services delivered to these populations (including those delivered by third sector services in receipt of public funding).
The PfG presents outcomes and indicators for the entire population of Northern Ireland.
These outcomes are effectively policy goals, the achievement of which will depend on the contributions made by a plethora of interventions and programmes. Therefore, no single programme or intervention can be held solely accountable for the achievement of any PfG outcome. Rather, it is the sum of the contributions of agencies, programmes and services that move us towards the realisation of outcomes for the population. And so those who plan or provide interventions are answerable for the extent to which their activities deliver the contributions promised (performance accountability) but not for the delivery of PfG outcomes (population accountability).
Funder
Fund recipients
Individuals
Community
Practice
PfG
Figure 6 An example ripple-map
24
We found programme planners and service managers wrestling to separate out their responsibility for managing performance accountability for a programme or service from population accountability for policy level outcomes.
We reduced much of this confusion by reinforcing two messages.
Programme designers and service delivery agents can be responsible only for defining and reporting on their own contributions to PfG outcomes and not for PfG outcomes in themselves.
Contributions must be linked to PfG outcomes but this link need not be linear or directly measurable.
We supported these assertions by facilitating public funders to create a theory of change and ripple-map for their funding programme. These made it easier to distinguish between those effects that can clearly be attributed to the programme, the funder, or the delivery agents and those to which the programme can claim only a more distant contribution.
Although there is still some way to go in exploring fully the division of responsibility between funder and fund holder for effecting different classes of change, this project has seen a small shift, from collecting data about changes to individuals alone, to asking what impact funders and their funding have on organisations, sectors and systems.
understanding the design processes and rolesBeing clear about the process from the start (be it co-design or otherwise) allows organisations to make decisions about participation based on competence, time and resource commitments. It also creates shared expectations for all participants around roles, milestones and end points.
This means funders need to make early decisions about the sort of design process to follow, how flexible this will be, and what powers any consultees or co-designers will and will not have in shaping activities.
Participants in this project said programme design would run best if everyone had a basic level of familiarity with the process so they knew what the key steps were and which issues would be decided when. We created Co:De to help with this and will develop and refine it over time.
We have also identified the need for a template explaining co-design and what a co-design group is for, what it will do, what its powers are, who does what and what input and behaviour is expected of all members. This role and remit template can then be adapted to particular programmes and to any sub-groups.
At the Arts Council of Northern Ireland, our
presentation helped Arts officers get to grips with the concept of
their being responsible for making grants to create contributions to moving PfG indicator 27, rather
than being held solely accountable for delivering increased engagement in arts and
cultural activities.
25
Co-design supports outcomes
This project showed that co-design is valuable in creating outcomes focused funding because it assembles the knowledge needed to understand how apparently simple outcomes are most likely to be created in a complex real world.
Using co-design requires the funder to think about what skills, knowledge and networks might be needed and thus which organisations and people to engage.
We found that co-design stimulated funders to think beyond consulter and consultee to seeing ‘co-designers’. As a result, they invited to the table a wider range of people whom they felt had a contribution to make and those invited felt that the funder wanted to hear them. This is a good foundation for creating mutual respect and trust. These are vital because the knowledge pool of co-design becomes powerful only when everyone is prepared to listen to one another and accept all perspectives as valid.
This respect for people’s experience needs to be built on respect for their motivation. Having occupied different silos for so long departments, organisations, and sectors all develop their own interpretation of each other’s raison d’être. Recognising that these may be inaccurate is the basis for forming a working relationship based on mutual trust.
The earlier co-design is adopted, the better it works. We observed that inviting people to join as co-designees from the start, rather than introducing the process part-way through, seemed to contribute to a stronger design team and more coherent process.
Addressing potential conflicts of interest early on also aids collaboration and avoids confusion. This means being clear about decision-making powers, commissioning processes and accountability structures from the start. This includes establishing clarity about access to information that may be relevant to the tendering process.
Whether bringing people together within a funder (eg Big Lottery Fund), or a VCSE organisation (eg Resurgam Trust) or across Departments and sectors (the two women’s programmes) this project demonstrated the power of combining knowledge and perspectives to mitigate fixed assumptions about direction or approaches.
Given the importance of co-design to outcomes focused funding, we need to increase understanding of the process across all sectors engaged in the creation of funding programmes.
Civil servant: “Having members from that
thematic area work with us was an absolute must … they were able to provide a level of information which government and statutory
bodies do not hold”
Practical tips from across the project
Have written remits and roles
Creating role and member remit descriptions, or terms of reference, for the co-design group, and any sub group, helps everyone understand the process and their role in it. Be clear about the distinction between co-design, consultation and consensus.
use sub-groups Placing lots of pages in front of lots of people can create stressful confusion. Create an outcomes sub-group to shape drafts of the theory of change and outcomes framework and report to the main body of stakeholders on their progress, getting approval as they go along. Make sure that everyone in the wider group has sufficient understanding of the process to follow progress.
Prepare draftsUse experienced facilitators to work up an outline theory of change to present to the outcomes sub-group for discussion and amendment and do likewise for each step towards the eventual outcomes framework.
Without supporting explanation, co-designers can find the materials and process new and overwhelming. Having people who were neither funder nor fundee explain the process created an interpretive bridge, or brokerage space, which also helped foster openness to the new approach.
Show leadership Different leadership approaches make a difference. This was clear across the various strands of this project. Personal commitment to outcomes focused funding was infectious, with committed leaders sparking good responses from others.
Create collaborative buy-inUnderstanding alone is not sufficient; buy-in is essential. Being involved in agreeing the theory of change, which is the skeleton for any outcomes focused programme, brings transparency to the entire process. Delivery agents have a far greater ownership of the measures against which they will be held to account when they can see the line back to a framework which they helped to shape.
use group workCo-design is group-work and requires group work skills. Expect and accommodate a ‘form, norm, storm and reform’ pattern.
Recognise the impact of a new partner or representative; when one member of a group changes the entire group dynamic alters. Ensure new members are fully briefed on and understand the process and all materials to date. Provide them with support outside of group meetings if they need to ask questions and get up to speed. Otherwise new members may query earlier stages during business meetings, which risks re-visiting matters already agreed and slows down progress.
Be flexible In the ideal world, ultimate goals are agreed and an outcomes framework constructed before programmes are up and running. The real world, however, is no observer of the right order of things. Operational matters may have to be addressed before costings are drawn up, or delivery plans shaped before budget approval is assured.
The two DfC women’s programmes were an example. Circumstances meant conversations about delivery had to run parallel with the outcomes design process, rather than flow from it. Nonetheless, we eventually brought the two strands together meaning that all work and data collection remained aligned with the programme goal and its contribution to the PfG.
It is important to have an approach to good impact practice and associated tools that are sufficiently flexible to work in the real world of funders.
26
Resurgam Trust: Having explained to a
core of the Trust’s team, we widened out so that
there was a solid foundation of understanding for buy-in to the results
later on.
27
Timing may be tight, or not right, personnel may change and external circumstances alter; these may make a straight line design path unrealistic, but they do not make it impossible. The clear principles, process and practices that underpin our approach mean you can always find and re-join the path when the time is right.
Do not oversimplifyIn designing our tools, we were very aware of the delicate balance between making them sufficiently simple to work across assorted situations yet not going so far that they fail to acknowledge that the issues being addressed are anything but simple.
It is dangerously easy to manufacture approaches which, whilst making outcome metrics crystal clear, over simplify the real world to the extent that they lose any descriptive or predictive power. We guarded against this by elaborating theories of change with rationales and enablers and by incorporating direct experience of multifaceted issues by co-designing with VCSE partners.
By starting off with complex explanations of compound social changes we were able to identify meaningful measures before separating them in to the simple metrics required for report cards and the more nuanced information required for full evaluation and reflection.
DfC. Programme for Women Involved in Community
Transformation: Timing meant that we worked on the outcomes
framework at the same time as activities were being shaped up.
We addressed this by working with a sub-group of delivery agents and
the Department to prepare the outcomes framework and bring
it to the wider design group.
The drawback of this was that the wider group did
not gain the same level of knowledge of the process or the
thinking underpinning it. In an ideal world, the full group would get a thorough introduction to
the process, before the sub-group was delegated to
prepare the drafts.
Delivery team member: “Not one of the strands in
this action learning project exactly conformed to the
planned process, yet all gained improved capacity
to design outcomes centred funding programmes.”
28
This project sought to create outcomes that support the most effective use of funding to deliver positive social change. As an action learning project, our activities were loosely defined so that we could respond to funders’ needs. Now, however, we can draw on project learning to clarify the next steps.
next steps
29
Working with fundersFunders do themselves a disservice by looking almost exclusively at the indirect effect of their funds on individuals. In fact their impact is often wider and affects different beneficiaries to those imagined.
In the context of public sector funding for the VCSE sector, this project suggests that while awareness of OBA™ has grown in the year since the Embracing Change research, there is a continuing need to develop understanding of outcomes in general to support the successful implementation of OBA™ in particular. There is also a need to extend experience and confidence around co-design as an approach to shaping funding programmes. This extends across all sectors.
To support effective outcomes focused funding we need to work with funders to:
1 enable more funders to explore the full range of the effects of their funding and funding processes across direct and indirect beneficiaries;
2 grow understanding of the general principles of outcomes focused funding;
3 foster skills in using co-design to develop outcome focused funding;
4 enhance experience of using co-design to shape service delivery;
5 deepen the skills and experience needed to lead outcomes planning and co-design; and
6 more widely share learning about outcomes focused funding across independent funders, NICS and government.
Working with the VCSe sectorCENI is widening and deepening the VCSE sector’s awareness of outcomes through the Inspiring Impact NI project and the Regional Infrastructure Support Programme; however the sector has limited experience of co-design. Few people have made the transition from consultee to co-designer.
To support the VCSE sector in the move to outcomes focused funding we need to:
7 increase the capacity of the VCSE sector to take part in co-design by increasing awareness, understanding and skills; and
8 deepen the sector’s understanding of how best to respond to the requirements of outcomes focused funding.
Working across sectorsCo-design is a powerful tool when shaping outcomes focused funding programmes because it works across silos and sectors to harness knowledge and align enthusiasm to a shared goal. This sharing does not apply at the planning stage only.
Lessons will emerge as the experience of outcomes focused funding increases, be it in the independent or public sector.
To enhance capacity for co-design and learn from the application of outcomes focused design we need to work across sectors to:
9 gather and share learning about the use of co-design across sectors; and
10 monitor the wider impact of outcomes focused funding on the VCSE sector, public and independent funders.
Action
In addressing each of these points we need and welcome your help so do get in touch if you can:
share ideas, information and insights to help us shape responses;
run your own demonstration project to test new approaches;
link us into wider networks; or
resource the development of the tools or events necessary to success.
30
i Take responsibility for impact and encourage others to do so too.
Impact is embedded in the culture of your organisation. All staff and volunteers see it as their responsibility to create impact through their work, to play their part in showing how their work makes a difference and sharing what they learn from it with others. Leaders of your organisation see delivering impact as their primary responsibility and duty.
ii Focus on purpose.
You’re clear about your purpose as an organisation: the difference that you exist to make. You’re able to explain your expected impact in meeting this purpose. You can describe how you will create this impact, plainly and in ways that everyone can understand.
iii Involve others in your impact practice.
At all stages of impact practice you look for appropriate opportunities to involve others. As a minimum this includes involving those with direct experience of the organisation’s activities. It could also include other organisations doing similar work or funders.
iv Apply proportionate and appropriate methods and resources.
You’re realistic in your impact practice—you apply time, effort and methods proportionate and appropriate to the scale and scope of the work. If necessary, you keep things simple and do what you can to focus on impact. You choose methods that fit with the values and ethos of your organisation, and you’re clear about the purpose of collecting impact information, who will use it and for what.
v Consider the full range of the difference you actually make.
You keep in mind that you might have an impact beyond your intended beneficiaries, a negative as well as a positive impact, or impact that you hadn’t planned. You acknowledge that in some
cases you have to balance positive impact on your beneficiaries with negative impact on others. You seek to understand if your work is having a longer term or wider impact than anticipated. Equally, you know that changes you see may have occurred without your involvement, be short term, or be the result of the work of others.
vi Be honest and open.
You’re honest and open about the impact you plan to have, and your findings on the difference you have or haven’t made, as well as what you have learnt from these findings and how you will act on these lessons. You’re also honest and open about the scope of your impact practice. You and your funders discuss what hasn’t gone so well and learn from it.
vii Be willing to change and act on what you find.
You’re ready to change as a result of finding out what impact you are or are not having, and to accept you may not be having the impact you intended. You commit to learning from your impact assessment and you actively use it to inform planning and action. Your funders are also willing to learn and change.
viii Actively share your impact plans, methods, findings and learning.
You communicate your impact plans, methods, findings and learning to ensure others know what you’re trying to achieve and to contribute to a wider view of what does or doesn’t work. You share appropriate information inside and outside the organisation. You acknowledge the limitations of your findings.
AppendicesAppendix One - The Code of Good Impact Practice. Key principles.
31
Funders’ Principles
1 Apply proportional and appropriate rigour and resources
2 Be flexible, open and transparent
3 Acknowledge our respective independent values
4 Recognise that everyone can contribute to impact practice.
Drivers of good impact practice for funders
There is a key driver for each stage of the impact practice cycle. Here are some practical actions for funders.
PLAn
Driver 1: Be clear about the difference you, and those you support, want to make.
Define and articulate the difference you intend to make.
Seek to understand the difference the people and organisations you support intend to make, and the contribution your support will make to this.
Identify the type and level of evidence needed to assess the difference made.
Communicate intended impact and discuss impact practice with people and organisations you support.
Explore a range of research methods and impact assessment tools.
Consult with stakeholders in the development of your approach.
Seek to identify sources of shared data and the potential for shared measurement.
DO
Driver 2: Support people and organisations in their impact practice, and resource your own impact practice.
Think ‘impact’ and invest resources in making a difference.
Put in place systematic approaches for gathering relevant evidence for your own work.
Encourage applicants to cost in an appropriate level of external or self-evaluation in funding proposals, or embed impact assessment in a ‘funding plus’ support package.
Work with the people and organisations you support to enable them to gather relevant evidence.
Engage and collaborate with others, and seek to use shared measurement approaches as appropriate.
ASSeSS
Driver 3: Identify the difference made and assess how and why it was made.
Take steps to ensure that the data you gather and evidence you report is robust.
Seek to assess the contribution that your funding has made to the people and organisations you support.
Explore what others are doing in the field.
ReVIeW
Driver 4: Share and act on learning and seek to improve impact practice.
Share learning with the people and organisations you support and other delivery stakeholders to help inform their impact practice.
Seek to identify who else might benefit from the learning.
Share learning with external stakeholders to help inform policy development where appropriate.
Use learning from impact evidence to inform your strategy and policy, and to help prioritise your grant-making and support.
Regularly review your impact practice.
Seek feedback from people and organisations you support on your impact practice.
Offer feedback to people and organisations you support on their impact practice.
Amend your impact practice in light of feedback and learning
Appendix Two - Funder’s principles and drivers of impact practice. Summary version
Awareness and
understanding
Assessment and Action
Identification
Designing Outcomes Focused Funding ProgrammesCreating programmes that align with organisational or PfG outcomes
Sharing Learning
Funder type Body / Programme or Policy Project or Team Presentations Measuring up! Overview Theory of
Change Indicators Measures - Report Cards Data Tools Co-design Further
Meetings
Dept. for Communities Fresh Start Agreement.
Section A 3.9 “Ending paramilitarism and tackling
organised Crime” B5 “Support for transition”
CASE STUDY 1
Programme for Women Involved in Community
Transformation Team, VCD• • • • • •
Above with Women’s Early Intervention Programme
Team, VCD • • • • • • •
Resurgam Trust CASE STUDY 2
Policy and Innovation Team, VCD • • • • •
Resurgam Trust and Fermanagh Trust - TBUC
Policy and Innovation Team, VCD • • •
Social Innovation within Innovate NI
Policy and Innovation Team, VCD • • • • •
Potential future delivery of Social Enterprise Innovation
Hubs
Policy and Innovation Team, VCD and DfE and Invest NI • • • •
Generalist advice services VCD Programme team • •Volunteer infrastructure
support programme (VISP)VISP partners •
NDPB Arts Council of Northern Ireland
Research and Arts Officers •Causeway Coast and Glens
BoroughPolicing and
Community safety • • • •Derry City and Strabane
DistrictArts and culture •
Independent Big Lottery Fund NICASE STUDY 3
All teams across the NI portfolio • • • •
32
Appendix Three - Participants and strands in the Funding outcomes action learning project
Awareness and
understanding
Assessment and Action
Identification
Designing Outcomes Focused Funding ProgrammesCreating programmes that align with organisational or PfG outcomes
Sharing Learning
Funder type Body / Programme or Policy Project or Team Presentations Measuring up! Overview Theory of
Change Indicators Measures - Report Cards Data Tools Co-design Further
Meetings
Dept. for Communities Fresh Start Agreement.
Section A 3.9 “Ending paramilitarism and tackling
organised Crime” B5 “Support for transition”
CASE STUDY 1
Programme for Women Involved in Community
Transformation Team, VCD• • • • • •
Above with Women’s Early Intervention Programme
Team, VCD • • • • • • •
Resurgam Trust CASE STUDY 2
Policy and Innovation Team, VCD • • • • •
Resurgam Trust and Fermanagh Trust - TBUC
Policy and Innovation Team, VCD • • •
Social Innovation within Innovate NI
Policy and Innovation Team, VCD • • • • •
Potential future delivery of Social Enterprise Innovation
Hubs
Policy and Innovation Team, VCD and DfE and Invest NI • • • •
Generalist advice services VCD Programme team • •Volunteer infrastructure
support programme (VISP)VISP partners •
NDPB Arts Council of Northern Ireland
Research and Arts Officers •Causeway Coast and Glens
BoroughPolicing and
Community safety • • • •Derry City and Strabane
DistrictArts and culture •
Independent Big Lottery Fund NICASE STUDY 3
All teams across the NI portfolio • • • •
33
Snow
The room was suddenly rich and the great bay-window wasSpawning snow and pink roses against itSoundlessly collateral and incompatible:
World is suddener than we fancy it.
World is crazier and more of it than we think,Incorrigibly plural. I peel and portion
A tangerine and spit the pips and feelThe drunkenness of things being various.
And the fire flames with a bubbling sound for worldIs more spiteful and gay than one supposes –
On the tongue on the eyes on the ears in the palms of one’s hands –There is more than glass between the snow and the huge roses.
Louise Macneice. 1936. Faber Book of Modern Verse. Written in 77 Malone Road, Belfast
Behind every managerial decision or action are assumptions about human nature and human behaviour.
It is probable that one day we shall begin to draw organization charts as a series of linked groups rather than as a hierarchical
structure of individual “reporting” relationships.”
Douglas McGregor. 1960. The Human Side of Enterprise
34
navigating Change
Published by Inspiring Impact NI @ CENI
129 Ormeau Road Belfast BT1 1SH
T +44 (0)28 9024 8005 @iimpactni
June 2017
