Upload
caroline-king
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Measuring OutcomesHow do we understand the changes we are making …
Workshop for Funding Fair 2013
supporting voluntary
andcommunity
action
VAW’s Just For You project
Using mainly the Charities Evaluation Services (CES) outcomes-focussed approach to planning, monitoring and evaluation to:
1. Make your work more effective.2. Improve how you communicate the
outcomes of your services (especially to funders).
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
What’s the story?
1. Clear about the changes (outcomes and impact) that we are seeking to achieve.
2. Clear about the relationship between these and what we do.
3. Identifying quantitative and qualitative indicators of changes we make.
4. Robust measurement methods.5. Clear and concise reporting.
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Outcomes Planning Triangle
Overall Aim or
Mission
Specific Aims: This is why we do our
work
Objectives:This is what we do
Need: a problem or issue, or situation where something needs to change to make things better, for a person, a group of people, an environment or an organisation.
Outcomes: the changes, benefits, learning or other effects that happen as a result of our work.
Inputs: the resources put into a project to carry out an activity.
Outputs: the detailed activities, services and products we do or provide.
Impact: lasting change brought about by our work
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Task – 5-10 minutes
• Think about the Outcome or Outcomes (change) that motivate you most – use the triangle if it helps.
• Share with your neighbour
• Not a test! – rather a way for us all to focus our thinking on the changes that we each individually are involved in bringing about.
supporting voluntary
andcommunity
action
Need: children are falling behind at school because of problems at home
THEORY OF CHANGE The beginning of making a difference Angela Kail, Tris Lumley NPC April 2012
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Making sure that outcomes are realistic
• Many charities have grand aims, such as ‘the alleviation of poverty’. But aims like these are too large for a charity to achieve on its own, so it is not sensible to think about how to measure them.
• A plan for change helps charities focus on concrete, defined aims and outcomes, which are potentially measurable.
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Understanding progress towards the final goal
• Some final goals cannot easily be measured. They involve change that happens too gradually or change that happens in the lives of people who are difficult to track.
• Planning intermediate steps that lead to the end goal can help organisations work out whether they are making a difference towards that end goal, by measuring the intermediate steps.
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Measuring outcomesOutcome How will we know?Reduced isolation for older people
There will be a change in the number of older people stating that they feel isolated
Reduced vandalism in the area There will be a change in the number of incidents reported
Small voluntary organisations have a better understanding of income generation
There will be a change in the level of understanding of income generation within small organisations in the area (and longer term a change in income)
http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Outcome indicators
• Outcome indicators are well defined pieces of information that can be assessed or measured to show whether outcomes have been achieved.
• Outcomes indicators enable us to measure progress towards planned outcomes
• They are signs or clues that you monitor and can be qualitative or quantitative.
• Each outcome has at least one indicator - some have many indicators.
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Outcomes indicators – outcome ‘people are ready for work’
• An example of a quantitative indicatormight be the number and type of qualifications gained by people attending a training centre. • An example of a qualitative indicator mightbe the change in confidence of those attending the centre (whether it increased, decreased or stayed the same).
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Robust measurement methods 1
• Knowing from the start what the need is, what change you seek to make (aim), the change you will make (outcome), the ways that change - and any others - will be demonstrated and measured (outcome indicators) …. you already know the what and why.
• So what is left is HOW?
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Robust measurement methods 2
• Collect only the information you need and can use
• Use methods that are sensitive to the people you want to collect information from
• Involve others in the choice and design of your information collection methods
• Fit this info collection into other work processes as much as possible.
supporting voluntary
andcommunity
action
• Interviews• Self-completion tools (surveys, tests, forms,
diaries)• Observation• Keeping records and notes• Group activities – focus groups, group
interviews, • Visual/audio methods – photos, video, voice
recordings• Social media – blogs, podcasts, twitter
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Task
Identify with your neighbour one way each that you are familiar with or have used recently to collect information about your work.• Did the information demonstrate a change? • If so, what was that change?• Was it quantitative or qualitative info?• How robust was it – is it open to challenge,
will it ‘fall down’ if questioned?
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Assessing Change: Developing & using outcomes monitoring tools *
1. Planning your outcomes monitoring2. Selecting appropriate methods3. Developing monitoring tools
• Designing your own tools• Finding and using off-the-shelf tools
4. Preparing to collect outcomes information5. Collecting outcomes information*By Diana Parkingson and Avan Wadia for CES Oct 2010http://
www.ces-vol.org.uk/Publications-Research/publications-free-downloads/publications-free-assessing-change
http://www.biglotteryfund.org.uk/funding/funding-guidance/applying-for-funding/
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Overall aim
Specific aims:This is why we do our work
Objectives:This is what we do
Clear and concise reporting
1) Clear purpose
2) Defined aims
Outcomes: 4) Demonstrate results
Outputs: 3) Coherent activities
5) Evidence
6) Lessons learned
C) Evaluation
A) Planning B) Monitoring
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
Discussion and
Questions
supporting voluntary
andcommunity
action
supporting voluntary
andcommunity
action
THANK YOUEmma Plouviez
Organisational Development OfficerVoluntary Action Westminster