Upload
jared
View
58
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Creating Value, Evaluating Impact. ‘Self-evaluation is a vital part of the discipline imposed in undertaking creative work’ (Moriarty, 2002). Kerry Traynor November 2012. What is Evaluation and why do it? A bout the Project Measuring Impact Theories and Models Next Steps. - PowerPoint PPT Presentation
Citation preview
Creating Value, Evaluating Impact
Kerry Traynor November 2012
What is Evaluation and why do it?About the ProjectMeasuring ImpactTheories and ModelsNext Steps
‘Self-evaluation is a vital part of the discipline imposed in undertaking creative work’
(Moriarty, 2002)
What is evaluation?Evaluation involves gathering evidence
before, during and after a project and using it to make judgements about what happened.
The evidence should prove what happened and why, and what effect it had.
The evidence can also help you to improve what you are doing during the project and what you do next time.
(Woolf, 2004 cited in Arts Council, 2011)
Why evaluate?‘Self-evaluation is hard work and time-consuming. The
reward is that it can give us the ability to do things beyond the best of our present available knowledge’
(Moriarty, 2002)
evaluation helps with planning, as it makes you think about what you’re aiming to do, how you will do it and how you will know if you’ve succeeded
on-going feedback keeps you on track and helps to avoid disasters evaluation helps you to adapt/change as you go along evaluation is a good way of dealing with ‘quality assurance’ – you’re
keeping an eye on things to make sure quality is maintained evaluation helps prove the value of what you are doing evaluation records your contribution to the field you are working in your evaluation can help others working in the same field information you collect can also be used for reporting back to those with
an interest in the project (eg participants, funders) and telling others about what you’ve done
the evidence you collect can support future funding applications (Arts Council, 2011)
Evaluation helps the arts… Rigorous evaluation of our work enables us to : accumulate a collective body of evidence contributes to the ‘collective practice wisdom of the sector’ builds a record of our ‘history and achievement’
(Arts Victoria, 2002)
Cannot evaluate all projects in the same way and in the same depth Can be more or less formal, and more or less detailed Artistic judgements about process, materials, form and content Judgements about the results of what you did and what you have
produced Process as well as product - quality and impact of both
(Arts Council, 2011)
Planning your evaluation: What kinds of information or evidence you are going to include in
your evaluation e.g. what people say, what they have done (process and finished
work), what you have done, how participants/audiences responded? What questions you are going to ask? How you plan to answer those questions –what sort of information
you need to answer the questions and how you will collect it. When you should collect the information. How you will collect the information? e.g. keeping a register, asking people in a questionnaire, asking them
to video their thoughts about a project, keeping a diary, taking photographs, etc.
How you are going to make sense of the information you have collected?
How you are going to present the results of the evaluation? Who you are going to share it with and how?
(Arts Council, 2011)
About the Project
Awards for All funding / Nov 12 – Aug 13
Aims Understand how impact is currently measured and evaluated in small
enterprises in the creative industries Understand the strengths/weaknesses/opportunities/risks of evaluating
impact Review models of impact evaluation and identify appropriate model(s)
for use in creative communities network Identify needs and deliver support to evaluate impact more effectively
Outcomes◦ Impact Evaluation Strategy for Safe Productions and the Creative
Communities Network – finding a shared approach that can measure the impact of the whole network
◦ One-to-one support for members in implementing impact evaluation◦ Interactive blog / web archive of useful info
http://evaluatingimpact.wordpress.com/
Background Charity Manager in Toxteth for 6 yrsDirector / Chair of Safe Productions for 10 yrsGrants Assessor for BBC CIN for 10 yrsResearcher & lecturer at LJMU for 8 yrs Manager of Community Media EnterpriseMSc in Governance (Creative Industries)PhD in Impact of Local Media
What do we mean by impact?Inputs + Outputs = Outcomes Outcomes – What would have happened anyway = ImpactDifference achieved by a project/serviceSocial, environmental, financial
ExerciseIntroduce yourself and your organisationWhat do you believe are the main
impact(s) of your organisation?What is your experience of impact
evaluation?What has worked well for you? What hasn’t worked so well? What would you like to do better?What do you hope this could achieve?What are your fears / concerns?
Models of Impact Evaluation Social Return on
Investment (SROI) European Foundation for
Quality Management (EFQM) Excellence Model
Prove It? Quality First Social Enterprise
Balanced Scorecard SIMPLE 3rd Sector Performance
Dashboard AA1000 Assurance
Standard
The Big PictureVolunteering Impact
Assessment Toolkit ISO9001:2008Global Reporting
Index (GRI) Investors in People
(IIP)Eco-MappingEU Eco-Management
and Audit Scheme (EMAS)
PQASSO
Why SROI?Robust methodology developed
through academic peer review process
Flexible, universally applicable(Relatively) straightforwardEndorsed by Cabinet OfficeBecoming common practise
SROI: A simplified example Inputs : £20,000 Outputs : 20 unemployed people complete a 6 month
training programme and get a qualification Outcomes : 5 people get a job But : 2 people would have got a job anyway Impact : 3 people move into employment Attach monetary values (returns) to that impact e.g.
◦ Reduced costs to state of paying unemployment benefits ◦ Increased income to the state from employment taxes
Calculate returns over 5 years Divide total returns by investment e.g.
◦ £100,000 / £20,000 = 5:1 Or, for every £1 invested, £5 is created in benefit for
society
A collective approach – find your common impacts
Moving people towards employment / rehousing / financial independence
Improving the physical environment Improving people’s health & wellbeing
Agree common methodology
Measure and evaluate over time
Convert into monetary values
Find your individual and collective Social Return On Investment Rates
H&W: A simplified exampleWEMWBS 7 or 14 point
questionnaire5 response codesFor participants aged 13+Each participant completes at
start/endPoints awarded for each response
H&W: A simplified exampleOver 1 year, 500 people
participate across all member organisations
Results in an average increase of 5 points per person, or
A total increase of 2,500 points across the area
Can be analysed by postcode/age/gender etc.
Visualising our collective impact
Visualising our collective impact
Visualising our collective impactThe Health and Wealth of Nations
The Decline: The Geography of a Recession
But does that really capture impact… ?
Can we really put a price on social impacts? Will SROI let us tell the whole story? Or will we need more? Eg. case study, images, quotes, narrative etc. Creativity and the Creative Industries – arts & health Community development – what is valuable and sustainable? How
do we build strong and resilient communities? Social Capital theory – our social, economic and cultural stock
(Bourdieu, 1984) The ties that bind us - bonding & bridging (Putnam, 2000) Decline in social capital = decline in moral and social standards Increasing social, democratic and civic engagement Pride, shared identity, shared vision - challenging the negative,
promoting the positive Alternative socio-economic models – 21 hours (nef) Can we use these theories and models to find a better way to
demonstrate our impact?