Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Norfolk Arts & Wellbeing Programme
Evalua8on process and headlines Dr Anni Raw, Durham University
Ac8on Research Evalua8on • Two ques)on areas: – What outcomes? – What learning?
• Evalua)on material of differing kinds, for different purposes
• Quan)ta)ve data (data that is quan)fiable) = against KPIs (Key Performance Indicators)
• My role: maximise the ‘ac)on learning’ dimension – transferable learning from the process
Evalua8on Framework • Ambi)ous strategy agreed • Par)cipatory evalua)on process: collec)ve vision – aspira)ons for the work – Group consensus process (ICA), led by Mike White
• Criteria for assessing achievement of desired outputs: agreed collec)vely – drawn from specialist knowledge of consor)um partners…
• A rich, extensive evalua)on framework created • (the following slide shows example page – 1 of 7)
Outcome area
Success criteria
Evidence (how do we know if this is happening?)
Other evidence Examples (what this looks like in prac)ce – e.g.s)
S8mula8
ng and
enjoyab
le ac8vity th
at im
proves
health and
wellbeing
Improved wellbeing, Posi)ve social impact
• Posi)ve change recorded: 5 steps to wellbeing, or WEMWS
• Change in YP aWtude and behaviours to others
• Posi)ve change in YP own self assessment
For YP and older people: Feel less isolated Learnt /done something new Altered aWtudes/ Behaviours For personal budget holders Improved confidence, communica)on, teamwork Be\er access to info Be\er informed decisions
A\endance at sessions A\en)on/engagement Interac)vity Collabora)ve working Willing to try things Willing to persevere Asking ques)ons Explaining to others Eye contact Improved concentra)on
Increased exposure to enjoyable arts experiences/ crea)ve expression
• Involved in s)mula)ng C&C ac)vi)es
• Experiencing new C&C ac)vi)es
• Opps for crea)ve self-‐expression
• Enjoyment, fulfilment, sense of achievement
A\ending, taking part Producing crea)ve outputs Discovering new talents, interests Greater confidence in own ideas and crea)vity
Arriving on )me; staying for whole sessions; returning each week; keen to have a go; crea)ng work; prepared to share work with others; not dismissing or prejudging own work
Recogni)on of individual achievements
• Sharing outcomes / achievements within group
• Sharing with wider group, family, other residents
• Achieving Arts Award credits
End of session sharings Work displayed in pubic spaces Work displayed in residents’ rooms Enrolment on Arts Award Commitment to comple)ng Arts Award
Curiosity overriding nega)vity Taking pride in comments of others Showing enjoyment in own crea)vity Achievement of Arts Award
My role in the process
3 overarching ques)ons for my brief -‐ What has the programme contributed to understanding:
1. How effec*ve arts and cultural interven*ons are in addressing health and wellbeing agendas?
2. What opportuniAes and challenges are presented to organisaAons working collaboraAvely in this field?
3. What benefits there are (financial, organisaAonal and qualitaAve) to commissioners of the CreaAve CommuniAes ConsorAum model of working?
My Processes
• Interviews with delivery partners • Colla)ng documenta)on from partners • Encouraging narra)ve-‐based approaches*: – Mapping project narra)ves/journeys
• Dis)lling and reflec)ng back learning from the delivery experiences
*2 examples of narra)ve tools in use:
Example of Crea8ve Scrapbook in use King’s Lynn Arts Centre Trust
Evidence that the programme has provided ‘s*mula*ng and enjoyable ac*vity that improves health and
wellbeing’?
Including the following descriptors: Increased exposure to enjoyable arts and cultural experiences and crea)ve expression? • Involvement in s)mula)ng crea)ve and cultural
ac)vi)es • Experiencing new crea)ve and cultural ac)vi)es
not undertaken before • Opportuni)es for crea)ve self-‐expression • Gained enjoyment and sense of fulfilment and
achievement
Numerous examples across all projects One example of offering something very new: ‘... we were asked to conduct a ‘passive’ ac)vity because of the severity of the levels of demen)a amongst par)cipants. We did so but built in op)onal par)cipatory involvement if par)cipants wanted to join in. ...Our ar)sts created an immersive experien)al sound environment using looped synths and ambient tones including the use of a Hang Drum. Many of the residents decided to join in.’
Recogni8on of individuals’ achievements?
Including the following descriptors: • Sharing outcomes and achievements within the group • Sharing achievements with wider group, family, other residents • Achieving Arts Award credits
This aspect generally naturally built in to project ac8vi8es
(e.g. performances and sharing events) ‘We had a showcase – a premier night screening of the par)cipant’s film, with a red carpet, and posh guests, and a professional photographer. There were all sorts of comments -‐“I’ve never been on a red carpet before”, “I’ve never had a professional photograph taken before!”’
Improved wellbeing of users, and posi8ve social impact?
Including the following descriptors: • Posi)ve change recorded through Five Steps to Well-‐being or WEMWBS assessments • Evidence of change in young people’s aWtude and behaviour to others • Evidence of posi)ve change in young people’s own self-‐assessment
Young Carers’ Transi8on Worker Comments
“They were so enthusias)c. Below is a text message I received upon reaching home. It brought tears to my eyes (several )mes during the evening I was in a similar state!) ‘I haven't had that much fun in ages, well since my brother died -‐ thank u soo soo much I'm so grateful xx’ What I saw last night was a rarity – they let themselves go and forgot their responsibili)es and had FUN.”
A symbol of impact: Commitment Con)nued commitment and a\endance by young people long aoer the project ended • ‘We’re s)ll in touch with all the young people we worked with in the project. We’d never met the group before this, they didn’t know we existed, didn’t know how we might be able to help them or what we were. But now they are coming back to us for a number of reasons... to see shows, exhibi)ons, do the arts award with us, work experience, volunteering...
• They are avid par)cipants. If I emailed them right now about an event, they would all come.. One young person from the group came in last week, for help to add what he’d done in the project into his UCAS applica)on/personal statement, because he though it said something valuable about him.’
Wider recogni8on of the value of arts and culture interven8ons to health and wellbeing?
Eg: Posi)ve change in aWtude of health and social care professionals to role of arts and culture: • Health and social care staff know how to access informa)on about arts and culture opportuni)es • And have be\er understanding of benefits of crea)ve interven)ons for clients and users
‘Catalyst’ of having funding to work with new partners; partners become enthusias8c to repeat ‘There has been follow up from a school (a commissioner) – so the pathway forward may not be so much through ‘robust evidence’, but through partnership experience – hearts and minds, empirical, experien)al awareness – just being there, and knowing your kids. What is happening communicates already quite effec)vely to partner/commissioners...’
‘We delivered 4 sessions to the Hospital, and they were such a success, the hospital has now commissioned us to work with them on a regular basis...’
Quote below adds insight into achieving wider understanding across sectors:
‘As a health and social care worker, I couldn’t deliver half of the outcomes without doing the Lord Mayor’s procession, and the arts and craos, the radio, making things and stuff [So would you choose to do those kinds of cultural acAviAes again?] Yes, absolutely. All those things add so much value, but we need to talk about it in the right way... There’s an issue with language. The terms ‘arts and culture’ might be something that many people don’t understand. People understand through experiencing it.’
Summary findings of the qualita8ve evalua8on of the Programme:
1) Extremely valuable and rich wellbeing-‐focussed arts and cultural work;
2) par)cularly strong in inclusion of groups rarely accessing such ac)vity;
3) Strong wellbeing impacts in situ, (captured in photographs and quotes, & reflec)ons by deliverers);
4) Persuasive enough to create new opportuni)es for con)nua)on or repeat work, through partners seeing wellbeing impact on the groups they know so well;
Con8nued… 5) Evalua)on systems designed to capture
qualita)ve data rather patchy, (overload of evalua)on processes, and hiccoughs in the evalua)on strategy);
6) Enormous learning reported to me as evaluator during the la\er stages of the programme.
(The subject of the main evalua)on report.)