Measuring Quality in the Cultural Sector Standardised Metrics + Big Data = New Data Culture?

‘What Does Data Have To Do With Me?’ British Museum, June 5th

John Knell, Culture Counts

Our Mission

To help the cultural sector measure the quality of their work on their own terms,

and to communicate the value they create through big data

Anybody else feel lost?

What We Found:

• No Common Language• Struggling to define ‘intrinsic’ elements of value• Very little info sharing or benchmarking• No rigorous assessment of quality of outcomes post event• Manual, old-fashioned research• No Data Culture – ‘the plural of anecdote is not data’

How do we measure?

• Triangulation of evaluation by self, peers and public

• Online data capture through the Culture Counts system via a web app or via post event surveys (urls posted via email)

• Current quality of experience metrics have 9 dimensions evaluated by all 3 groups

• 5 extra dimensions evaluated by self and peers

Quality dimensions (self, peer, public)Dimension Metric Statement

Rigour ‘It was well thought through and put together’

Distinctiveness ‘It was different from things I’ve experienced before’

Captivation ‘It was absorbing and held my attention’

Relevance ‘It has something to say about the world in which we live’

Meaning ‘It meant something to me personally’

Challenge ‘It was thought provoking’

Enthusiasm ‘I would come to something like this again’

Presentation ‘It was well produced and presented’

Local impact ‘It is important that it’s happening here’

Quality dimensions (self & peer only)

Dimension Metric Statement

Concept ‘It was an interesting idea / programme’

Originality ‘It was ground-breaking’

Risk ‘The artists / curators really challenged themselves with this work’

Excellence (global)

‘It is amongst the best of its type in the world’

Excellence (national)

‘It is amongst the best of its type in the UK’

Simple interface for respondents

Average self, peer and public scores for ‘In My Father’s Words’ (awarded after the event)

What types of insights are generated?

• The triangulation element is vital to the insight and reflection process

• All responses are interesting in their own right, but cultural organisations tell us that it is the triangulation that generates new insight for them on the extent to which they are achieving their creative objectives

• Being able to anticipate how audience members and peers will respond to a work is a reflection of the quality / maturity of creative and cultural leadership in an organisation

What types of insights are generated?

• Integrating quality metrics with audience size and profile and financial data allows for a sophisticated assessment of– How successfully organisations balance creative,

commercial and audience objectives– How different kinds of cultural experience create

different kinds of value• Developing prompts to help organisations

understand their evaluation results and use them to reflect on their creative practice

The data is only a starting point for a richer

conversation about your creative practice and

audience relationships

Elongate and Enhance the Cultural Experience

• There is an untapped demand for audiences to give feedback and enter into a richer dialogue about their cultural experiences

• Culture Counts and this type of approach offers a potentially powerful way of elongating and enhancing the cultural experience

• Integrate with social media to increase the sense of shared experience

Contact US

John@john-knell.com www.culturecounts.cc