Understanding online audiences ara conf 28 aug 15 martin bazley upload version

Understanding online audiencesUnderstanding online audiencesUnderstanding online audiencesUnderstanding online audiencesPlanning and implementing

research into online audiences

ARA conference, Newcastle 28 August 2014

Martin BazleyMartin BazleyMartin BazleyMartin BazleyDigital Heritage Consultant

Martin BazleyPreviously:• Teaching (7 yrs)• Science Museum, London, Internet

Projects (7yrs)• E-Learning Officer, Museums,Libraries and

Archives (MLA) South East (3yrs)• Founder: Digital Learning Network DLNET

Reduced version: most images removed

Martin BazleyNow:• Developing online resources, websites,

user testing, evaluation, training, consultancy…

• HLF digital projects Mentor and Monitor

• Martin Bazley & Associateswww.martinbazley.com

This session: raise awareness - not in depthThis session: raise awareness - not in depth

Why do we need to evaluate and Why do we need to evaluate and research our audiences?research our audiences?

What tools are there for gathering What tools are there for gathering

data?data?

How do you plan audience research?How do you plan audience research?

Why do we need to evaluate Why do we need to evaluate and research our audiences?and research our audiences?

Why not just Why not just

'design things for users'?'design things for users'?

Web users also sometimes don’t 'get it'

People use the web differently…

… from the way they access printed material books, object labels in galleries,

magazines, newspapers, and information screens, etc

For most people the web is a predominantly

visual medium

Whose site is this?

Don't Make Me Think by Steve Krug

Classic, entertaining introduction to improving website usability

Your Your usersusers

Your users• Who uses your website?

• Why would they want to use it?

• How would they find it?

• What do they get out of it?

• What do they dislike about it?

Who are your users?Who are your users?

Different users, different needs...How do you get it right for everyone?

You can’t get it right for everyone.

You need to make clear decisions such as...• Who is it for? • What does it offer them?• How will they use it?

To do that, you need to find out about your usersfind out about your usersfind out about your usersfind out about your users

What do teachers want?

https://vimeo.com/18867252https://vimeo.com/18867252https://vimeo.com/18867252https://vimeo.com/18867252 Oli Knight timesaver Oli Knight timesaver Oli Knight timesaver Oli Knight timesaver

https://vimeo.com/18888798https://vimeo.com/18888798https://vimeo.com/18888798https://vimeo.com/18888798 key ideas not lesson key ideas not lesson key ideas not lesson key ideas not lesson plansplansplansplans

This is what This is what This is what This is what this teacher this teacher this teacher this teacher wants - fairly common wants - fairly common wants - fairly common wants - fairly common viewviewviewview

What tools are there for What tools are there for gathering data?gathering data?

Data gathering tools

• Qualitative: Qualitative: Qualitative: Qualitative: focus groups, “free text” questions in surveys, interviews

• Quantitative: Quantitative: Quantitative: Quantitative: web statistics, “multiple choice” questions in surveys, visitor tracking

• Observational:Observational:Observational:Observational: user testing, ethnographic

Online surveysSurveyMonkey

www.surveymonkey.com

Web statsGoogle Analytics GA

The best way to learn GA is to use it:

www.google.com/analytics/

Web stats: Focus on trends rather than absolute values

Planning audience researchPlanning audience research

How can we get a sense of who our visitors are and how they react to what we offer?

How do we gather data to help us improve what we do?

How do we measure success from the user's point of view, and / or against our own objectives and constraints?

For example, how justify investment (or lack of it) in social networks etc?

Define audience Define audience Define audience Define audience research goalresearch goalresearch goalresearch goal

Analyse data

Collect data Use results to guide changes

Plan methodology

Define audience research goal

Analyse data


Plan methodologyPlan methodologyPlan methodologyPlan methodology


Analyse data

Collect data Collect data Collect data Collect data Use results to guide changes

Plan methodology


Analyse dataAnalyse dataAnalyse dataAnalyse data


Plan methodology


Analyse data

Collect data Use results to guide Use results to guide Use results to guide Use results to guide changeschangeschangeschanges

Plan methodology

Define audience Define audience Define audience Define audience research goalresearch goalresearch goalresearch goal

Analyse data


Plan methodology

SCA guidancehttp://sca.jiscinvolve.org/wp/audience-publications/

Good overview Step by step approach

Lots of sources of information:Culture 24 Let’s Get Real

http://weareculture24.org.uk/projects/action-research/

Trimptonshire Archives is a small local authority record office.

A small number of items across various collections have been digitised on an ad hoc basis and some have been available online for just over a year. There is a searchable online catalogue.

As part of a funding bid, target audiences identified were: • Schools • Higher Education courses• Specialists• Interested individuals• Family researchers

Online Audiences workshop activity

Online Audiences workshop activity

No audience research has been carried out so far.

Overall aim: improve online provision for usersSuggested objective for this research: assess user

satisfaction of current website and identify options for improvement

ActivityActivityActivityActivity Work in a small group. Decide on a project manager. They have final say

during discussion, and also present research approach at the end. Refine research objectives, identify the information you want and choose data-

collection methods. Agree an audience research plan to get data gathered and analysed within 16

weeks, using no more than 15 days’ staff time and £4500 – see table.

Data gathering activity Staff time (days)

Timescale (weeks)

Costs (£)

Online survey (in-house) 6 8 200

Online survey (consultant) 2 8 800

Phone survey (in-house) 6 3 200

Phone survey (consultant) 3 3 1200

Focus groups (in-house) 7 5 200

Focus groups (consultant) 2 5 1500

Web analytics (consultant) 1 2 500

User testing (in-house) 4 3 200

User testing (consultant) 1 3 900

Analysis (in-house) 5 2 0

Analysis (consultant) 2 2 1200Not more than: 15 days 16 weeks £4500

(These are not real values, and anyway are highly variable.)

Don’t spend too long on the figures – focus on the rationale for using each data collection method, and overall objectives.

Remember to consider what you will actually do with the data once you have it.

Crit roomCrit room

Crit room

Simulated user testing- Learn how user testing works- Get feedback on specifics of websites

Remember this is just a simulation of real user testing!

Some issues for archivesSearch our catalogue: "What do I search

for?Content:

"Why can't I click through to read the documents?" "It is hard to make out what it says." (transcripts) "Why would I want to look through this stuff?"

Solutions?'Educating' people on how to use your

existing website and catalogue is an uphill struggle. They don't have to use your site.

A better approach is to help them want to.

Ways people use online collectionsBrowsers - Followers - Searchers - Researchers (MHM)

To engage Browsers Browsers Browsers Browsers you need a few strong 'jewels' / in-your-face interesting stories

FollowersFollowersFollowersFollowers: accesible narrative contentSearchersSearchersSearchersSearchers: may search for family name or

pet topic - offer suggestions for onward links / structured searches

ResearchersResearchersResearchersResearchers: just leave them to it - they will put up with anything!

www.slideshare.net/martinbazley

www.digitallearningnetwork.net

www.martinbazley.com

Martin: 0780 3580 727 - phone for a chat

Extra slides not used in session

Some of these may be useful - please feel free to call for clarification or more info

Strengths and weaknesses of Strengths and weaknesses of Strengths and weaknesses of Strengths and weaknesses of different data gathering different data gathering different data gathering different data gathering

techniquestechniquestechniquestechniques

Data gathering techniquesUser testingUser testingUser testingUser testing

- early in development and again near endOnline questionnairesOnline questionnairesOnline questionnairesOnline questionnaires

– emailed to people or linked from websiteFocus groupsFocus groupsFocus groupsFocus groups

- best near beginning of project, or at redevelopment stage

Visitor surveysVisitor surveysVisitor surveysVisitor surveys - link online and real visits

Web statsWeb statsWeb statsWeb stats- useful for long term trends /events etc

Need to distinguish between:

Diagnostics Diagnostics Diagnostics Diagnostics – making a project or service better

Reporting Reporting Reporting Reporting – to funders, or for advocacy

Online questionnairesOnline questionnairesOnline questionnairesOnline questionnaires(+) once set up they gather numerical and

qualitative data with no further effort – given time can build up large datasets

(+) the datasets can be easily exported and manipulated, can be sampled at various times, and structured queries can yield useful results

(–) respondents are self-selected and this will skew results – best to compare with similar data from other sources, like visitor surveys

(–) the number and nature of responses may depend on how the online questionnaire is displayed and promoted on the website

Focus groupsFocus groupsFocus groupsFocus groups

(+) can explore specific issues in more depth, yielding rich feedback

(+) possible to control participant composition to ensure representative

(–) comparatively time-consuming (expensive) to organise and analyse

(–) yield qualitative data only - small numbers mean numerical comparisons are unreliable

Visitor surveys Visitor surveys Visitor surveys Visitor surveys



(–) responses can be affected by various factors including interviewer, weather on the day, day of the week, etc, reducing validity of numerical comparisons between museums

Web statsWeb statsWeb statsWeb stats(+) Easy to gather data – can decide what

to do with it later(+) Person-independent data generated - it

is the interpretation, rather than the data themselves, which is subjective. This means others can review the same data and verify or amend initial conclusions reached

Web statsWeb statsWeb statsWeb stats(–) Different systems generate different

data for the same web activity – for example no of unique visits measured via Google Analytics is generally lower than that derived via server log files

(–) Metrics are complicated and require specialist knowledge to appreciate them fully

Web statsWeb statsWeb statsWeb stats(–) As the amount of off-website web

activity increases (e.g. Web 2.0 style interactions) the validity of website stats decreases, especially for reporting purposes, but also for diagnostics

(–) Agreeing a common format for presentation of data and analysis requires collaborative working to be meaningful

Activity: Planning an audience research project

When to evaluate or test and why

• Before funding approval – project planning

• Post-funding - project development

• Post-project – summative evaluation

Testing is an iterative process

Testing isn’t something you do once

Make somethingMake somethingMake somethingMake somethingMake somethingMake somethingMake somethingMake something=> test it => test it => test it => test it => test it => test it => test it => test it

=> refine it=> refine it=> refine it=> refine it=> refine it=> refine it=> refine it=> refine it=> test it again=> test it again=> test it again=> test it again=> test it again=> test it again=> test it again=> test it again

Before funding – project planning• ****Evaluation of other websitesEvaluation of other websitesEvaluation of other websitesEvaluation of other websites

– Who for? What for? How use it? etc– awareness raising: issues, opportunities– contributes to market research– possible elements, graphic feel etc

• ****Concept testing Concept testing Concept testing Concept testing – check idea makes sense with audience– reshape project based on user feedback

Focus group

Research

Post-funding - project development• ****Concept testingConcept testingConcept testingConcept testing

– refine project outcomes based on feedback from intended users

• Refine website structureRefine website structureRefine website structureRefine website structure– does it work for users?

• ****Evaluate initial look and feel Evaluate initial look and feel Evaluate initial look and feel Evaluate initial look and feel – graphics,navigation etc

Focus group

Focus group

One-to-one tasks

Card sorting - get various people to try out the website structure before you build it

Post-funding - project development 2• ****Full evaluation of a draft working Full evaluation of a draft working Full evaluation of a draft working Full evaluation of a draft working

version version version version – usability AND content: do activities work, how

engaging is it, what else could be offered, etc

Observation of actual use of websiteactual use of websiteactual use of websiteactual use of websiteby intended usersintended usersintended usersintended users,

using it for intended purposeintended purposeintended purposeintended purpose,

in intended contextintended contextintended contextintended context – workplace, classroom, library, home, etc

Post-funding - project development 3

• Acceptance testing of Acceptance testing of Acceptance testing of Acceptance testing of ‘‘‘‘finishedfinishedfinishedfinished’’’’ website website website website– last minute check, minor corrections only– often offered by web developers

• Summative evaluationSummative evaluationSummative evaluationSummative evaluation– report for funders, etc– learn lessons at project level for next time

Website evaluation and testingNeed to think ahead a bit:

– what are you trying to find out?

– how do you intend to test it?

– why? what will do you do as a resultyou do as a resultyou do as a resultyou do as a result?

The Why?Why? should drive this process

Happy to help - phone number on site:Happy to help - phone number on site:Happy to help - phone number on site:Happy to help - phone number on site:

Martin BazleyMartin BazleyMartin BazleyMartin Bazley0780 3580 7370780 3580 7370780 3580 7370780 3580 737

www.martinbazley.comwww.martinbazley.comwww.martinbazley.comwww.martinbazley.com

More information / advice / ideas

Who forWho for……??What for?What for?

How will they use it?How will they use it?

Learning resource: iterative planningLearning resource: iterative planningLearning resource: iterative planningLearning resource: iterative planning

content � � curriculum (find a match)

Learning activities � � Learning outcomes

(find a match)

Filtered by your specific audience needs

Who forWho for……??What for?What for?

How will they use it?How will they use it?

Who for what for ...• Who for? (audience)

Need to be clear from start• mum + 2 children looking for something to do this

weekend• teachers of yr5/6 in local area with whiteboards• men interested in gadgets

Who for what for ...• What ‘real-world’ outcomes?

What will they do as a result of using the site? • make a donation• plan a visit to a museum• buy a train ticket• think differently about learning disability

Who for what for ...• How will they use it? (user experience)

What do they actually do on the site? • browse and read articles

• working alone or in pairs? (learning resources)• lean forward or sit back?• Browsing, following, searching…

• Also Where, When and Why?

Examples of teacher feedback• Vimeo videos• http://vimeo.com/18888798 Key ideas• http://vimeo.com/18892401 Lesson starter• http://vimeo.com/18867252 Timesaver

Who for what for ...• Website appraisal

– For each example note first impressions

• WhoWhoWhoWho is it for?is it for?is it for?is it for?• WhatWhatWhatWhat does it offer them?does it offer them?does it offer them?does it offer them?• HowHowHowHow will they use it?will they use it?will they use it?will they use it?

Martin BazleyMartin BazleyMartin BazleyMartin BazleyOnline experience consultantOnline experience consultantOnline experience consultantOnline experience consultant

Evaluating online learning resources in the classroom

Key point:

for a site designed for schools, for a site designed for schools, for a site designed for schools, for a site designed for schools,

the most effective user testing observationsthe most effective user testing observationsthe most effective user testing observationsthe most effective user testing observations

will be made in a real classroom situationwill be made in a real classroom situationwill be made in a real classroom situationwill be made in a real classroom situation

National Archives Moving Here project

For teachers of 8 – 14 yr olds

History Geography and Citizenship

Features: Interactives, activity sheets, audio and video clips

Moving Here Schools:Moving Here Schools:Moving Here Schools:Moving Here Schools:For 8 – 14 yr olds studying:

History Geography and Citizenship

Features:

Interactives, activity sheets, audio and video

clips

1. preliminary testing 1. preliminary testing 1. preliminary testing 1. preliminary testing sessionssessionssessionssessions –

conventional user-testing with teachers (at TNA)

2. in-class testing 2. in-class testing 2. in-class testing 2. in-class testing –––– teachers used the Moving

Here Schools site with pupils in their own classrooms This meant sitting at the back of the classroom

observing and taking notes…

Evaluation: 2-phase approach

Site ready in parts – but not too ready:

The environment had a significant impact on how the site was used.

The class dynamic within the different groups contributed to how much the students learned.

The environment and social dynamics

The environment had a significant impact on how the site was used.

The class dynamic within the different groups contributed to how much the students learned.

in-class testing picked up elements not there in conventional user testing.

teachers in preliminary user testing did not spot some problems until actually in the classroom. For example…

interactive activities:interactive activities:interactive activities:interactive activities: looked big enough when

viewed on a screen nearby…

… but text/images too small for some children to see from the back of the class…

…so interactives needed to be viewable full-screen

Only spotted during in-class testing:

…so interactives needed to be viewable full-screen

content:content:content:content: when students tried to read

text out loud, teachers realised some text was too difficult or complex

activity sheets:activity sheets:activity sheets:activity sheets:

some sheets did not have spaces for students to put their names - caused confusion when printing 30 at same time…

Manchester Art Gallery art interactive

For teachers of 8 – 11 yr olds, and for pupils

History Art and Citizenship

Features: interactive with built in video, quiz, etc,

plus activity sheets and background info

Martin Bazley www.ICTICTICTICT4444LearningLearningLearningLearning.com.com.com.com




'This classroom user testing is all very well, but...'

How can you see everything in a class of 30 How can you see everything in a class of 30 How can you see everything in a class of 30 How can you see everything in a class of 30 children children children children –––– don't you miss things? don't you miss things? don't you miss things? don't you miss things?

You see things in a classroom that You see things in a classroom that You see things in a classroom that You see things in a classroom that don't arise in one-to-one testingdon't arise in one-to-one testingdon't arise in one-to-one testingdon't arise in one-to-one testing

They are the real issuesThey are the real issuesThey are the real issuesThey are the real issues


How can you see everything in a class of 30 How can you see everything in a class of 30 How can you see everything in a class of 30 How can you see everything in a class of 30 children children children children –––– don't you miss things? don't you miss things? don't you miss things? don't you miss things?

You see things in a classroom that You see things in a classroom that You see things in a classroom that You see things in a classroom that don't arise in one-to-one testingdon't arise in one-to-one testingdon't arise in one-to-one testingdon't arise in one-to-one testing

They are the real issuesThey are the real issuesThey are the real issuesThey are the real issues


Doesn't using a specific class with particular Doesn't using a specific class with particular Doesn't using a specific class with particular Doesn't using a specific class with particular needs skew the results?needs skew the results?needs skew the results?needs skew the results?

» For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a 'neutral' environment? 'neutral' environment? 'neutral' environment? 'neutral' environment?

» ‘‘‘‘neutral environmentneutral environmentneutral environmentneutral environment’’’’ ? ? ? ? –––– no such thing - any test will be no such thing - any test will be no such thing - any test will be no such thing - any test will be subjective, and in any case:subjective, and in any case:subjective, and in any case:subjective, and in any case:

» Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see effects of factors like those. effects of factors like those. effects of factors like those. effects of factors like those.



Doesn't using a specific class with particular Doesn't using a specific class with particular Doesn't using a specific class with particular Doesn't using a specific class with particular needs skew the results?needs skew the results?needs skew the results?needs skew the results?

» For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, For example, low ability, poor English, equipment not working, behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a behaviour issues, etc - are results as reliable as those in a 'neutral' environment? 'neutral' environment? 'neutral' environment? 'neutral' environment?

» ‘‘‘‘neutral environmentneutral environmentneutral environmentneutral environment’’’’ ? ? ? ? –––– no such thing - any test will be no such thing - any test will be no such thing - any test will be no such thing - any test will be subjective, and in any case:subjective, and in any case:subjective, and in any case:subjective, and in any case:

» Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see Testing is to make website work well in classroom, - need to see effects of factors like those. effects of factors like those. effects of factors like those. effects of factors like those.



Can't my Web developer do the testing for us?Can't my Web developer do the testing for us?Can't my Web developer do the testing for us?Can't my Web developer do the testing for us? » best not to use external developer to do user best not to use external developer to do user best not to use external developer to do user best not to use external developer to do user

testing - conflict of interest testing - conflict of interest testing - conflict of interest testing - conflict of interest » also likely to focus more on the technical also likely to focus more on the technical also likely to focus more on the technical also likely to focus more on the technical

aspects of the site than on effect on the aspects of the site than on effect on the aspects of the site than on effect on the aspects of the site than on effect on the teacher and pupils. teacher and pupils. teacher and pupils. teacher and pupils.

» observe classes yourself but use an observe classes yourself but use an observe classes yourself but use an observe classes yourself but use an independent evaluator for key decision independent evaluator for key decision independent evaluator for key decision independent evaluator for key decision pointspointspointspoints



Can't my Web developer do the testing for us?Can't my Web developer do the testing for us?Can't my Web developer do the testing for us?Can't my Web developer do the testing for us? » best not to use external developer to do user best not to use external developer to do user best not to use external developer to do user best not to use external developer to do user

testing - conflict of interest testing - conflict of interest testing - conflict of interest testing - conflict of interest » also likely to focus more on the technical also likely to focus more on the technical also likely to focus more on the technical also likely to focus more on the technical

aspects of the site than on effect on the aspects of the site than on effect on the aspects of the site than on effect on the aspects of the site than on effect on the teacher and pupils. teacher and pupils. teacher and pupils. teacher and pupils.

» visit a classroom yourself but use an visit a classroom yourself but use an visit a classroom yourself but use an visit a classroom yourself but use an independent evaluator for key decision independent evaluator for key decision independent evaluator for key decision independent evaluator for key decision pointspointspointspoints



I don't have the time or budget to do this!I don't have the time or budget to do this!I don't have the time or budget to do this!I don't have the time or budget to do this! » need cost no more than conventional user testing. one person need cost no more than conventional user testing. one person need cost no more than conventional user testing. one person need cost no more than conventional user testing. one person

could attend a one-hour class session in a school, giving the could attend a one-hour class session in a school, giving the could attend a one-hour class session in a school, giving the could attend a one-hour class session in a school, giving the teacher the same small token payment teacher the same small token payment teacher the same small token payment teacher the same small token payment

» This programme had evaluation built into project: 6.7% of This programme had evaluation built into project: 6.7% of This programme had evaluation built into project: 6.7% of This programme had evaluation built into project: 6.7% of total Schools site budget. total Schools site budget. total Schools site budget. total Schools site budget.

» Allow 5 -10% of total project budget for user testing Allow 5 -10% of total project budget for user testing Allow 5 -10% of total project budget for user testing Allow 5 -10% of total project budget for user testing

=> videos


Video clips • Moving Here key ideas not lesson plans etc

http://www.vimeo.com/18888798 • http://www.vimeo.com/18892401 Lesson

starter• Time saver http://www.vimeo.com/18867252

S

User test early

Testing one user early on in the project…

…is better than testing 50 near the end

Two usability testing techniques

““““Get itGet itGet itGet it”””” testing testing testing testing- do they understand the purpose, how it

works, etc

Key task testingKey task testingKey task testingKey task testing- ask the user to do something, watch how

well they do

Ideally, do a bit of each, in that order

User testing – who should do it?• The worst person to conduct (or interpret)

user testing of your own site is…–– you!you!you!you!you!you!you!you!

• Beware of hearing what you want to hear…

• Useful to have an external viewpoint• First 5mins in a genuine setting tells you

80% of what’s wrong with the site

Strengths and weaknesses of different data Strengths and weaknesses of different data Strengths and weaknesses of different data Strengths and weaknesses of different data gathering techniquesgathering techniquesgathering techniquesgathering techniques

Data gathering techniquesUser testingUser testingUser testingUser testing

- early in development and again near endOnline questionnairesOnline questionnairesOnline questionnairesOnline questionnaires

– emailed to people or linked from websiteFocus groupsFocus groupsFocus groupsFocus groups

- best near beginning of project, or at redevelopment stage

Visitor surveysVisitor surveysVisitor surveysVisitor surveys - link online and real visits

Web statsWeb statsWeb statsWeb stats- useful for long term trends /events etc

Need to distinguish between:

Diagnostics Diagnostics Diagnostics Diagnostics – making a project or service better

Reporting Reporting Reporting Reporting – to funders, or for advocacy

Online questionnairesOnline questionnairesOnline questionnairesOnline questionnaires(+) once set up they gather numerical and

qualitative data with no further effort – given time can build up large datasets

(+) the datasets can be easily exported and manipulated, can be sampled at various times, and structured queries can yield useful results

(–) respondents are self-selected and this will skew results – best to compare with similar data from other sources, like visitor surveys

(–) the number and nature of responses may depend on how the online questionnaire is displayed and promoted on the website

Focus groupsFocus groupsFocus groupsFocus groups

(+) can explore specific issues in more depth, yielding rich feedback



(–) yield qualitative data only - small numbers mean numerical comparisons are unreliable

Visitor surveys Visitor surveys Visitor surveys Visitor surveys



(–) responses can be affected by various factors including interviewer, weather on the day, day of the week, etc, reducing validity of numerical comparisons between museums

Web statsWeb statsWeb statsWeb stats(+) Easy to gather data – can decide what

to do with it later(+) Person-independent data generated - it

is the interpretation, rather than the data themselves, which is subjective. This means others can review the same data and verify or amend initial conclusions reached

Web statsWeb statsWeb statsWeb stats(–) Different systems generate different

data for the same web activity – for example no of unique visits measured via Google Analytics is generally lower than that derived via server log files

(–) Metrics are complicated and require specialist knowledge to appreciate them fully

Web statsWeb statsWeb statsWeb stats(–) As the amount of off-website web

activity increases (e.g. Web 2.0 style interactions) the validity of website stats decreases, especially for reporting purposes, but also for diagnostics

(–) Agreeing a common format for presentation of data and analysis requires collaborative working to be meaningful




SCA guidancehttp://sca.jiscinvolve.org/wp/audience-publications/

Good overview Step by step approach

Culture 24 Let’s Get Realhttp://weareculture24.org.uk/projects/action-research/

Crit room‘simulated user testing’

Crit room protocolCrit room protocolCrit room protocolCrit room protocolSimulating user testing – usually one-to-one

in quiet roomNo one (especially site stakeholders) other

than tester say anything for first part of session

In this simulation we will focus on- Look and feel of site- Usability- Content

Happy to help - phone number on site:Happy to help - phone number on site:Happy to help - phone number on site:Happy to help - phone number on site:


