77
Usability and evaluation – focus groups and other techniques Martin Bazley University of Brighton 25 Nov 10

10 11 25 univ of brighton usability and evaluation module shelley boden

Embed Size (px)

DESCRIPTION

Slides used during talk on focus groups and other audience techniques, with Shelley Boden at Univ of Brighton.

Citation preview

Page 1: 10 11 25 univ of brighton usability and evaluation module shelley boden

Usability and evaluation – focus groups and other

techniques

Martin Bazley

University of Brighton25 Nov 10

Page 2: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin BazleyPreviously• Teaching (7 yrs)• Science Museum, London,

Internet Projects (7yrs)• E-Learning Officer, MLA South East

(3yrs)

Currently• Vice Chair, DLNet (was E-

Learning Group for Museums, Lib, Archives)

• Consultancy, websites, training, user testing, evaluation …Martin Bazley & Associateswww.martinbazley.com

•Slides and notes available afterwards

Page 4: 10 11 25 univ of brighton usability and evaluation module shelley boden

www.martinbazley.com

Page 5: 10 11 25 univ of brighton usability and evaluation module shelley boden

Users don’t always ‘get’ what we are offering:

a real–world analogy

Page 6: 10 11 25 univ of brighton usability and evaluation module shelley boden

Even a slight difference in viewpoints…

…can cause real problems for users

Page 7: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 8: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 9: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 10: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 11: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 12: 10 11 25 univ of brighton usability and evaluation module shelley boden
Page 13: 10 11 25 univ of brighton usability and evaluation module shelley boden

In a conflict between visual affordance…

…and written instructions

visual affordance almost always wins

Page 14: 10 11 25 univ of brighton usability and evaluation module shelley boden

Another example

Page 15: 10 11 25 univ of brighton usability and evaluation module shelley boden

So what is the point of all this?

Page 16: 10 11 25 univ of brighton usability and evaluation module shelley boden

People interact with digital material differently…

Page 17: 10 11 25 univ of brighton usability and evaluation module shelley boden

… from the way they use books, object labels, magazines, newspapers, etc

Page 18: 10 11 25 univ of brighton usability and evaluation module shelley boden

For most people the web is a

predominantly visual

medium

Page 19: 10 11 25 univ of brighton usability and evaluation module shelley boden

Website usersWebsite users

Page 20: 10 11 25 univ of brighton usability and evaluation module shelley boden

Website users• Who uses your website?

• Why would they want to use it?

• How would they find it?

• What do they get out of it?

• What do they dislike about it?

Page 21: 10 11 25 univ of brighton usability and evaluation module shelley boden

How do you get it right for everyone?

• Answer:• You can’t get it right for everyone.

• You have to make choices, and stick to them:

• Who is it for? • What..• How…

Page 22: 10 11 25 univ of brighton usability and evaluation module shelley boden

Who for…?Who for…?What for?What for?

How will they use it?How will they use it?

Page 23: 10 11 25 univ of brighton usability and evaluation module shelley boden

Who for what for ...• Who for? (audience)

Need to be clear from start• mum + 2 children looking for something to do

this weekend• teachers of yr5/6 in local area with whiteboards• men interested in gadgets

Page 24: 10 11 25 univ of brighton usability and evaluation module shelley boden

Who for what for ...• What ‘real-world’ outcomes?

What will they do as a result of using the site?

• make a donation• plan a visit to a museum• buy a train ticket• think differently about learning disability

Page 25: 10 11 25 univ of brighton usability and evaluation module shelley boden

Who for what for ...• How will they use it? (user experience)

What do they actually do on the site? • browse and read articles

• working alone or in pairs? (learning resources)• lean forward or sit back?• Browsing, following, searching…

• Also Where, When and Why?

Page 26: 10 11 25 univ of brighton usability and evaluation module shelley boden

Who for what for ...• Website appraisal

– For each example note first impressions

•Who is it for?•What does it offer them?•How will they use it?

Page 27: 10 11 25 univ of brighton usability and evaluation module shelley boden

Websites for different Websites for different audiencesaudiences

Page 28: 10 11 25 univ of brighton usability and evaluation module shelley boden

Websites for different audiences

The following tips are based on • numerous evaluation sessions• numerous user testing sessions• talking to other people who use

websites• talking to other people who make

websites

Page 29: 10 11 25 univ of brighton usability and evaluation module shelley boden

‘General users’There is no such thing as a general

user

Are you a general user?

Page 30: 10 11 25 univ of brighton usability and evaluation module shelley boden

Adults/families with general interest

• What does the site tell me at a glance?

• genuine enthusiasts = interested whatever the website looks like, and will spend some time looking around it or phone up for more information if required.

Page 31: 10 11 25 univ of brighton usability and evaluation module shelley boden

Adults/families with general interest

• But most will not bother unless something engages them within a few seconds

• The questions people might like answered within a few seconds of arriving on a museum site probably include:

Page 32: 10 11 25 univ of brighton usability and evaluation module shelley boden

Adults/families with general interest

• Where is it? Further down the home page it says ‘alongside Middle Wallop airfield’ but I have no idea where that is.

• a schematic map on every page, or at least on the home page and visit info, would really help in attracting visitors who don’t know the area

Page 33: 10 11 25 univ of brighton usability and evaluation module shelley boden

Adults/families with general interest

• What’s the rough cost and roughly how long might I/we want to spend there?This would give me an idea of whether to view it as a place to pop into on the way somewhere or combine it with another attraction; or whether it requires more serious investment of time or money

Page 34: 10 11 25 univ of brighton usability and evaluation module shelley boden

Adults/families with general interest

• What kind of experience will I get?I know there will be ‘displays’ – it is a museum!

• but will there be people around to help bring the place alive for me, my spouse, my children or friends?

• – or are there events, or things to do like dressing up in a pilot’s uniform, or games to play, etc?

Page 35: 10 11 25 univ of brighton usability and evaluation module shelley boden

Websites for schools‘Serve the National Curriculum’ or ‘extend or

enhance’?‘enhance’ sounds good but most teachers want:

1.curriculum specific – by all means cross-curricular but with one scheme of work or topic as ‘headline’ (think ‘product byline’)

2.ready-to-use – teachers may want to adapt to their own situation (esp second time round), but most will not have time – offer at least one ready to use version

3.minimal preparation and with time commitment (preparation time and class time) clearly specified

4.flexible/adaptable/extensible where possible

Page 36: 10 11 25 univ of brighton usability and evaluation module shelley boden

Foundation and KS1 (3-7yrs)

Production of materials for this age range is particularly tricky:

• aim at teachers not children, so…• good bank of images, videos or other

mainly visual assets • think of interactive whiteboards

Page 37: 10 11 25 univ of brighton usability and evaluation module shelley boden

Key Stage 2 (7 to 11 years old)

• keep no of words on each page to a minimum, say 50 in total

• illustrate key ideas visually as well as verbally and use audio if possible

• do not assume that the teacher can be over their shoulder at all points – so keep the instructions and processes simple

• try to use language, images, ideas, and settings that will appeal to the target audience

Page 38: 10 11 25 univ of brighton usability and evaluation module shelley boden

Key Stage 3 (11 to 14 years old)

• For KS2 a cross-curricular approach is OK (for example they may use the same site for Geography and Science) but:

• At KS3 cater for a single subject (and scheme of work)

• (Can offer suggestions for cross-curricular working, but remember generally each teacher teachers only one subject each.)

Page 39: 10 11 25 univ of brighton usability and evaluation module shelley boden

‘Lifelong learners’

• for (non-specialist) interest level think of 12 yr olds

• Identify a particular audience with specific interests/motivations for using your site

• then focus on constraints to allow successful design to proceed. (In a formal learning setting constraints often implicit in the course, physical set up etc.)

Page 40: 10 11 25 univ of brighton usability and evaluation module shelley boden

Specialist researchers

• Fact-oriented, less graphics and design, more text and specifically relevant images, with good search facility

• Examples of specialist researchers: – HE students and staff– experts or enthusiasts in this subject

area

Page 41: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Website evaluation and testing

Need to think ahead a bit:– what are you trying to find out?

– how do you intend to test it?

– why? what will do you do as a result?

The Why?Why? should drive this process

Page 42: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Test early

Testing one user early on in the project…

…is better than testing 50 near the end

Page 43: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

When to evaluate or test and why

• Before funding approval – project planning

• Post-funding - project development

• Post-project – summative evaluation

Page 44: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Testing is an iterative process

Testing isn’t something you do once

Make somethingMake something=> test it => test it

=> refine it=> refine it=> test it again=> test it again

Page 45: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Before funding – project planning

• *Evaluation of other websites– Who for? What for? How use it? etc– awareness raising: issues, opportunities– contributes to market research– possible elements, graphic feel etc

• *Concept testing – check idea makes sense with audience– reshape project based on user feedback

Focus group

Research

Page 46: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 47: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Post-funding - project development

• *Concept testing– refine project outcomes based on

feedback from intended users

• Refine website structure– does it work for users?

• *Evaluate initial look and feel – graphics,navigation etc

Focus group

Focus group

One-to-one tasks

Page 48: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 49: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 50: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 51: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 52: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Post-funding - project development 2

• *Full evaluation of a draft working version – usability AND content: do activities

work, how engaging is it, what else could be offered, etc

Observation of actual use of website

by intended users,

using it for intended purpose,

in intended context – classroom, workplace, library, home, etc

Page 53: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 54: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 55: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 56: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 57: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 58: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 59: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 60: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 61: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 62: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Post-funding - project development 3

• Acceptance testing of ‘finished’ website– last minute check, minor corrections only– often offered by web developers

• Summative evaluation– report for funders, etc– learn lessons at project level for next time

Page 63: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Two usability testing techniques

“Get it” testing- do they understand the purpose, how

it works, etc

Key task testing- ask the user to do something, watch

how well they do

Ideally, do a bit of each, in that order

Page 64: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

Page 65: 10 11 25 univ of brighton usability and evaluation module shelley boden

Martin Bazley

www.ICT4Learning.com

User testing – who should do it?

• The worst person to conduct (or interpret) user testing of your own site is…– you!you!

• Beware of hearing what you want to hear…

• Useful to have an external viewpoint• First 5mins in a genuine setting tells

you 80% of what’s wrong with the site• etc

Page 66: 10 11 25 univ of brighton usability and evaluation module shelley boden

Creating resources for Creating resources for use on whiteboardsuse on whiteboards

Page 67: 10 11 25 univ of brighton usability and evaluation module shelley boden

Using whiteboards

• http://www.teachers.tv/ict/whiteboardtips

• http://smarttech.com/trainingcenter/tutorials.asp#

• http://www.prometheanplanet.com

Page 68: 10 11 25 univ of brighton usability and evaluation module shelley boden

Roles of IWB

… at different points in the lesson / learning cycle

– Starter

– Main

– Plenary

Page 69: 10 11 25 univ of brighton usability and evaluation module shelley boden

Interactive” means

• “lots of things moving on screen, clickable, automatic response, quizzes etc

• interaction between students, teacher and screen – activities, conversation, cognitive engagement, etc

first meaning used mainly by companies trying to market whiteboards, software etcas ‘interactive’

second used mainly by educators

Page 70: 10 11 25 univ of brighton usability and evaluation module shelley boden

Resources for use on whiteboards - examples

• Ford Madox Brown MAG• Tate Tools• Museum Network Artworks• Museum Network Myths• National Portrait Gallery Mary

Seacole• National Gallery• Museum of London Fire of London

Page 71: 10 11 25 univ of brighton usability and evaluation module shelley boden

Resources for use on whiteboards - examples

• Britons at War• Wartime in Bedford • http://www.movinghere.org.uk/

schools/default.htm • www.mylearning.org/overview.asp?

journeyid=409 • www.mylearning.org/overview.asp?

journeyid=441

Page 72: 10 11 25 univ of brighton usability and evaluation module shelley boden

Some examples

– http://www.nhm.ac.uk/nature-online/british-natural-history/index.html

– http://www.manchestergalleries.org/the-collections/highlights-of-the-collection/narrativeobject.php?irn=876

– www.seayourhistory.org.uk/content/view/39/77/

– http://www.portsmouth.gov.uk/business/2781.html

Page 73: 10 11 25 univ of brighton usability and evaluation module shelley boden

Whiteboard resource exercise

• We are using Powerpoint (or Word if you prefer) just to summarise your ideas on the board.

• Don’t spend too long formatting / designing – just focus on – what items, text and links would be on screen,

and– What teacher / pupils would do with them

Page 74: 10 11 25 univ of brighton usability and evaluation module shelley boden

‘templates’

• The following slide (Britons at War) is a sample web page – think about how people would get to your whiteboard page(s)

• The next slide is an outline template for a whiteboard page – edit or ignore this completely.

• Make sure you have at least something to show on screen

Page 75: 10 11 25 univ of brighton usability and evaluation module shelley boden

75

SearchAdv search

Britons at War banner

Home

Topics

Resources

Schools area

Ways to use this site

About this site

Search

BaW logo

Home | About | Links | Contact | Sitemapan online resource for schools

NMPFT logo

link to home

YFAlogo

link to home

Topic: Bombing Thumbnails of photos + films, etc for this topicLink to Photo pages

BombingRationingLocal lifeHome GuardWomen and childrenEvacuation

Guy Fawkes Day 1944

Prisoners of War

Whit Sunday in Hyde Park

VE Day

Brief introduction to Topic: Bombing

AsfdAsdfsadf

Page 76: 10 11 25 univ of brighton usability and evaluation module shelley boden

About this image (caption, copyright, etc) Also key question?

Title / heading if needed – otherwise delete this box.

Page 77: 10 11 25 univ of brighton usability and evaluation module shelley boden

More information / advice / ideas

Martin Bazley0780 3580 737

www.martinbazley.com