Practical Approaches to Usability Testing Julie Nowicki President Optavia Corporation WEB2000

Practical Approaches toUsability Testing

Julie NowickiPresident

Optavia Corporation

WEB2000

Practical Approaches to Usability Testing WEB2000

2

Goals

Put usability testing in perspective• In the bigger picture of usability• Compared to other techniques

Understand the breadth of usability testing methods available

Be able to focus and choose the right method• Depending on the web site’s development

stage• Balancing resource costs against benefits of

various methods

Practical Approaches to Usability Testing WEB2000

3

Topics

The role of usability testing Definition and comparison to other

techniques Designing the usability test

• Planning• Choosing specific methods

Tuxedos or jeans? Summary: Getting what you want

• And what your customers want!

Practical Approaches to Usability Testing WEB2000

4

E-commerce and Usability

Three E-commerce segments• B2C (Business-to-Consumer)• B2B (Business-to-Business)• B2E (Business-to-Employee)

All focus on the customer Ability of customer to

effectively complete a task, withhigh satisfaction levelsis key to the bottom line

Usability

Practical Approaches to Usability Testing WEB2000

5

Qualities of a usable Web site

Learnability• Can customers easily grasp how to use the site the

very first time? Memorability

• Do customers remember how the next time? Ease-of-use

• Can customers navigate easily?• Is the design consistent and predictable?

Efficiency• Can customers complete tasks and transactions

effectively? Appeal

• More than visual: do customers feel positive about the experience?

Practical Approaches to Usability Testing WEB2000

6

How can usability testing help?

Learnability Measure time-on-taskMeasure success rate

Ease-of-use Measure errors/false startsTrack navigation patterns (and why)Record comprehension of words and graphics

Efficiency Measure time-on-task

Appeal Record facial expressions and commentsAdminister post-test survey

Practical Approaches to Usability Testing WEB2000

7

Characteristics of a usability test

Uses real customers (users) Customers perform actual tasks with

the site or a prototype Data is recorded

• May be qualitative or quantitative• Includes customer comments

Provides objective information Facilitated by a skilled specialist Observed by a skilled specialist

Practical Approaches to Usability Testing WEB2000

8

Limitations of usability testing

Based on a few customers Guinea pig effect

• Test environment cannot duplicate the real world

• Users know they are being watched Not thorough by nature

• Testing a few tasks, not every aspect of site usage

• Some issues are not readily apparent• Use of color and fonts• Accessibility

Practical Approaches to Usability Testing WEB2000

9

Usability testing compared…

Web site log statistics Automated use testing Focus groups QA and system testing User acceptance testingProven usability techniques: Heuristic evaluation and guidelines

reviews Walkthroughs

Practical Approaches to Usability Testing WEB2000

10

Tip

Usability encompasses customer-centered design and evaluation.

However…Usability testing is an evaluation technique.

Usability testing is not a design technique.

Practical Approaches to Usability Testing WEB2000

11

Customer-centered design process

AnalysisAnalysis

EvaluationEvaluation

Understand objectivesUnderstand customersUnderstand their tasks

Develop design conceptSpecify the details

Test with real customers (usability testing)Review with design specialists

DesignDesign

Iterative!

Actual TimelineMonth 1 2 3 4 5-7

VisionField StudyCustomer analysisTask analysis

WorkflowReview Analysis Doc

Conceptual ModelPrototype SketchesStart Interact Spec Doc

1st Online Prototype

Refine Interact Spec Doc

2nd Online Prototype(real code)

Finish Interact Spec Doc

DevelopUserAssistance

WalkthroughUsability Testing with Sketches

Usability Testing with Online Prototype

Guidelines ReviewUsability Testing

Ongoing mini-reviews

Practical Approaches to Usability Testing WEB2000

13

Anatomy of a usability test

Plan and Design

Report and Recommend

Conduct

Plan: Answer the big questionsSelect the test methodsGet participants

Practical Approaches to Usability Testing WEB2000

14

Overview of usability testing methods

To prepare for the test, you must1. Plan

Answer some big-picture questions Understand objectives, customers, and tasks

2. Choose the test methods Considering a number of parameters, Pick the best usability testing methods for

your situation

Practical Approaches to Usability Testing WEB2000

15

Planning:Big decisions to make first

What are you trying to accomplish? When will you do the test?

• Based on product lifecycle

What are the customer profiles? What tasks will customers do? How will you present the results? How will you make recommendations?

• Explore design alternatives

Practical Approaches to Usability Testing WEB2000

16

What are you trying to accomplish?

Find out how your site compares to the competition

Find out if your new design works for• Existing customers• New customers

Get statistically sound data

Practical Approaches to Usability Testing WEB2000

17

Tips for setting objectives

Prioritize the four usability components• Learnability• Ease-of-use• Efficiency• Appeal

Be specific• NO “Registration is easy to use.”• YES “New customers should be able to

complete the registration process in 5 minutes without help.”

Practical Approaches to Usability Testing WEB2000

18

Tip

Usability testing done early in the design cyclereduces risk.

Practical Approaches to Usability Testing WEB2000

19

The Cost to Make a Change

1

10

100

Adapted from Rapid Application Development by James Martin

Usability

U-test a must

Usability

Practical Approaches to Usability Testing WEB2000

20

Don’t get fooled by online development tools!

Sketches are highly advantageous• They are easy to throw away• People don’t assume the system is done• People are more likely to suggest changes• Usability testing can be done with sketches

Tip

Practical Approaches to Usability Testing WEB2000

21

Tip

Usability testing done late in the production processhelps you plan for the next iteration.

Practical Approaches to Usability Testing WEB2000

22

Who are the customers?

Marketing information such as• Age• Income• Education• Career

Experience with the Internet Experience with online shopping Anticipated frequency of use (your site) Environment they work in

• Home, office…

Practical Approaches to Usability Testing WEB2000

23

What are the tasks?

Creating task scenarios is a must! Don’t use language that leads the user

• NO “Go to My Profile and change your address to…”

• YES “You have moved. Change your address to…”

Practical Approaches to Usability Testing WEB2000

24

Presenting the results

Consider allowing observers during the test• Make sure they do not interfere

Highlight tape• Compelling and efficient

Results of data collection Summary of issues, prioritized by

• Usability criticality• Implementation effort

Recommendations

Practical Approaches to Usability Testing WEB2000

25

The next step

Once you have• considered what you want to accomplish,

and when• identified your customer profiles• determined the tasks you want customers

to perform• decided how to communicate the results

you are ready to design the test methods

Practical Approaches to Usability Testing WEB2000

26

Test methods:Parameters that you must select

Equipment and facilities What to measure

• Quantitative and qualitative

How to collect measurements Use of participants (protocol) How many participants Level of facilitation Level of participant verbalization

Practical Approaches to Usability Testing WEB2000

27

Equipment and facilities

Conference roomPaper notes

One cameraPaper notes

Portable labLogging software

Lab facilityLogging software

Practical Approaches to Usability Testing WEB2000

28

Example Usability Labs

From www.usabilitysystems.com

From www.microsoft.com/usability

Practical Approaches to Usability Testing WEB2000

29

Advantages of video tape

Records very valuable, compelling qualitative data• Page navigation• Expressions• Comments in context

Provides a real snippet of an actual customer experience

Watch out• Someone needs to understand the equipment• Time to setup, review, create highlight tape

Practical Approaches to Usability Testing WEB2000

30

What to measure: Quantitative

Time on task Number of errors Successful completion of task (yes/no)

How you collect measurements is determined by facility and equipment

Practical Approaches to Usability Testing WEB2000

31

What to watch for: Qualitative

Navigation patterns• Getting lost?

Predictability• Understand graphics?• Understand words?• Looking for it to work like something else?

Flexibility• Looking for a quicker way?• Looking for help?

Practical Approaches to Usability Testing WEB2000

32

Protocols (Use of participants)

Individual Paired Comparison

• Good for competitive analysis• Between-subjects: each customer uses one

site• Within-subjects: each customer uses both

sites

Practical Approaches to Usability Testing WEB2000

33

How many participants?

5-8 participants will find 80% of usability problems

4 is good minimum number

Qualitative dataEarly prototype

Qualitative and quantitativeLater in development

Paired or comparison protocols

3 4 5 6 7 8 9 10 11 12

Statistically sound results

Practical Approaches to Usability Testing WEB2000

34

Level of facilitation

Facilitator in another room

Facilitator in same roomDiscussion after each task

Facilitator and participant interact during task

Practical Approaches to Usability Testing WEB2000

35

Level of participant verbalization

Think-aloud Heads-down work

Practical Approaches to Usability Testing WEB2000

36

Interaction

XQualitative dataDiscussion-orientedInformal environmentThink-aloud

Quantitative dataHeads-down taskFormalLab

Mostly qualitativeUser works aloneFormalLab

High

LowLow High Participant verbalization

Faci

litati

on

Putting it all together

LoFi Prototype(paper)

HiFi Prototype(online)

Pre-production Production site

Qualitative data

Quantitative Data

Conference rm Conf rm or lab Lab Lab

Hi facilitation Medium Medium Lo facilitation

Hi participant verbalization

Lo participant verbalization

3-4 users 4-8 users 4-8 users 6-12 users

Participants <1 hour

<1-2 hours 1-2 hours Participants 2+ hours

Individuals Individuals Indiv’s or pairs Indiv’s or pairs

Days Weeks

Informal Formal

Practical Approaches to Usability Testing WEB2000

38

Now that we’ve said that…

Can you do informal testing on a production site?• Sure!

Can you do quick, but formal testing in a lab on a prototype?• Sure!

Practical Approaches to Usability Testing WEB2000

39

What if…

You’re in the fast-paced e-commerce environment

You want customer data but you need it• Now (yesterday)• Whenever you make significant changes

You want guidance in making design decisions or changes

Variation

LoFi Prototype(paper)

HiFi Prototype(online)

Pre-production Production site

Qualitative data

Quantitative Data

Conference rm Conf rm or lab Lab Lab

Hi facilitation Medium Medium Lo facilitation

Hi participant verbalization

Lo participant verbalization

3-4 users 4-8 users 4-8 users 6-12 users

Participants <1 hour

<1-2 hours 1-2 hours Participants 2+ hours

Individuals Individuals Indiv’s or pairs Indiv’s or pairs

Days Weeks

Informal Formal

Practical Approaches to Usability Testing WEB2000

41

Resource costs will be determined by your choices

Cost of facilities and equipment Time for each participant Your time resources

• Planning test• Conducting test• Reporting and decision-making

Staff resources and skill levels Generally all go up as you move from

informal to formal

Practical Approaches to Usability Testing WEB2000

42

Results

“When [we] introduced an electronic expense payment system, users declared the application incredibly efficient – as early as the testing phase. That never would have happened if we hadn’t had the usability group to help us meet our business objectives. No company should ever move forward on a project in today’s business environment without some usability in their back pocket.”

From Supervisor of Integrated Business Practices

Usability is more than just usability testing…

It is a process

Julie NowickiPresidentOptavia CorporationJNowicki@optavia.com608 260 9000www.optavia.com