Usability with Project 16/04/10

© Simeon Keates 2008

Usability with Project16/04/10Susanne Frennert

Page 1


What is usability?

The user having an acceptable experience when they use the product !

What are the characteristics of an acceptable experience?• Successful (effectiveness) • Acceptable amount of time (efficiency)• Pleasure rather than stress (satisfaction)

ISO 9241-11 definition: usability as a goal • The extent to which a product can be used by specified users to achieve

specified goals with effectiveness, efficiency and satisfaction in a specified context of use

High usability means a system / product / website is: easy to learn and remember; efficient, visually pleasing and fun to use; and quick to recover from errors

Page 2


Cost-justifying usability

Page 3


The need to cost-justify

Not all companies (and managers) appreciate the benefits of usability They may cite other factors as more important Examples:• Tight deadlines• Functionality already developed• Limited money/resources

Page 4


Cost-justifying usability

However, these are often “false economies” Examples:• Deadlines:• Releasing the “wrong” product on time is as bad (or worse) as releasing the

“right” product late• Existing functionality:• Existing functionality should have nothing to fear from usability, if it is the

“right” functionality• Limited resources:• Putting the “right” resource in at the “right” time can make the overall project

more efficient

Page 5


Organisation Types

Page 6

Engineer-centric Design-centric Sales & marketing centric

•Might never have had UX team•Products created from interesting technology•Territorial about UI

•View building applications and websites as creative endeavour•Primary focus may be creating designs that other designers will like and respect

Represented by powerful people who “know” their costumer…but don’t know “all” the costumer

Challenges•Tendency to deem a product usable if it is possible to do the task•They rely heavily on their own experience in UI design•They feel they are doing the right thing for the customer, even if they don’t have evidence

Challenges•Define the user experience in terms of aesthetic rather than ease of use•Focus on visuals rather than workflows•Rely heavily on their own instinct about users•Focus only on data that confirms their viewpoint

Challenges•Rely heavily on customers’ self-report and costumer suggestions to assess usability•Disproportionately weigh their “biggest” or “loudest” customers•Often overconfident in their ability to “know” what the costumer need


Good usability are good business

Reduced development and software maintenance costs Increased product sales, market share, and revenue Improve customer satisfaction and loyalty Increased site visits and conversion Decrease customer support costs Improve employee productivity Improved brand perception and media coverage

Page 7


Reduced development and Software Maintenance Costs

Integration of usability testing can contribute to decreased development costs by• basing feature decisions on solid user testing and evidence decreases the

possibility of serious problems later in the process• many costs are associated with maintenance deal with unmet, or

unanticipated, user requirements

“An average of 48% of application code is devoted to the user interface

and 50% of the development time required for the entire application is

devoted to the user interface portion. Given this estimate, incorporating

usability into the design of the UI that customers and end users are interacting with is critical.” (Weinshenck 2005)

Page 8


Increased Product Sale, Market Share, and Revenue

On e-commerce or software as a service site, many potential product sale are lost due to poor site usability.

A 2001 study found that 5% of online shoppers typically fail to find the product or service they are looking for, with as many as 65% of online shopping visits resulting in failure (Souza 2001; Mayhew & Tremaine 2005)

Usability can be also used to increase advertising revenue; a revamp of Yahoo! Overture’s Search marketing tool found a 9% increase in click-through rate after only two weeks and “phenomenal” customer response (Cooper 2008)

Page 9


Increased Site Visits and Conversions

Usability can also increase the frequency with which users visit the site, contributing directly to the overall market share and customer base.

eBay redesign their “Sell your Item” form, which had a high rate of drop-off for first time users, based on their user experience consulting and usability testing. A predicted 1% improvement in the drop off rate contributed to a revenue increase of $1.2 million a year (Braun 2002; Wilson & Rosenbaum 2005)

Page 10


Improve Customer satisfaction and Loyalty

A product that is easy to use and fun to use creates satisfaction and loyalty in its users

One early study of e-commerce trust found that good navigation and presentation were essential in creating trust, and one of the authors found a link between quality in design and trust in the channel and the company’s brand (Karat & Lund 2005)

Page 11


Decreases customer support costs

The average call to a software development center is estimated to cost between $12 and $28 per call, and can cost up to $250

(Karat & Lund 2005; Mauro 2005)

“A leading manufacturer of printers released a product that had a serious usability problem in the installment and operation of the printer driver software. This problem was so difficult to solve that more than 50% of the first 100,000 users called the customer service line. The cost was nearly $0.5 million per month. The total cost of replacing the printer driver was $900,000. It was clear upon analysis that the problem could have been identified and corrected at a fraction of the cost if the product had been subject to even the simplest usability testing” (Mauro 2005)

Page 12


Improve Employee Productivity

Supporting improved, enjoyable-to-use products can increase employee satisfaction and improve the quality of customer service, as well as reducing re-work and “downtime”. These factors contribute to overall improved job satisfaction, which can lead, long term, to improve employee retention (Robb & Pfefer 2003)

“After Oracle changed the navigation structure on its database manager, database administrators were able to perform their duties 20% faster” (Black 2003)

Page 13


Brand perception and media Coverage

Products with great user experience are recognized for it. Users talk to each other about products they like, increasing sales due to world-of-mouth.

Positive interactions can affect the brand's “halo”.

“Apple’s iPhone, while accounting for only a small blip in the worldwide mobile phone sale, had the most media coverage of any product launched in history, and will likely have long-term effects on the mobile business” (Maurio 2008)

Page 14


Rosson and Carroll “Usability Engineering”

“Cost-benefit analysis of usability activities contributes to more systematic usability engineering …”

“… BUT benefits are difficult to quantify, so estimates will often be overly conservative.”

Issues: Benefits (e.g. customer satisfaction) are harder to quantify and predict

accurately Costs are very concrete and easy to identify Thus, can be difficult to justify usability…

Page 15


Example eBay(Kelly Braun, usability engineer at eBay design labs)

Sell Your Item Form (SYI)

Page 16


Sell your Item Form at eBay(Kelly Braun, usability engineer at eBay design labs)

• Look at page statistics and see where people leave process

• Estimate usability impact

• Use data to gain insight for re-design

They assumed the complexity of the page was contributing to the drop out rate.

How much revenue could be generated if they reduced the drop off rate by 1%?

Page 17


SYI revenue increase calculations(Kelly Braun, usability engineer at eBay design labs)

Reduce drop out by 1% Increase # of listings by 1%

1.1 million listings per day 11,000 more listings per day.

Minimum listing fee = $0.30

11,000 listings x $0.30 = $3,300 per day• $1,204,500 in listing fees per year

Page 18


SYI Usability Costs(Kelly Braun, usability engineer at eBay design labs)

• One usability engineer for 3 months (team input and testing)

• Testing assistant for 2 weeks

• Participant incentives

• Additional UI design (1 week) to address usability issues and redesign.

Page 19


Cost-benefits analysis of usability

A cost benefit analysis determines how well, or how poorly, a planned action will turn out.

1.A cost benefits analysis finds, quantifies, and adds all the positive factors. These are the benefits.

2.Then it identifies, quantities, and subtract all the negative factors, the costs.

3.The difference between the two indicates whether the planned action is advisable.

Page 20


Example Cost-benefit Analysis for Usability testingEstimated Costs for Usability Testing Value (GBP)

Evaluator’s Time: •Test Material Preparation (16 hours x GBP20) = GBP320•Subject Recruitment Time (40 hours x GBP20) = GBP800

1,120

Subject Payment: •14 subjects x GBP50 = GBP700 700

Laboratory/Equipment: •7 full days x GBP1,000 per day 7,000

Post-Evaluation: •Transcribing user comments and observation data (14 subjects x average 2 hours per subject = 28 hours x GBP20) = GBP560

•Data analysis and report write-up (40 hours x GBP20) = GBP800

1,360

Total Costs 10,230

Estimated Benefit for Usability Testing

Increased Sales: •36,000 potential buyers•25% additional Conversion Rate after optimizing usability = 9,000 •An average profit margin = GBP5 •Estimated increased sales: 9,000 x GBP5

45,000

Estimated Benefits 45,000

Page 21

In short, comparing these benefits and cost for the first year alone, an increased sales profit of GBP 34,770 is estimated.


Example Cost-benefit Analysis for Heuristic Evaluation

Costs for heuristic Evaluation Value (GBP)

Pre-evaluation:Test Material Preparation (24 hours x GBP20) = GBP480 Expert Recruitment Time (5 experts x 1 hour x GBP20) = GBP100

580

5 experts payment (5 x GBP500) 1,500

Post-Evaluation: •Writing up lists of usability problems (20 hours x GBP20) = GBP400 •Severity Ratings of Usability Problems (24 hours x GBP20) = GBP480 •Report write-up (40 hours x GBP20) = GBP800

1,680

Total Costs 3,760

Estimated Benefit for Heuristic Evaluation

•Increased Sales: 36,000 potential buyers •25% additional Conversion Rate after optimizing usability = 9,000 •An average profit margin = GBP5 •Estimated increased sales: 9,000 x GBP5

45,000

Estimated Benefits 45,000

Page 22

In summary, comparing these benefits and costs for the first year alone, a benefit of GBP41,240 is expected.


Tailor your technique to the “problem”

Page 23

What do the customer want to find out

Time to market

Who needs it and when

Hidden agendas


Scenario

A company is planning to launch a service to replace an existing service.

Marketing want to run a focus group to understand how users are using the current service to see how the new service can be improved.

Usability is new in the company. Is a focus group the best choice to answer their questions?

Alternatives?

Page 24


What’s wrong with focus groups?

They are good for:

• Exploring what users or customers want • Gaining consensus on a concept

But..

• Provide little depth from any participant • Capture opinion, not behavior • Strong group members can skew results • Provide weak qualitative data

You can get better data for the same cost with in-depth interviews

Page 25


Selecting the “right” tool?(Withney Quesenbury and Daniel Szuc, 2005)

To understand the environment…

think observation, not usability testing

To test a concept…

think rapid prototyping, not detailed specs

To find market advantages…

think about an early competitive usability test

Page 26


Methods for learning about the users

Page 27

To Use

Learn about the users in their own context

Site visits (ethnography / contextual inquiry) to observe users in their own setting

Know their goals and how they work Scenarios of uses and task analysis to explore and document their workflow

Identifying factors in the environment Context of use audit to understand their environmental, social and access needs

Identifying quantitative demographics Review (or conduct) market research on product usage, user demographics, etc

Document user analysis Create personas to document key aspects of different user groups


Methods for collecting business data

Page 28

To Use

Learn about a new business context Conduct stakeholder interviews to collect input from different parts of the business

Find trends or gaps in business process Review problem reports from technical / customer support for usability problems

Understand traffic patterns Review site logs for patterns of traffic on a web site

Understand the competition Conduct a competitive auditor comparative usability test with competitive products


Methods for analysing information and tasks

Page 29

For Use

Understanding how users do their tasks today

Task / workflow analysis techniques to document steps and variations

Learning about relationships between information or tasks

Card sorting to create logical groups users will understand

Deciding how to organise the task in the application

Navigation and flow charts that group the interaction steps into screens

Exploring different interaction designs

Paper prototyping and task walkthroughs (with users or personas)


Methods for evaluating designs in progress

Page 30

For Use

Team review of a conceptual design

Walkthroughs using scenarios and task models

Collecting input from colleagues on the design

Hallway reviews to collect rapid input

Testing aspects of the design, or reviewing progress

Usability evaluation with users

Measuring overall success in meeting usability goals

Summative usability testing with users

Satisfaction surveys after release


Selling usability(Withney Quesenbury and Daniel Szuc, 2005)

Start with a small usability effort

Example:

If a company never done any form of usability testing, run an informal usability trial with an internal user. When the company management sees that even such a simple activity produces good insights, they may be more confident about doing employing user trials and using usability engineers.

Page 31


Selling usability(Withney Quesenbury and Daniel Szuc, 2005)

Have team members participating. People are more open to ideas they have helped create. Participating lets them see that it’s not “magic” but real, valid techniques

Example:

Use participatory walk-troughs to get everyone thinking about the problem

Page 32


Selling Usability(Withney Quesenbury and Daniel Szuc, 2005)

Don’t attack the product or design –the people who created it are in the audience

Page 33


Intangible costs of Human Factors software lifecycle

1 - The selection of non-critical design decisions for user studies 2 - The establishment of too high a level of usability 3 - Falling into the trap of overdesign 4 - Communication problems between Human Factors specialists and

software designers

Page 34


Selection of non-critical design decisions for user studies

Not all design decisions affect the overall “quality” and “acceptability” of the final UI

How do you determine which ones are worth investigating?• Examples: Arial or Helvetica? 11 pt or 12 pt? Icon names?

Most usability experts rely on experience and intuition to decide… … but they can be wrong!

Costs: Wasted user study time Discovery of important changes may be put off until later

Solution: Make your user studies flexible enough to allow discovery of new problems

Page 35


Establishment of too high a level of usability

Some UI performance targets simply cannot be met Example: “This system must be learnable within 1 week” This may appear to be a reasonable target … … but what if the task itself takes >1 week to learn?

Costs: Design and development time chasing unrealistic performance targets

Solution: Set realistic targets This can be informed by your task analysis studies Also, aim for incremental improvements in performance over design

cycles

Page 36


Falling into the trap of overdesign

Prototyping software (UIMS) can make it very easy to add more features (“bells and whistles”)• Examples: borders, colours, icons, images, etc.

How to know when to stop??? How to know what are valuable additions?

Costs: Designer’s time + implementation time

Solution: Strong management control

Page 37


Communication problems between Human Factors specialists and software designers

There is a knowledge gap between HF specialists and the coders/developers…

Unless there is a shared common language and understanding, it is difficult to communicate effectively

Costs: Time lost in establishing a common language/understanding Time lost developing the wrong thing (or the thing wrongly) Time lost designing solutions that simply cannot be implemented

Solutions: HF specialists learns to develop own UI Better corporate culture of communication

Page 38


Intangible benefits from Human Factors software lifecycle

1 - Adoption of features that save time 2 - Avoiding system sabotage problems 3 - Enhancing the ability to solve conceptual problems using the

software system

Page 39


Adoption of features that save time

Feature adoption by users is reduced when the system is so complex that users give up trying to learn the advanced features• i.e. they satisfice

Users will typically adopt the least complex system that offers them the required functionality• Even if it is less efficient

The Product Acceptance Analysis should identify unnecessary features

The User Tests should reduce feature complexity

Page 40


Avoiding system sabotage problems

Being asked to use a system that is inappropriate, difficult or inadequate can lead to employee frustration

This frustration can lead to employees “taking it out on the system”• Example: entering incorrect or incomplete data• Example: reporting false system failures

This is referred to as “system sabotage”

Focus groups in Market Analysis and Product Acceptance stages are designed to address this issue• They test the receptivity of the target users to the new system

Also, the Task Analysis ensures that the final product matches the needs of the users as closely as possible

Page 41


Enhancing the ability to solve conceptual problems using the software system

Increasing the user’s cognitive load to use the system … … decreases the user’s cognitive capacity for “solving the problem”

So although the problem may be “solved” … … there may have been a “better” solution

User Testing is designed to remove complexities from the system … … and thus support users’ creative problem solving

Page 42


Recommendations for Human Factors inclusion

Page 43

The HF lifecycle stages and the type of cost reduction that they are most likely to effect

Cost reduction item Related lifecycle stage

Increased system adoption Market analysis

Product acceptance analysis

User testing and evaluation

Reduced training costs Task analysis

User testing

Reduced user errors Task analysis

User testing

Transfer of design changes to an earlier stage in the lifecycle

Prototype construction

User testing (on prototype)

Product survey (next re-design)

© Simeon Keates 2008Page 44

Cost of running User Tests vs. size of user population

Page 44

Size of user population

$

Break-even point

Benefits of testing

Cost of testing

User testing not cost-effective

User testing cost-effective


Summary of cost-benefit analysis

For any large user group, the benefits will usually outstrip the costs … …often by a significant margin

For smaller user groups, the case is less clear-cut … … so either decrease costs …• e.g. through “discount” usability methods

… or examine the intangible benefits more closely

Page 45


Exercise


Exercise

Conclude your user trials

If you want feedback on your group presentation – then prepare slides for next Friday• NOTE – this is optional, but recommended


Tips for an effective presentation of usability results

Executive summary

• Describe methodology (depending on your audience, you might need to explain or demo the product, describe the usability testing process, or provide background information. At a minimum , your audience will usually want to know something about the participants in the test and the tasks they were asked to perform)

• Who, what, when, where, and how• Describe how tests were conducted• Profile users and describe sampling• Detail data collection methods

• Succinctly explain the analysis• Provide screen captures (pictures really do work better than words in most cases)• Include tables and graphs• Provide examples

• Identify strengths and weaknesses• Recommend improvements

Page 48

Documents

Usability with Project 16/04/10