07

Prof. Garzotto

HCI – Lesson 7

Notes on Communicating Usability Problems

07

The following notes are meant to provide some general

guidelines for a fully professional usability inspection report –

your HM report should be more concise

WARNING

3

Relevance

• Discovering usability breakdowns is a necessary but not sufficient condition for successful usability analysis

• Communicating problems effectively to the relevant stakeholders is crucial for the success of the mandate

• An excellent usability analysis may fail if the communication fails

• This communication happens in form of written reports and/or presentations

Goal: Provoke “Habit Change”

• Habit Change (change of perspective): “I thought we had a good design, now you convinced me we have important problems to work on”.

• Objectives: – 1. Convince stakeholders and/or peers of the

importance of your analysis – 2. Provide them with the necessary elements to

orchestrate the strategies for addressing the usability problems

5

Usability Problem Lifecycle

Problem and Representation

• Solving a problem simply means representing it so as to make the solution transparent.

– Hebert A. Simon, The Sciences of the Artificial

7

Key Principles

• Abstraction Level

• Separating Concerns

• Recognize Interconnections between problems

• Requirements vs Design Solutions

• Grounding your Findings

• Respecting the Design

• Prioritizing Results

• Technicality

• Organizing the Results of the Usability Inspection

Abstraction Level - Examples

• Poor:

– P1: “Subscribe” and “Enroll” appear as overlapping in meaning. The user may be confused as to where to go to register to this service.

– Example: <screen shots> (without comments)

– Recommendation: Clarify the two labels with more distinctive terms, such as, for example: “Subscribe to the Newsletter” and “Enroll in our Program”.

• Better:

– P1: Generic navigation labels. Approx. 85% of the main navigation labels critical for key tasks are too generic or overlapping in meaning. This may cause users problems when….

– Example 1: commented screenshots …. “Subscribe” and “Enroll” appear as overlapping in meaning. The user may be confused as to where to go to register to this service.

– Recommendation: Redesign the main navigation labels by making them more distinct and specific. This will help users in clearly recognizing the underlying content before clicking on them. For example, …

Abstraction Level

• For each problem, make sure that you identify and characterize first the general problem, as independently as possible from examples of occurrences.

• The example must be just one instance of occurrence of the problem

• This difference is crucial in large or very large design

• Less relevant when inspecting small applications or problems on very specific features

Abstraction Level

• Raise the level of abstraction as appropriate to capture salient issues across several instances – Use abstraction levels to groups problems together meaningfully,

when they are simply variants of the same issue

– Use your usability knowledge (heuristics) and familiarity with the design to reason across different levels of granularity

• Do not suggest a fixing “by example”

• This demonstrates your ability to master the design in its complexity

11

Separating concerns

• Analytically decompose the various issues involved in a usability issue into salient design dimensions (content, info architecture, navigation, semiotics, graphics, layout, etc.)

• This is fundamental to help organize the re-design activity

• Assign your specific problem to one and only one dimension

• This will help you organize your problem set around specific concerns for the designers (see later: organization).

Separating Concerns - examples

• Use schemas to communicate design aspects which may not be emerging at the page level (information architecture, operations, overall navigation, flows)

Properly explain,

comment your schemas, restate the problem in narrative

Separating Concerns

• Be analytical and rich in your communicating your problems

• Make sure that the message (the key element of the problem) is getting across. Be redundant if appropriate and use multiple strategies to convey the concept (be creative!)

• Do not just let the design sit there, make it talk (engage with it!)

• Communicate metrics when appropriate (e.g. n. links, n. objects, units)

Basic unit of problem communication

• Problem Name:

• Reference Scenario:

• Design Dimension / Source Heuristics:

• Problem Description/Characterization:

– 2 elements: (a) Design defect (cause)

– (b) Expected difficulty for the user (UX consequences)

• Examples (with illustrations/commented pictures):

• Severity (think to reasons to justify your rating):

• Requirements for Improvement

Separating Concerns - example

Separating Concerns - example

Separating Concerns - example

Use “comparative” strategies:

- Compare design units

- Do not expect that

designers

recall the elements

of the design

you are referring to

Indicating Requirements vs Design Solutions

• Examples

• Poor: – P1: The secondary navigation bar is very crowded in terms of overall number of links

(avg. 37), and groups (avg. 5) and subgroups (avg. 8). The criterion by which elements are grouped is not apparent, as well as the overall ordering and prioritization of the groups.

– Example: <screen shot> specific section, groups, links with comments

– R1: <new menus redesigned>

• Better: – P1: The secondary navigation bar is very crowded in terms of overall number of links

(avg. 37), and groups (avg. 5) and subgroups (avg. 8). The criterion by which elements are grouped is not apparent, as well as the overall ordering and prioritization of the groups.

– Example: <screen shot> specific section, groups, links with comments

– R1: a recommended criterion to order links within the designed groups may be A-Z because….; choose criterion across groups according to the importance of the tasks to be supported. For example, Task 4 is critical…

Indicating Requirements vs Design Solutions

• For each problem, recommendations for improvements should indicate the strategy to follow to get the solution(s), but NOT a dictated solution.

• Recognize that there are many specific solutions to address the problem

• You may not ideate the right solution in your usability analysis (it is not your job now) – The actual solutions are negotiated in the overall design economy

– If appropriate, provide multiple options (e.g. 2-3) to go about addressing the problems

• In other words, a solution may be suggested (as part of a general recommendation), but not imposed.

Recognize interconnections between problems

• Reason explicitly about the trade-offs to consider

• Your recommendations (across design dimensions) may influence one another

• When commenting on a problem recommendation, be aware of what are other elements this may affect – Example: recommending very specific and precise labels (semiotics)

may impact the layout strategies

– Show awareness of these interconnections

• This will help designers think to “systemic” solutions

21

Ground your Findings

• Give reason of your findings

• Drawing on elements which can gain credibility – Experience: expertise of the analysts

– Consequences: expected impact on the user experience

– Anomalies: compliance with the standard and conventions

– Significance: the reasoning behind the design can be questioned

• Provide arguments, and use, illustrate examples intelligently to convey the importance of your analysis

Respecting the Design

• Ultimately, show respect for the work of the designers

• There is a rationality (maybe wrong) in the design – Show respect for the possible rationale (plausible but perfectable) of the

current design decisions, and engage with it.

– E.g. we understand there is a lot of content available and there is the need of providing more than 50 access links; however, instead of displaying them all at once a possible strategy is to…

• Double-check the accuracy of your problems – A functionality may not be easy to discover. Be thorough and detailed

before indicating that it is missing. Maybe it is not apparent, difficult to find, not intuitive, transparent.

• Acknowledge good design when it occurs

Prioritizing Results

• The traditional severity level is one factor to prioritize problems (a severity level is attached to each problem)

• Other factors: – “weight” , “population”, “significance” of a given design dimension:

• “navigation” has a lot of problems (even if minor issues)

• Overall logic of info arch is very problematic (few, major concerns)

– “significance” of a scenario • “Find a venue” has many, many obstacles…

• “the buying process” is key to the mission of the applications

– Estimated re-design “effort”

• Simple “fixing”, or major re-thinking, re-engineering, …

24

Technicality

• Avoid usability jargon

– Dot not talk “usabilish”

• Adapt your concepts and wording to the target audience you are communicating with

– E.G. You are communicating in front of the project managers and design/development team (with different expertise)

• For example, your audience does not necessarily know what is an “heuristic”, what is a scenario, how your analysis has been carried out

• “Culturally” translate the technical jargon into meaningful insights (focus on the substance) – Provide references where appropriate for technical specifications

Organizing Usability Inspection Results

• Strategy 1: Group Problems By Scenario

– Scenarios (by population/significance)

– Within each scenario/task (by design dimension or by severity)

– Works when most of your problems can be convincingly and coherently justified with reference to critical scenarios

• Strategy 2: Group Problems By Design Dimension

– Dimensions (by population/significance)

– Within each dimension: by severity (not by heuristics)

– Reference of each problem to a scenario is recommended (when appropriate) but not necessary

• The choice of the appropriate strategy depends on the nature and distribution of the problem uncovered

Organizing Usability Inspection Results

• Additional sections that may deserve separate grouping of problems (e.g. homepage, search engine, etc.)

• Provide overview of your inspection results, with overall metrics, to give an idea of the magnitude of your results

• For example, you can map your results over the overall design architecture to show: – Most critical areas

– Presence of most (severe or dimension X) problems

Navigation problems

Layout issues

Content issues

Organizing Usability Inspection Results

The Usability Inspection Report

– Cover (1 page)

– Table of contents (1 page)

– Executive Summary (1 page)

– Overview of the Application (2 pages)

– Methods Used (1 page)

– Usability Inspection Results (ca. 20-25 pages)

– Recommendations for improvement (2-3 pages)

– References (1 page)

– (Appendixes): N pages • Synopsis of usability problems / recommendations

• Additional material

– Ca. 30 pages (appendixes excluded)

Readings

• Bolchini, D., Colazzo, S., Guidelines for Describing Usability Problems, in Proc. HCII 2005 International Conference on Human-Computer Interaction, Las Vegas, USA, 2005.

• Nielsen, J., Mack, R., Usability Inspection Methods, Wiley, 1994 [Chapter 11].

• Bruce Tognazzini, www.asktog.com

• See example of usability inspection report on OnCourse/Resources/Examples Midterm Projects