Embed Size (px)
DESCRIPTION
Slides from my course "Usability Testing and QA" taught at BAVC 7/18/2014
Citation preview
Usability Testing & QAHere early? While we'ʹre waiting for the others...
1. Identify some websites you'ʹd like to testBest if they are likely to be new for your classmates. If you have a site (or sites) that you work on, even better!
2. For each website you'ʹve chosen, identify 3-‐‑5 key tasksWhat are the 5 most important things a user should be ableto accomplish on the site?
Usability Testing and QAA practical approach
towards creating better user interfaces
Who am I?Shilpa Thanawala
@skthana
And you?
Name
Role at your company or agency
Background (eg. designer? developer? etc.)
Experience with usability testing
How you think you’ll use what you learn today
Why this class?
Housekeeping
Everyone knows where things are at BAVC?
Cell phones on silent, calls outside
Initial and sign the roster
Evaluations at the end of the day
Load the shared Google Doc
(for questions, backchannel interactions, sharing links)
http://bit.ly/ux-‐‑july-‐‑18
What is Usability Testing?
Usability testing refers to evaluating a product or service by testing itwith representative users. Typically, during a test, participants willtry to complete typical tasks while observers watch, listen and takesnotes. The goal is to identify any usability problems, collect qualitativeand quantitative data and determine the participant'ʹs satisfaction
with the product.usability.gov
Image credit: templatemonster.com
Usability testing is a technique used in user-‐‑centered interactiondesign to evaluate a product by testing it on users.
wikipedia.org
Image credit: Mediamatic
Usability is a quality attribute that assesses how easy user interfacesare to use.nngroup.com
Image credit: Optimum7.com
First, some historicalhighlights
Early 20th CenturyImprovements in industrial efficiency
In WW I, reduced work motions for soldiers to assemble &take apart weapons in the dark.
http://www.measuringusability.com/blog/usability-‐‑history.php
1947
John Karlin -‐‑ how short phone cords should be
“It is not so much that Mr. Karlin trainedmidcentury Americans how to use the telephone. It is, rather, that by studying the psychologicalcapabilities and limitations of ordinary people, he
trained the telephone...”Excerpt from New York Times article
1980’s
New methodologies developed and published
Usability becomes a profession
Defined as a function of efficiency, effectiveness, andsatisfaction
first publishedPsychology of Everyday Things
1990’s
Usability matures
Diminishing returns from testing more than 3-‐‑5 participants
Books by Jakob Nielsen
2000 -‐‑ todaySteve Krug’s Don’t Make Me Think
Remote testing
Statistical analysis of usability data
Mobile devices, touch and gestures
http://www.measuringusability.com/blog/usability-‐‑history.php
Why?
Why spend valuable time and money doingusability tests?
It'ʹs the best way to learn how users actually interact withyour website. Watching > interviewing
You don'ʹt really know your users'ʹ needs. Users are not all thesame. And they'ʹre not your Mom.
All websites have problems, and the serious ones are easy tofind.
Why spend valuable time and money doingusability tests?
You'ʹll learn things you didn'ʹt know that you didn'ʹt know.
Watching users gets everyone on the same page.
Watching users will make you a better designer / developer /product manager / content writer / etc.
Usability testing doesn'ʹt have to be expensive or time-‐‑consuming.
What Testing Can'ʹt Tell You
Limitations of Usability Testing
Won'ʹt fix your site'ʹs problems (it'ʹs not a user training session)
Results are dependent on the questions you ask (tasks)
Can'ʹt model social interactions very well
Isn'ʹt always representative of real situations
For more on issues testing the social web see Dana Chisnell'ʹs talk
When?
When should usability testing bedone?
Before you design or build anything
Before you implement a change
Throughout the project
After implementing a fix
Throughout the life of the website
As soon as possible & repeat often!
Demo: A short usability test
Qualitative vs. Quantitative
Qualitative vs. Quantitative
Quantitative Usability StudiesDesigned and conducted to produce data for statistical analysis
Each measurement represents a count or an amount
Distances, weights, quantities, etc.
Qualitative Usability StudiesFocused on verbal descriptions of users'ʹ experiences
Each measurement is a description or category
Words, sentences, feelings, yes/no
Categories are not quantitative, even if they are assignednumbers
userfocus.co.uk
“...qualitative methods are much better suited for answeringquestions about why or how to fix a problem, whereas quantitativemethods do a much better job answering ‘how many’ and ‘how much’
types of questions.”
More on Quantitative methods at
"ʺWhen to Use Which User Experience Research Methods"ʺ, nngroup.com
measuringusability.com
Usability Metrics
Effectiveness, Efficiency, and
Satisfaction
EffectivenessAccuracy, correctness. Are users able to achieve their goals?
EfficiencyEase of use, speed. How much effort (time) does it take to
complete a task?
SatisfactionUsers'ʹ perception. What does the user think about their
experience?
More at , usability.gov wikipedia.org/wiki/Usability
Some Metrics for Common User Tasks
From Measuring the User Experience – Tullis & Albert
Performance-‐‑based Metrics
Task SuccessCommonly used, pass / fail, or degrees of success, shouldhave clear end-‐‑state.
Task TimeParticularly important for repetitive tasks
ErrorsIncorrect actions leading to significant time loss, additionalcosts, or task failure
EfficiencyEffort and time needed to complete the task
LearnabilityEffort and time needed for the user to learn how to use theinterface
Issue Metrics
Usability findings based on users'ʹ behavior
Consistent / repeatable, or idiosyncratic?
Remain open-‐‑minded to the unexpected
Other Metrics
Self-‐‑Reporting: Users'ʹ perceptions, feelings
Behavioral / Psychological: Eye-‐‑tracking, stress,unprompted verbal expressions
Combinations / Comparisons: Interpretations based onmore than one metric
Card-‐‑Sorting: How to organize information in a way thatmakes sense to users.
A/B Testing: Comparison of two alternate designs
Accessibility: Usability for those with disabilities ( )WCAG
Bias
Impossible to eliminate entirelyParticipant background, knowledge, comfort level
Task definitions
Methodology (testing process, session length, how muchtalking)
Artifacts of the prototype or product
Environment (lighting, noise, cameras, distractions)
The Facilitator (experience, skill, style)
ExpectationsMeasuring the User Experience – Tullis & Albert
Number of Test Participants
What'ʹs the right number of users to recruit fora study?
3-‐‑5? 4-‐‑6? 10? 20? 100?Those who favor a small number feel 80% of the mostimportant usability problems are detected by the first fewusers.
Cases in which more users may be required:You need to be sure you'ʹve captured as many problems aspossible
You have more than one user group (persona)
You have a large number of screens, complex design, widevariety of tasks
You'ʹd like to account for evaluator bias
Why we'ʹll focus on qualitative testing today
Anyone can do itNo knowledge of statistics or data analysis needed; based onverbal feedback, observation, and empathy
Cheap, fast, and easyGreat for small budgets, busy teams, minimal resources... andfrequent repeats
It gets the job doneMore than sufficient to identify a website'ʹs most serioususability problems
Break10 minutes
DIY Usability TestsBased largely on Rocket Surgery Made Easy – Steve Krug
Establish your GoalsWhat do you want to learn about your site?Can users find products easily?
Is it clear to users what the site is about?
Is the information organized intuitively?
Is this new feature distracting?
Keep an open mind to allow for unexpected results.
Define your TasksCreate scenarios based on essential, real-‐‑world user goals
Tasks should be realistic
Tasks should be actionable
Phrase tasks so as not to give away the answer
Print each scenario on its own piece of paper. (Avoid labels or numbers.)
"ʺTurn User Goals into Task Scenarios for Usability Testing"ʺ — nngroup.com
Recruiting Test Participants
What kinds of participants?
Where to find them?
How many?
How to compensate them?
What kinds of participants?
How representative of actual users?
Most serious UI problems will be discovered by non-‐‑
representative users.
Testing with non-‐‑representative users is better than obsessing
over finding representative users.
Key requirements: not part of the team, fluent in the UI'ʹs
language
Where to find participants?Where do your users hang out?
Advertise on the site, message boards, social networks
Craigslist
How many users?
For most of us doing DIY testing, 3-‐‑5 participants is fine.
No-‐‑shows or last-‐‑minute cancellations are common. Have 1-‐‑2
backup participants lined up.
Phone-‐‑screen, scheduling, day-‐‑before reminders, directions,
parking, NDA'ʹs... use a checklist
CompensationShows appreciation for participant'ʹs time and effort
Encourages participants to be engaged and enthusiastic.
Gift cards, free product or service.
Equipment & Setup
Hint: you don'ʹt need a lab
Bare minimum (if you'ʹre the only one): a computer in a quiet
room, a pen and notepad, and 2 chairs
For a team, add a remote observation room, a good mic, and
screen-‐‑sharing (Skype or GHangout). And provide
irresistable snacks.
Can use screen-‐‑capture software (but most of the time it
won'ʹt be watched)
Can record the participant (but it'ʹs not really needed)
Make it a Team EffortTry to get everyone to attend.
Assign a trustworthy team member to manage theobservation room
The Session ScriptStart with a template & edit to suit your needs
Welcome / introduction
Logistics, release forms
Necessary background information
Test: general questions
Test: scenario activities
Wrapup, follow-‐‑up questions
Conclude and provide honorarium
Some Test Session Script Templates
Free templates at: , , infodesign.com Steve Krug'ʹs site Book siteguide by Rubin, Chisnell & Spool
Debrief
Only for those who attended at least one session
Schedule it on the same day (next day latest), 60-‐‑90 min.
Provide food!
Get all observers to state (or stick) UX problems they
observed
Get general agreement on which are most serious & will be
tackled first (2-‐‑3 maximum)
Remind everyone when the next round of testing will take
place
Run your own usability test1. Select a website to test
2. Determine goals
3. Identify at least 3 tasks
4. Formulate scenarios for testing each task
Lunch1 hour
Run your own usability test1. Prepare your script
2. Rehearse and refine
3. Print scenarios
Run your own usability test1. Test two participants (~20 min each)
2. Take notes
Run your own usability test1. Debrief within your original group
2. Identify 2-‐‑3 usability issues
ToolsTesting session checklists
, ,
Usability testing software , ,
Online / Remote services , , ,
Usability.gov Krug'ʹs checklist checklist on book site byRubin, Chisnell & Spool
Silverback (Mac) Morae (Win) Camtasia
Usertesting.com Loop11 many more usabilityhub.com
So how do we fix it?
Fixing usability issuesThe bad news: most issues never get fixed
Make the smallest change possible to "ʺstop the bleeding"ʺ
Large changes might introduce new issues
It'ʹs not fixed until you verify by ... testing again!
Testing Accessibility
Why is accessibility important?
“The Web is an increasingly important resource in many aspects oflife: education, employment, government, commerce, health care,
recreation, and more. It is essential that the Web be accessible in orderto provide equal access and equal opportunity to people with
disabilities. ”From The W3C'ʹs Web Accessibility Initiative
Is my site accessible?
Unplug your mouse / turn off your trackpad
Why -‐‑ blind, low-‐‑vision, compromised motor-‐‑control, no
hands
How to test -‐‑ start at url, use tab / shift-‐‑tab and arrows to see
whether you can access all controls, links, and fields on the
page, in the right order. Popups and dialogs should be
usable.
How to fix -‐‑ proper HTML heirarchy, tabindex attribute,javascript plugins
Is my site accessible?Check color contrast
Why -‐‑ low-‐‑vision; 1 out of 12 have color deficiency
How to test -‐‑ online tools ( , ),browser plugins ( )
How to fix -‐‑ shift color palette to improve contrast, increasefont sizes for low contrast areas, avoid colorful backgroundimages
Check my Colors Contrast RatioGrayscale Tool
Is my site accessible?Turn off images
Why -‐‑ images useless to low-‐‑vision or blind users, lowbandwidth
How to test -‐‑ block images in browser, WebAIM tool( )
How to fix -‐‑ proper alt attributes, don'ʹt use images when textcan be used
WAVE
Accessibility Resources and Tools
Quick accessibility tests & fixes
WebAIM
W3C'ʹs Web Accessibility Initiative
SitePoint'ʹs Easy accessibility checks
Mobile / Touch
Mobile Usability
Context / environment (noise, lighting, glare)Many more contexts beyond desktop
BandwidthMobile devices force us to think about bandwidth again
Touch target sizeAverage fingertip size = 10mm Adjustments in spacing, button size Touch is also being used on large screens
Driving / attention What usability issues contribute to driver distraction?
Mobile Usability Testing Resources
NNGroup recommendations
Article on UXMagazine
Break10 minutes(back at 3:50)
A/B Testing Activity
A Usability Test using PaperPrototypes
Paper prototype of a kids’ website
Credit: BlueDuckLabs / YouTube.com
View online
A/B Testing Activity1. Select one usability issue from your previous activity
2. Brainstorm at least two ways to fix it
3. Print the page and use the paper, scissors, tape, and pencilsto mockup both cases
4. Test each one on a separate participant
5. Debrief in your group. Which worked better, A or B?
Accessibility Testing Activity1. Click the URL at the top of the browser window
2. Without using the mouse again, try using only the keyboardto navigate
3. Can you access all the links, menus, and controls?
4. Can you tell where you are?
Wrap-‐‑upUsability testing is an essential tool for improving yourwebsite or product
Anyone can do a simple usability study
Test as early as possible, as frequently as possible
Tailor the type of testing you conduct (qualitative /quantitative, metrics, number of participants) to give you theinformation you'ʹre after
Use the available tools and resources to help you run yoursessions
Wrap-‐‑upGet your whole team involved
Small adjustments are preferable (and faster) than a fullredesign
Remember to test your fixes, too
Stay aware of non-‐‑standard users (accessibility) and thechanging technological landscape (new platforms, new uses)
ResourcesWebsites / Blogs: , ,
,
Quantitative Testing: , ,
nngroup.com usability.gov sensible.com(Krug) UIE
measuringusability.com Measuring theUser Experience – Tullis & Albert usability.gov
Thank you!
Shilpa Thanawala | @skthana
