View
117
Download
5
Category
Tags:
Preview:
DESCRIPTION
Although usability and user experience may seem synonymous, they are separate and much different concepts. While usability is well defined in standards, UX has no agreed upon definition because it relates to a more nebulous attribute-user satisfaction. Both are, however, key ingredients for successful system deployment. Because they don’t know how to measure and evaluate UX, many teams ignore this important attribute until the end of development. Philip Lew discusses how to model both usability and UX by breaking each attribute down into measurable characteristics-learnability, user effectiveness, user efficiency, content quality, user errors, and more. Phil shows you how to derive measurements and metrics that your development and team can employ to benchmark, analyze, and improve both usability and UX. Beyond the measurements, Phil discusses case studies in which measurements have driven significant usability and user experience improvements.
Citation preview
BT12 Concurrent Session 11/8/2012 3:45 PM
"The Why and How of Usability and User Experience (UX) Testing"
Presented by:
Philip Lew XBOSoft
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Philip Lew XBOSoft, Inc.
After working for twenty years in various management and technical positions in product development and management, CEOPhilip Lew now leads the direction and strategy of XBOSoft (xbosoft.com). He has helped hundreds of organizations assess the quality of their software, examine software quality processes, and set forth measurement plans so they can consistently improve software quality using systematic methods. Phil has authored articles in IEEE and ACM journal publications and trade journals; presented at several conferences on web application usability, user experience, and quality evaluation; and is conducting post-doctorate research focused on user experience measurement frameworks and software learnability.
1
The Why and How of Usability and User Experience (UX) Testing
Philip Lew
Agenda
• What is usability and UX and why i t t?important?
• Usability and UX – What are they• How to set up models and measurements• Case study and examples in testing and
evaluating applicationsevaluating applications
2
What is Usability-UX and Why is it Important
The Year of Mobile?
• Last year was the year of mobile in terms f th th i b th bil d thof the growth in both mobile usage and the
availability of mobile sites and mobileapps. • Time to redesign your mobile site,
because your existing version is probably far below users' growing expectations forfar below users growing expectations for user experience quality.
• What do you think users expect?
4
3
History Repeats Itself• In the beginning, the Web was experimental —it was
acceptable to have a somewhat shaky, experimentalacceptable to have a somewhat shaky, experimental website.
• Many sites were crippled by misguided design advice• Many didn't know any better (because they didn't do
usability studies). Now, people simply expect websites to work.For mobile last year it might have been cool simply to• For mobile, last year, it might have been cool simply to have an app.
• Now, that app better be good. Expectations have gone up.
5
Web and Mobile User Expectations
• Business models have changed– Instead of paying
upfront and ‘owning’ the software
– Pay as you go, pay by subscriptionp
• Cloud and mobile converge
• Behavior and expectations have changed
4
Basic Usability Concepts
User Experience
Usability Effect
Usability Design
7
Design-Test and Evaluate
Design• What will the mobileapp do?
Test and Evaluate
Test and Evaluate
mobileapp do?• Is it a conversion of existing app?
• What functions will a user really
Release
8
access?
5
Usability - Design Perspective
• Understandability• Learn-ability• Operability• Attractiveness• Navigation• Responsiveness-performance
What else can you think of?
9
Usability-Effect
D t hi h ifi dDegree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a
Source: ISO 25010
efficiency and satisfaction in a specified context of use.
10
6
Usability-Effect“Context” and “Specified”
• User role specified users
• Objective• Task• Environment
specified goals
Environment• Domain• …
specified context of use
What else can you think of?
11
Usability-EffectUser Experience
• Satisfaction– The degree to which users are g
satisfied in a specified context of use. Satisfaction is further subdivided into sub-characteristics:
• Likability (cognitive satisfaction)
Source: ISO 25010
• Pleasure (emotional satisfaction)• Comfort (physical satisfaction)• Trust
• Including many other factors experienced over time and other channels
12
7
Task Ease(effectiveness and efficiency)
• Buttons are a key tool in the user experience designer’s box of tricks.box of tricks.
• Prioritize tasks - Some tasks more important than others. • Understand the objectives of the application • Understand which tasks are really important. • Majority of your app’s value is provided by a small
number of tasks.• Remove any friction that inhibits the user’s progress. • Choose your words carefully to make it clear what the
buttons or functions do
13
What could add ‘friction’ ?
Design and Test For the Task• Highly specific tasks.
– You are in an electronics store looking at the Minolta GS234 camera It costs $120 25 in the store CheckGS234 camera. It costs $120.25 in the store. Check costco.com to see if you can get a better price online.
• Directed, but less specific. – Find a moisturizer with SPF 45 or above that is
suitable for your skin." (Use Walmart app.)• Open-ended, but restricted to a predetermined site-app.
– "See if you can find any interesting pictures related to today's news." (While using the USA Today app.)
• Web-wide tasks that let users go anywhere. – "Find out which is the tallest building in the world."
14
8
Usability
Basic Design Issues and ConceptsConcepts
Design Testing and Evaluation
15
Design Concepts Test and Evaluation Criteria
• Context• Context• Positioning• Color usage• Size usage• Size usage
9
User Context is King
• Mis-Targeted user b k dbackground can cause loss of effectiveness
• Know your users.
17
Who are your users?What are their special needs/characteristics?
Positioning• Effectiveness of
application buttons ppis affected by position.
• Emphasis is gained by – removing clutter– placing the button
where the user’s eyes will mostly likely be.
18
10
Using Color
• Color creates emphasis. • Dependent on the importance of the
application’s function.• Make easy to for user to see the most
important stuff.Color can affect usefulness• Color can affect usefulness.
19
LinkedIn - ColorCreate an account!!!
• The ‘View Full Profile’ button– LinkedIn wants you to click that.– You’ll be prompted to create an account!
• Button has a unique color that isn’t shared with any other part of the design.
• Page has tons of information, this button still stands out.20
11
Don’t Overuse a Color
21
Size Matters• Time taken to point at an object is directly influenced
by the size of that object. • Big is beautiful depending on function-what you want
the user to do.• Size can dictate the button’s importance over
everything else on the page. • Take the Firefox page for example, Mozilla don’t mess
around with subtlety here.y• Good design communicates priority.• With one massive button on the page, that priority is
obvious.
22
12
Using Size AND Color
23
Multivariate Testing• Small changes can make a big
differencedifference.• Laura Ashley arrived at this design
following multi-variate testing of 5 different options
• Differences in the testing included:g– Link colors and locations
• The "Go to checkout" button was dark gray instead of green
• Achieved 11% increase in checkouts 24
13
Let’s Go Mobile
Usability and UX For Mobile
25
Usability - Desktop to Mobile• Just loading websites on the phone is not
enoughenough • What matters for users
– Time to load the website– User interface– Accessibility of various functions available
• Usability design needs to change, otherwise usability effect and UX will suffer
26
14
Different Platforms Need Different Design
• Horizontal swiping now used to "flip" through deck-of-cards or carousel features. – Include a visible cue when people can swipe
• Avoid swipe ambiguity: don't use the same swipe gesture to mean different things on different areas of the same screen. – Use the same meanings for mobile phones
and tablets – Mobile-device users typically expect to horizontally
swipe while desktop websites avoid horizontal scrolling
• Mobile sites should perform better than full sites when used on a mobile device.
Nov 2012 Better Software East 2012 27
Let’s Step Through a Few Examples
Usability and UX For Mobile
28
15
Laura Ashley Mobile Site
• Same as web versionG ti k t• Green sticks out
29
Laura Ashley
• What is wrong here?
30
16
Laura Ashley – 9/2012
31
LA MS
• Consistent greenOb i l t t• Obviously want you to click and add to the bag
32
17
LAShopping bag
• Clear green signals… BUYBUY
33
LA MobileApp
• Main ScreenD t i b tt• Decent size buttons
• No priority or importance weighting
34
18
LA Mobile App
• Big buttonsG d f bi fi• Good for big fingers
• Semi-meaningful icons on the bottom– Note contextual
location
35
Buy or Share?
• No placement of importance or priorityimportance or priority
36
19
Select Size
• Good navigationG d i b tt• Good size buttons
• No priority
37
In my Basket
• What should I do now?now?
• Buy or share?
38
20
Shopping Basket
• Empty or Checkout?
39
Bloomberg
• Simple Information AppApp
• Simple colors• Meaningful Icons• Big enough to see
40
21
Fidelity
• Easy to use scroll– Thumb Friendly
• Simple colors• Very limited task
based functionality
41
Fidelity
• Size and ImportanceP tt t• Pretty easy to see what they want you to do
• Consistent icons on bottom
Si l– Simple– Not everything
42
22
Yelp
• Easy on the eyesS i i f l• Semi-meaningful icons
43
Yelp
• ConsistentFi f i dl• Finger friendly
• Consistent location of icons on bottom
Similar to Fidelity
44
What does this mean?
23
Making Assumptions
• Web sites automatically sense the platform and
it h t ‘ bil it ’switch to a ‘mobile site’ (for instance, United Airlines, Google, and others) – Provide mobile users with
more efficient web experience. e pe e ce
• Has the potential of making too many assumptions regarding the users’ expectations.
45
Web-Mobile Integration
• Overall integration of web-apps needs to be thought out carefully with mobile scenarios in mind.
• Friend wanted to show me a product at www.costco.com.
• He sent me a link from his iPhone. I was at my desktop, and opened the link and got the mobile version.
Nov 2012 Better Software East 2012 46
24
Give a Choice to Avoid Making the Wrong Assumption
• Managing user expectations of how the applicationof how the application should behave needs to be thought out carefully.
• Context of the user and their expectations is key
• LinkedIn gives you a choice when accessing their site;when accessing their site; an optimized mobile experience via an mobile-app, or just their ordinary website.
Nov 2012 Better Software East 2012 47
Determine the Needs at Hand
• What need is the mobile application attempting to mobilize?
• How can the workflow on a mobile device be designed more efficiently to accomplish the task?
• How can the specific features and characteristics of a mobile device improve and complement the experience in contrast to the normal web-based papplication or other mobile platforms?
Nov 2012 Better Software East 2012 48
25
Determine the Needs at Hand• Fidelity implements top
scenarios their users want and optimizing efficiency for p g ythose user scenarios ONLY.
• Filling an order was a user scenario optimized just for the mobile platform.
• Note that it only has 4 data items to fill in with 2 beingitems to fill in, with 2 being scroll buttons, while the biggest button indicates precisely what the purpose of the task is.
Nov 2012 Better Software East 2012 49
Mobile Usability DesignBest Practices Summary
• Quick - Can you read the text without zooming or scrolling?scrolling?
• Simple Navigation-Task Oriented• Thumb Friendly• Visibility• Easy to Convert/Complete the task• Contextual
Design for the small screen. Don’t make users struggle to click tiny areas that are much smaller than their fingers.
• Seamless with main webapp• When you have a smaller screen, you must limit the
number of features to those that matter the most for the mobile use case.
50
26
What Is Designed, We Test
Understanding Design Leads to B tt U d t di E l tiBetter Understanding, Evaluation,
and Testing
What does want to achieve in the Usability UX
Standpoint?
52
27
Introduction to• iMedTablet - patient care coordination for
all types of long term care providersall types of long-term care providers. • Allows the user to securely access and
document patient care information in real-time utilizing Cloud technology, GPS technology and intra-facility gy ycommunications.
• Designed to seamlessly integrate with many pharmacy and long-term care software systems.
53
- Login Screen
54
28
-Med Admin Record
55
• Users are _________.• Heavily context oriented based on user expectations and habits.–Best design is not equal to the best usability.
29
Setting Up Usability Measurements
Measure and Improve
57
Big Picture on Usability and UX
User Experience
Usability Effect
Usability Design
58
30
Design-Test and Evaluate
Design
Test and Evaluate
Test and Evaluate
Release
59
Let’s Define UsabilityProduct (Design) Viewpoint
UsabilityUsability
Characteristic 1Characteristic 1 Characteristic 2Characteristic 2 Characteristic nCharacteristic n
Subcharacteristic 1Subcharacteristic 1 Subcharacteristic 2Subcharacteristic 2 Subcharacteristic nSubcharacteristic n
SimplenessSimplenessNavigationNavigation
Control StabilityControl Stability Button VisabilityButton Visability
# Buttons# ButtonsAttribute 1Attribute 1 Attribute 2Attribute 2PositionPosition Color UsageColor Usage
• Attributes expressed hierarchically• Any number of sub-levels is OK• Achieving the sub-attributes=achieving the high level attribute -measurable
31
Defining Usability from an Effect-Real usage Point of View
QualityQualityUsabilityyy
EffectivenessEffectiveness EfficiencyEfficiency Characteristic nCharacteristic n
AccuracyAccuracy CompletenessCompleteness Subcharacteristic nSubcharacteristic n
y
SatisfactionSatisfaction
ErrorsErrors Attribute 2Attribute 2 Attribute 3Attribute 3 Attribute 1Attribute 1
Potential Attributes Measurements for ‘Effect’ Usability
• Effectiveness– Completion rates– Completion rates– Error rate– Help usage
• Efficiency– Task time/Speed
• Learnability– Learning rate– Task time deviation– Change in task times over time period
32
Measurable Attributes
• Attribute name• Description and purpose• How to measure• What is measured• Measurement/Calculation
Need a model before you can start measuring…
• Range (min, max)• Objective• Current
Usability Measurement MethodsFocus groups
WalkThroughsS i f i
Logging
Heuristic Evaluation
Labs
ThroughsSatisfaction
Surveys
Let’s get started
Logging
33
Usability MeasurementAttribute Scale How Measure or
CalculationObjective Current
Help Access Percent ofUsers
Server Log files
% 30% 40%Users AccessingHelp
files
Task Simplicity
Keystrokes tofind/use a feature/function/information
Heuristic measurement
% 90% < 3 50%
65
ormation
Task Times Time finish‐time start
Server log files
% 90% < 3 50%
Accuracy Numbererrors
Server Log files
Integer or %
<5 10
Sample Heuristic EvaluationUsability Measure Elementary Partial1. Understandability 75%1.1 Label/icon recognizable 2 50%1.2 Function‐task recognizable 4 100%2. Navigation 63%2.1 Back‐Forth 3 75%2.2 Full‐mobile 2 50%3. Swiping 63%3.1 Consistency 3 75%3.2 User Awareness 2 50%4 Learnability 33%
Nov 2012 66
y4.1 Error prevention 1 33%4.2 Error detection 1 33%Total 58%
Through defining the model, then benchmarking with heuristic evaluation, we were able to determine weak areas and make recommendations for significant improvements
34
Usability LoggingMeasurement and Data Collection• Identify users by using session ID
to identify a unique user.to identify a unique user. • Iteratively insert code into the
application• Collect data• Analyze the data for each attribute
in different dimensions and aggregations
• Determine the need for further• Determine the need for further calculations and what attributes to measure further
• Revise the data we are collecting, adding or decreasing granularity
67
Satisfaction SurveysThe process:1 Calculate the usability score (satisfaction) of1. Calculate the usability score (satisfaction) of version X
2. Do the survey3. Change to version X.1 – make changes to the software directly correlated to the usability factors to either increase or decrease thefactors to either increase or decrease the usability score
4. Do the survey again 5. See if differences made change impact the survey results 68
35
Putting It All Together
Usability Design User ExperienceUsability in
Use
User Satisfaction
69
Getting Started with Measurement
• Produce an action planProduce an action plan– What usability attributes are important to your
organization?• Develop a model
– What data can you collect/Which technique can you usecan you use
• Maybe some elements of the model drop out-can’t be measured that easily
– Start collecting and developing benchmarks
70
36
Conclusion-Usability and UX • Paramount for today’s applications with users’ short
attention spans.• Designs should be as simple as possible for the user• Designs should be as simple as possible for the user. • UX is heavily influenced by expectations. • Make navigation easy on a small screen, thumb friendly,
and as intuitive as possible. • If the user cannot figure it out in 30 seconds, they are
gone. • Mobile-app versus your web-app - differentiate and
discern access from each platform. • Determine the features you really need and optimize
screens for certain workflows rather than trying to do it all.
71
Conclusion-Measurement• Defining UX is different for each organization• Need a model for your organizationy g
– What is most important to you depends on your users and their behavior and expectations
• The model is the foundation of what to measure, evaluate and improve
72
37
Thanks
Questions and Answers
www.xbosoft.com408-350-0508
Philip Lewphilip.lew@xbosoft.com
73
Recommended