Upload
hendrikknoche
View
1.011
Download
1
Tags:
Embed Size (px)
Citation preview
EPFL, spring 2012 - week 12!UX evaluation
today’s schedule
➝ review of UX evaluation plans ➝ test run of scripts ➝ individual walk throughs
overview
➝ about UX evaluations ➝ UX evaluation criteria ➝ UX evaluation methods ➝ tips and hints ➝ for more information
✱Attribution: parts of this presentation are based on "Forum Nokia’s UX Evaluation series
why UX evaluation? ➝ UX is a critical success factor for products and services
➝ We need confirmation during design and development that we are on the right track
➝ It is challenging for the software developer to keep an objective view of the developed software ➝ It is human nature to become “blind” of your own
outcomes ➝ Thus it is important to get a second opinion of the UX,
from outside of the development team
➝ Preferably from real users trying it in a realistic environment
The designer is not the user
Success factor
Simulate real use situations
Functionality Reliability Efficiency Usability Portability Maintainability
Suitability Maturity Time behavior Operability Installability Testability
Accuracy Fault tolerance Resource utilization
Learnability Adaptability Stability
… … ... … … …
usability and product quality
➝ there are many metrics to measure software performance and quality
Code coverage
Source lines of code
Function point analysis
Number of sw bugs
Implemented functional
requirements
SW Product Quality (ISO-9126)
UX and product quality
User Experience
Expectations, motives, actions, interpretations,
…
➝ measuring UX is not straight-forward
➝ there are many intervening issues
➝ product quality has a strong impact on UX
In this definition, UX is not a part of product quality.
A SW engineering
point-of-view?
Functionality Reliability Efficiency Usability Portability Maintainability
Suitability Maturity Time behavior Operability Installability Testability
Accuracy Fault tolerance Resource utilization
Learnability Adaptability Stability
… … ... … … …
SW Product Quality (ISO-9126)
UX evaluation criteria 1/5
basic usability heuristic UX drivers derived from the UX elements
as such issues
detailed for your product
drivers tailored for your product
➝ Nielsen (1993) on usability ➝ learnability
➝ Novices’ ability to reach a reasonable level of performance rapidly
➝ efficiency ➝ Expert user’s level of performance
➝ errors ➝ Number of errors users make ➝ User’s ability to recover from
errors ➝ Existence of catastrophic errors?
➝ memorability ➝ Casual user’s ability to remember
how to use a system
➝ satisfaction ➝ User’s subjective assessment of
the system
useful ness
utility
relia bility
ease-of-use
effici ency
acces sibility
connec ting
people
identifi cation
pleasure stimulation
usabi lity
social value
enjoy ment
UX
UX evaluation criteria 2/5
➝ Evaluate the design against the specified UX artefacts
use case storyboard accessing phone from web browser
Melvin, 35!
• Engineer from Germany • Lives with his wife, 2 kids and a dog. • Both parents need to sometimes travel due their jobs, and kids have lots of hobbies. • Would like better manage his everyday schedules with family.
Jason, 25!
• Customer Service Assistant. • Keeps a blog for his friends and family Uses his device for capturing images to blog. Most of the images related to snowboarding. • Plays guitar in a wildly not-yet-so-popular band.
Louise, 27!
• Web designer, graduate student on university. • SMS addict - 50 messages every day. • Likes arts, especially indie movies and old films. • Is currently organizing surprise parties for her friend.
UX evaluation criteria 3/5
➝ evaluate against benchmark ➝ products and experiences
UX evaluation criteria 4/5
➝ Run your tests with all the target devices and form factors
Bar Slide
Clamshell Swivel
Touch screen Touch only screen
Full hw qwerty Full sw qwerty
Resistive screen Capacitative screen
UX evaluation criteria 5/5
➝ the ultimate criterion: ➝ potential users try the product in their mobile context
people places Things time
context
Direct, spoken criteria
Indirect, unspoken criteria - originating from the
context
planning the UX evaluations 1/2
➝ for any UX artifacts, you can think can and should it be evaluated, somehow
evaluate concepts and designs
A step-by-step process would
be easy to manage.
In practice, design and
evaluation are often in a parallel
and iterative relationship.
specify needs
and context of use define the UX
concept create UI designs
specify needs and context of
use define the UX
concept create UI designs
evaluate the UI designs
planning the UX evaluations 2/2
Evaluate concepts and designs
➝ What artefacts do we want to evaluate ➝ Sketches, visualisations, prototypes,...
➝ What issues are we evaluating? ➝ Concept, interaction, grahical design, ...
➝ Who will do the evaluation? ➝ Experts, end users,...
My cool travel- mate concept
UX evalution: expert evaluation ➝ Usability experts evaluate a system based on
common heuristics (i.e. design principles) and personal expertise ➝ Common heuristics are based on knowledge gained
through expertise and experience ➝ Knowledge about perceptional and cognitive processes
- such as the function of memory – are utilized ➝ At least two experts take part in the evaluation to
ensure reliable results ➝ At the beginning of an expert evaluation, the value of
the system to its user and buyer/provider is determined ➝ Based on these values, usability criteria are defined
➝ At the end, the findings documented and prioritized. Solutions proposals are made
➝ Also user tasks can be defined and walkthroughs conducted as part of the evaluation
UX evaluation: focus groups ➝ a group of people (4-7) that has
something in common will be invited to semi-structured group discussion session to share their views on certain topic. Lasts usually 2 hours
➝ face-to-face sessions ➝ Sometimes on-line
➝ variations of the method ➝ Exploratory ➝ Feature prioritization ➝ Competitive analysis ➝ Trend explanation
UX evaluation: usability testing ➝ the test user is asked to conduct certain tasks with the
user interface under design ➝ paper prototypes can be used ➝ users are asked to think aloud
➝ sometimes good to test the UI with pairs of users to
trigger more discussion or to test social aspects ➝ usability problems are identified and prioritized based on
the user data gathered during the usability tests
Moderator
Test user Test cases
Observer
Tested sw
Special equipment can be useful to catch the
actions on mobile devices
UX evaluation: observation ➝ the researcher does not interrupt the user
but just observes the user’s actions ➝ the user is followed usually shorter time
than when shadowing in her/his own environment, and usually only certain activities/time period are observed
➝ good for studying users that cannot be interrupted
➝ reveals task flows and possible detectable problems in them and behavioral patterns but not motivations, attitudes, values, concerns
If observation is not possible (e.g. in hospital) video
recording can be alternative. Video can be gone through
with the participant(s) afterwards
don't listen to users
➝ watch users attempting to perform tasks in the UI ➝ watch what people actually do ➝ be careful with what people say they do ➝ don't believe what people predict they may do in the
future.
when you listen
beware: ➝ users often tell you what they think you want to
hear ➝ often based only on recall ➝ rationalized behavior
UX evaluation: make a mix
➝ an expert evaluation and a usability test can be used as complementary methods
➝ an expert evaluation require less time and preparation than a usability test
➝ expert evaluations are recommended before usability tests in order to support designing the test tasks
➝ a satisfaction questionnaire can be combined with a usability test
UX evaluation: tips and hints ➝ the UX evaluation criteria should be in-line with
the defined UX drivers, targets and requirements ➝ make sure to share your UX evaluation criteria
with the designers before they start their work ➝ do evaluations with real users
➝ expert evaluations can provide quick useful findings ➝ try to have a end user mindset
➝ ideally your evaluation methods should scale to weekly (or even daily) evaluations ➝ design & evaluation iteration loops do not have to be
big monolithic steps
➝ UX evaluation(s) should be an integral part of software development process, just like software testing
design evaluate
iterate