Embed Size (px)
DESCRIPTION
User testing is the best tool on the planet for ensuring a successful mobile experience. Collecting feedback from your users doesn’t have to be expensive, time-consuming, or a danger to your deadline. On the contrary, making usability a part of your normal workflow will save you money, speed up your projects, and make your customers love you.
Citation preview
Mobile User Testing: Recruiting Subjects, Paper-Prototyping,
Testing Apps, Recording Video, and
More
Derek Olson, VP, Foraker Labs [email protected]
@dereklolson
(avoid being a total
jackass to your
customers)
Avoid being a total…
jackass …to your customers
Apple’s HIG
Read it, even if you’re contemplating non-iOS
platforms
https://developer.apple.com/library/ios/#documentation/UserExperience/Conceptual/MobileHIG/Introduction/Introduction.html
User Testing: Planning Your Test
• Roughly 90 minutes per session
• 4 sessions per day
• Allow 30 minutes between sessions
• Allow 4+ weeks for recruting
User Testing: What to Test… Paper or Plastic?
Paper Prototypes: Pro: Inexpensive, easy to tweak, elicits more negative feedback. Cons: Cannot test gestures, unrealistic flow, test participant guilt.
Beta apps: Pro: Able to test gestures, realistic app flow. Cons: More expensive, harder to make quick changes in response to feedback, cumbersome provisioning.
http://www.nngroup.com/reports/prototyping/video_stills.html http://www.uistencils.com/
User Testing: Creating Tasks
• Focus on common or important actions • You are testing what the user is capable of doing, not what they
would do • Don’t lead the participant • Mix up task language • Ask the participant to read task aloud from paper • When possible, motivate with personal connections or info • Establish clear success criteria • Counterbalance task order
User Testing: Recruiting Participants
• Usually between 5 and 10 • DIY versus hiring a recruiting firm • Incentive costs (usually $25-$200) • Demographic spread • Getting "real" users versus warm bodies • Mobile device experience • Sidewalk usability
User Testing: Directions and Contact Info
• Provide clear directions, maps, and address
• Provide phone numbers and email addresses for questions
• Consider things like parking, building security, etc.
User Testing: Testing Location, Waiting Area
• Allow for extra people
• Plan on early/late arrivals
• Have snacks and water
• Make sure there’s a restroom nearby
User Testing: Room Lighting, Temperature, and Noise
• Lamps are better than overhead lights • Not too bright • Chairs • Ensure a comfortable temperature • High traffic vs. low traffic • Wireless connectivity • Tissues?
User Testing: Permission Forms & Releases
• Permission to video
• General release
• 1099?
User Testing: Using a Script
• Testing software, not you • Think aloud • Conversational dynamic • Asking questions • Introduce any facilitators or observers • Coordination between multiple facilitators • Negative feedback is best • Potential for frustration, reiterate point #1
User Testing: Browser re-set / test data refresh
• Clear cookies
• Clear browser cache
• Clear stored form data and saved passwords
• May need to re-set test data
• Do it after each participant!
User Testing: Observers
• No talking, even for questions
• No texting or emailing
• No laughing
• No helping
• When to tell a white lie about identity
User Testing: Running the Session
• Practice on someone first! • Participant is king • Speak clearly, repeat as needed • Answer questions as you can • Probe for reasons when it’s safe (e.g. what did you expect to see there?) • Pay attention • Remind participant about thinking aloud • Take continuous notes? • Thoughts on when to provide help • More white lies about failures/confusion
User Testing: Debrief
• Answer remaining questions
• Try to probe for reasons
• Listen to stories, but not for too long!
• Incentive provides closure
User Testing: Recording Video and Audio
• Should you bother?
• Document camera
• Web cams
• Using another mobile device
• Project X
User Testing: Results Analysis & Write Up
• Shorter is better
• Documents vs. to-do lists
• “Greatest hits” video
• Quantitative metrics, pros and cons
From User Testing to User-Centered Design
• Many small studies are better than fewer large ones
• Baseline study and follow-up can show ROI • Keep a count of UX bugs squashed, it shows
measureable progress, and tends to incentivize leadership to “add to the list.”
• Bring in observers
Thanks! Derek Olson VP, Foraker Labs [email protected] @dereklolson (303) 449-0202 www.foraker.com
