Top 10 User Research Pitfalls

Dan Arra Managing Director

2

PART 1 About the Speaker PART 2 What is Research? PART 3 10 Research Pitfalls

Agenda

3

ABOUT THE SPEAKER

About The Speaker: Dan Arra

•  Began career as an engineer in the aerospace industry.

•  Spent 10 years at software companies in systems engineering, product management and professional services.

•  Started software & technology companies and was responsible for leading numerous successful projects. Another 10 years.

•  Joined Macadamian, a software design & development studio. Work closely with design and software engineering teams to improve and create new products.

4

5

WHAT IS CUSTOMER RESEARCH?

User-centered Design

6

Good design is built on a solid understanding of your audience: specific goals, tasks and context of use.

UCD: understanding users and tasks

Confidential 3/22/13 7

Research Methods: Qualitative versus Quantitative

QUALITATIVE QUANTITATIVE attitude about

your product and competition

Focus groups Interviews

Surveys (online, telephone)

Behavior related to your

product offering

Ethnographic research (Observation, shadowing)

A/B Testing

usability of product

Usability testing

Usability testing

Confidential 3/22/13 8

9

10 RESEARCH PITFALLS

The most dangerous things about research!

10

Myth: Being Smart means you can do good UX Research & Design. Anybody can ask a question… but relevant, appropriate, constructive?

Design research IS real science!

11

Sci-ence [sahy-uh ns] noun •  systematic knowledge of the physical or material world gained through

observation and experimentation.

Reality!

12

Observe Think Design Test

! Rinse, repeat.

Our users don’t tell us what they need

13

Myth: We asked users what they wanted, so we’re covered.

Users are NOT designers

•  Users are focused on solutions •  Don’t consider context and other mitigating factors

•  Why? Why? Why?

14

Avoid jumping to Solutions too quickly

15

Staying in problem space leads to INNOVATION

16

Go to: http://www.macadamian.com/ for full story.

Staying in problem space leads to INNOVATION

17

Go to: http://www.macadamian.com/ for full story.

Staying in problem space leads to INNOVATION

18

Go to: http://www.macadamian.com/ for full story.

Staying in problem space leads to INNOVATION

19

Go to: http://www.macadamian.com/ for full story.

Talking to customers " Research

20

Who to ask... AND How to “ask”.

Consider Triangulating Research !

21

Stating, without knowing! “We must have this feature”! can be very to EXPENSIVE

22

Go to: http://www.macadamian.com/ for full story.

Numbers can be deceiving!

23

Sometimes numbers are not appropriate!!

24

Qualitative Quantitative Attitude

(Preference) •  Focus Groups •  Interviews

•  Surveys •  A/B Testing

Behavior (Performance)

•  Usability Walkthrough •  Ethnographic Research •  Job Shadowing

•  Usability Test •  Benchmark Test

Focus Groups & Usability Testing!NOT the same!

25

What people say, what people do, and what they say they do are entirely

different things.

Photo credit: http://www.webusability.co.uk/what-we-do/mobile-testing/

Tale of Two methods

26

Usability Testing Focus Groups Method One-on-one testing; observing representative

samples of end users using the product and measuring their performance on pre-determined tasks (n= # of individuals being tested)

Group discussion/exploration with up to 10 participants per group (n=1)

When to use Throughout development phase and during deployment

Pre-development; early in program verification

Key focus What will make the product easy for end users to learn and use?

What will motivate potential customers to buy the product? Determining market need.

Key motivation

How well can end users use the functionality a product provides? Can they realize the value promised?

What are the customers' wants and desires ?

Eight is sometimes enough

27

It just depends

28

Faulkner, Laura (2003) Beyond the Five-user Assumption: Benefits of Increased Sample

sizes in Usability Testing Behavior Research Methods, Instruments and Computers Volume

35 (3) 379-383

Raw data is NOT enough

29

Translate data into meaningful requirements

30

Don’t rely only on the Spec!

31

More than words can say!

32

Not just anyone can do research

33

UX Researcher – Prevents “heard what I wanted to hear.”

•  Consider bringing someone on board with experience, a Degree in Human factors, HCI and/or soft sciences (e.g., Psychology, Sociology).

•  If conducting true user research, you’ll want:

−  someone with knowledge and experience in research methods and/or experimental practices.

−  someone with the expertise to turn insights into design direction

•  Partner user researcher with SMEs, business analysts, and product managers for maximum success!

34

A Good Researcher!

!  can identify the best methods to get the right information !  is “familiar enough” with the subject material to determine the

right questions !  avoids leading questions !  has strong listening skills !  follows the the natural flow of the conversation !  can steer the conversation in the case of digression and guide

user to discuss new topics when one is exhausted !  remains neutral and puts the interviewee at ease !  focuses on the most recent experiences !  looks for ways to visualize and summarize results to be

actionable

35

36

Ask the right questions! Use the right method(s) Analyze appropriately Use the right people!

Remember

37

Improving AND growing business. Developing relationships = gold. Establishing credibility = save time. Learning/refining your craft.

Remember, also that you are!

Thank-You

darra@macadamian.com For more information, visit: www.macadamian.com Follow Macadamian on Twitter: @macadamianlabs

38

Confidential 3/23/13 39

”Innumeracy: Mathematical Illiteracy and Its Consequences”, by John Allen Paulos “Ambient Findability: What We Find Changes Who We Become”, by Peter Morville “Communicating Design: Developing Web Site Documentation for Design and Planning,” by Dan Brown The Design of Everyday Things Don Norman Designing Interfaces Jennifer Tidwell Do You Matter? How Great Design Will Make People Love Your Company Robert Brunner “Don't Make Me Think: A Common Sense Approach to Web Usability,” by Steve Krug

Design and Research Resources

Confidential 3/22/13 40

“The Elements of User Experience: User-Centered Design for the Web,” by Jesse James Garrett Emotional Design Don Norman Mental Models: Aligning Design Strategy With Human Behavior by Indi Young “Observing the User Experience: A Practitioner’s Guide to User Research,” by Mike Kuniavsky “Presentation Zen Garr Reynolds Sketching User Experiences: Getting the Design Right and the Right Design,” by Bill Buxton Universal Principles of Design William Lidwell “Visual Explanations: Images and Quantities, Evidence and Narrative,” by Edward Tufte

Design and Research Resources

Confidential 3/22/13 41

We create compelling user experiences for: •  Consumer and enterprise desktop applications

•  Interfaces embedded into physical products •  Mobile interfaces and applications on multiple platforms

•  Web applications and cloud-based applications

About Macadamian

Design + Development!

42 Confidential 3/22/13

43

Ottawa!

San Mateo!Yerevan!

Cluj, Romania!

Confidential 3/22/13

•  Over 200 employees!•  15 + years in business!•  Integrated design & development teams!

44

What We Do: UX services

•  UX Research −  Stakeholder Interviews −  Product Strategy Workshops −  Task Analysis −  On-line Surveys −  Usability Testing

•  Software Development −  Stand alone or combined with design −  PoC or market ready −  Technologies: Web, Cloud, Mobile,

Desktop −  Agile methodology −  Integrated with customer teams

•  UX Design −  Information Architecture −  Content Modeling −  Interaction Design −  Visual Design

•  Software Quality Assurance-QA −  QA Review: performance or process −  QA/QC for customer lead development −  QA/QC within our agile process −  Localization QA −  QA Script/Automation