Diving Deep: Uncovering Hidden Insights Through User Interviews

PREPARED BY

UNCOVERING HIDDEN INSIGHTS THROUGH USER INTERVIEWS

Susan Mercer, Experience Research Director March 8, 2016

Place Client logo here

• BA and MSc in Geophysics

• 20 years in software and web UI and UX design

• Developer

• Designer

• Web Producer

• Product Manager

• Researcher

• MS Human Factors in Information Design, Bentley University

• Twitter: @susanAmercer

INTRODUCTION

Susan Mercer

2

HxRefactored.com

Healthcare & Design Conference

http://hxrefactored.com/


HxRefactored.com

Healthcare & Design Conference

• Workshops:

• Design Studio: Building Design Consensus Into Your Process

• Experience Strategy: Storytelling, Journey Mapping, and Service Design

• Designing Motivational Interventions for Behavior Change: A Self-Determination Theory Approach

• Human-Centered Design Bootcamp & User Shark Tank for Startups

• Closing Keynote:

• MA Governor Charlie Baker – Talking about the Opioid Crisis and how we can help tackle the problem



DIVING DEEP: UNCOVERING HIDDEN TRUTHS

Marketing Research & User Research

6

Marketing Research User Research

Preferences

Attitudes

Purchasing Behaviors

Use Behaviors

Context of Use

Actions

Usefulness

Emotional Reactions

Engagement

Motivations

To elicit honest stories in an unbiased manner.

INTERVIEWING BEST PRACTICES

What is our Interviewing Goal?

7

To elicit honest stories in an unbiased manner.

INTERVIEWING BEST PRACTICES

What is our Interviewing Goal?

8

Participant Researcher

A study found 60% of people told an average of 3 lies to a stranger in the first 10 minutes of meeting1

INTERVIEWING TECHNIQUES

It’s not easy being honest

91 Feldman, R.S., Forrest, J.A., and Happ, B. R. (2002) Basic and Applied Social Psychology, 24(2), 163-170.


Understatements, Exaggerations, Concealments

10http://popchartlab.tumblr.com/post/35563754790/the-map-of-truth-and-deception-a-visual

It’s not easy:

• We want to make ourselves sound better (Social Desirability bias)

• We want to please the researcher (Researcher Acceptance bias)

• We may not remember everything

• We may not be fully conscious of our own motivations for our behaviors


Eliciting Honest Stories

11


Maslow’s Hierarchy

12http://liveitloveitblogit.com/2012/04/maslows-hierarchy-of-needs/

1. Set the stage for the interview

2. Encourage them to tell their stories

3. Be neutral, accepting and non-judgmental

4. Help them uncover their answers


Belonging and Safety – Make the Interview Feel Safe and Accepted

13

An interview is a conversation…sort of

• Casual conversations have “unwritten” rules 1

• Speakers take turns

• It’s polite to balance speaking and listening

• It’s polite for both parties to ask questions

• An interview is a lop-sided conversation

• One party asks questions, the other answers them

• Explaining the process builds rapport & increases interviewee comfort2


1. Set the Stage for the Interview

141 Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.2 Vallano, J.P. and N.S. Compo (2015), Psychology Public Policy and Law, 21(1), 85-99.

• Provide a high-level focus for the interview

• “Today, I’d like to understand your experience visiting the museum.”

• “I’d like to talk about your experiences with understanding and using your health insurance plan.”

• Don’t reveal details, so you don’t bias their answers

• If they know what you want to learn, they may change their answers to please you

• This is called Researcher Acceptance Bias



151 Randall, D.M. and M.F. Fernandes, Journal of Business Ethics, 10(11), 805-817.

• Let them know you are neutral and need to hear positive and negative experiences

• “I don’t work for <company>, so feel free to tell me the good and the bad things.”

• “I didn’t design this website, so if you like it, you won’t make me feel good; if you don’t like it, it won’t hurt my feelings.”

• This helps reduce Researcher Acceptance Bias



16

• Explicitly acknowledge that you want to understand their personal experience

• “We are here to learn how everyday people do their shopping. Everyone is different, and we want to understand what works best for you.”

• “If I wanted to know how I’m supposed to use this injection pen, I would have read the instructions. I’m here to learn how you use it in your world.”

• This helps to reduce Social Desirability Bias



17

• Show interest

• Maintain eye contact

• Use open body language, face the interviewee and lean in slightly

• Really Listen

• “Listening is a positive act: you have to put yourself out to do it.” – David Hockney

• “Most people do not listen with the intent to understand; they listen with the intent to reply.” – Stephen R. Covey

• Listen with your whole body

• Give the interviewee your full attention



18

• Use conversational tokens

• “Mhmm” or “Uh huh” used in an interview situation

• Resulted in interviewees replies increasing by 31% 1

• Use body language

• Head nodding while the interviewee is speaking

• Resulted in interviewees replies increasing by 50% 2



191 Matarazzo, J.D, Saslow, G., Wiens, A.N., Weitman, M. & Allen, B.V. (1963), Psychotherapy 1(1), 54-63.2 Matarazzo, J.D., Wiens, A.N., Saslow, G., Allen, B.V. & Weitman, M. (1964), Psychotherapy 1(3) 109-113.

• Use silence

• “People speak in paragraphs.” – Steve Portigal

• People generally dislike silence

• If you pause, they will continue to speak

• Sometimes that extra information will be really valuable



201 Portigal, S. (2013) Interviewing Users, Rosenfeld Media.

• Ask non-leading open-ended questions

• “Who is involved with making decisions about health insurance?”

• “What is one thing you would change in your home?”

• “When was the last time you reviewed your financial portfolio?”

• “Where do you store your medications?”

• “How do you know when it’s time to renew your medications?”



211 Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.2 Vallano, J.P. and N.S. Compo (2015), Psychology Public Policy and Law, 21(1), 85-99.

• Use “soft” questions to avoid judgments

• Example: Why don’t HIV patients take their medications 100% of the time?

• There are subtle emotional undercurrents here, so tread lightly

• Ask about the topic from multiple angles:

• “When was the last time you missed taking a pill?”

• “What kept you from taking it?”

• “Where are you when you take your pills?”

• “What time(s) of day do you take them?”

• “What do you think about when you take your pills?”

• Once a participant shuts down, it’s hard to get them talking again



22

• Being neutral is important to minimize Researcher Acceptance Bias

• Avoid responses indicating:

• Agreement - “Good”, “Great”, “Perfect”, etc.

• Disagreement - “Hmmm.”, “Really?”, etc.

• Unexpected answers – “Oh!”, “Interesting”, etc.

• Use neutral responses:

• “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, etc.



23

• Being accepting and non-judgmental is key to building trust

• You asked them to share their story, feelings, motivations, etc.

• Their values may be different from yours, accept their truth.

• Sometimes, some self-revelation on sensitive topics can make them feel more comfortable 1

• “I understand that it’s hard to be perfect when losing weight. I struggle with that too.”

• “I appreciate you sharing your challenges with paying your bills this month. It’s something I’ve struggled with before too.”

• This can also help combat Social Desirability Bias



241 Vallano, J.P. and N.S. Compo (2015), Psychology Public Policy and Law, 21(1), 85-99.

• Be prepared for anything

• Opinions on gun control while discussing refilling prescriptions

• Strong opinions against evolution being taught in a science museum

• An nervous gay man explaining that searching for a doctor who is gay-friendly is an important feature on his health insurer’s website



25

• Sometimes we don’t remember things when asked

• Sometimes an honest answer is “I don’t know”, or “I can’t remember”

• Focus on a specific experience

• Focusing on a specific, recent experience can help people remember details

• This often provides more concrete information than generalities

• “Tell me about the last time you went grocery shopping – what was your biggest headache?”

• “Think about the last time you went to the dentist – what were you feeling when you walked into the office?”


4. Help Them Uncover Their Answers

26

• Leverage recognition over recall

• Neuroimaging shows that recognizing information requires less brain activity than recalling information 1

• Also, our brains store information in networks

• So, thinking about “leaves” can quickly remind you of trees, plants, flowers, daisies, and the color green.



271 Cabeza, R., et al. (1997), Journal of Cognitive Neuroscience, 9, 254-265.

• Use photos to prompt their memory

• Show a series of photos and ask the interviewee to set aside photos related to the topic

• Example: “Set aside any photos that remind you of taking your injections.”

• Have the interviewee tell the story of why they selected each photo



28



29

• Key Themes (Rheumatoid Arthritis patients)

• Enjoying physical activities again – less pain means I can kayak again

• Regaining a sense of independence – I can pump my own gas again

• Enjoying Family – I can do activities with my family again

• Key Themes (Multiple Sclerosis Patients)

• Concern about Future Health - How long do I have to be healthy?

• Uncertainty that Medicine is Working – It’s a crapshoot

• Enjoying Family – I want to enjoy my family activities as long as I can



30

• Use collaging to help them articulate their values and motivations

• Provide photos, art supplies, flipchart paper

• Provide a focus: “Create a collage about what saving energy means to you”

• Let them create it

• Have them tell the story of the collage



31



32

• Key Themes

• People feel saving energy is important for the environment

• Several said they didn’t know if they were saving enough to make a difference

• Several renters wanted to save energy, but didn’t get feedback information, so they gave up

• Many said they don’t make enough of an effort to save energy



33

• Use Laddering to get beyond superficial answers

• People often have difficulty expressing their values when asked

• People can often articulate attributes of products/services they like

• Laddering follows the chain from attributes to values with a series of “Why?” questions



34

Attribute

Consequence

Value

• What is the most important feature on this mobile app for ordering pizza?

• Why are deals and coupons important to you?

• What is it about saving money that is important to you?

• Why is it important to save for their college funds?


Laddering Interview Example

35

• Deals and Coupons

• Because I like to save money, and not have my wife yell at me

• We are trying to save as much as we can for the kids’ college funds

• Education is expensive, but important. I want my kids to have better jobs than I do

Interviewer Participant


Laddering Interview Example

36

• Deals and Coupons

• Because I like to save money, and not have my wife yell at me

• We are trying to save as much as we can for the kids’ college funds

• Education is expensive, but important. I want my kids to have better jobs than I do

Participant

Attributes

Consequences

Values

• Designs based on users’ values resonate better than those based on attributes

• We could make Deals and Coupons prominent

• But, what if…


Why use Laddering?

37

1. Set the stage for the interview



4. Help them uncover their answers


Belonging and Safety – Make the Interview Feel Safe and Accepted

38

• Avoid questions about future behavior

• People are really bad at predicting future behavior

• Ask about recent past events

• Stick to specifics and avoid generalities

• People tend to “filter” with generalities and leave out important details

• Focus on specific instances of past behavior

• Ask questions like “How many times have you forgotten your pills in the last 2 weeks?”

• Triangulate with other research methods

• Don’t use interviews and surveys alone (self-reported answers)

• Observe people in action – usability tests, ethnographic observations

• Use other data sources – web analytics, purchasing data, etc.

METHODOLOGY

Other Ways to Encourage Honest Answers

• Self-Reported answers are not 100% reliable

• People have difficulty explaining why they do things

• They sometimes provide incorrect answers because they don’t know why

• But you often get very insightful information

• Triangulate with other research methods

• Don’t use interviews and surveys alone (self-reported answers)

• Observe people in action – usability tests, ethnographic observations

• Use other data sources – web analytics, purchasing data, etc.

METHODOLOGY

Other Ways to Encourage Honest Answers

Thank YouSusan Mercer, Experience Research Director

[email protected]