UX London Collaborative Research Workshop

UX LONDON

Research Together!

1

Hello!

2

I have a question…

3

Do you enjoy being right?

4

You are correct!

5

YESSS!

6

p0wned!

7

No.

8

?

9

>? !

10

Flickr/Chris Voll

11

Ego!

12

13

14

Where are you?

15

What We’re Doing Today Research & Collaboration

A Framework for Research Questions & Activities Biases & Objections

Understanding the Organization

Break

User Research Analysis

Models and Reports Getting Buy-In at Every Stage

Wrap Up

16

Research & Collaboration

17

18

19

20

21

22

23

People

24

25

Collaboration!

26

Collaboration!

27

28

True Collaboration

29

Chris Noessel

Collaboration requires behavior change

30

For Effective Collaboration

Have a plan Provide a rationale Define roles and responsibility Set expectations Communicate progress Reflect on performance

31

The 4 Virtues of Collaboration

Clarity & definition Accountability & ownership Awareness & respect Openness & honesty

32

Embrace Conflict

33

Why research?

34

Real World

Context

Organization Users

35

Things you

don’t control

Goals, assumptions &

resources

Needs & behaviors

36

Personal View

Personal View

Personal View

37

Shared Reality

38

Shared Understanding

39

A design project is a series of decisions.

40

What is

What ought to be

41

Research leads to evidence-based

decisions.

42

Data (alone) doesn’t change minds.

43

Objections!

44

6We don’t have the time.

45

$We don’t have the money.

46

$We don’t have the expertise.

47

We have a guy.

48

0

25

50

75

100

April May June July

We’ll just A/B test.

We have analytics.

49

!Let’s just make a prototype.

50

Steve Jobs

51

Any others?

52

Everyone wants better products, faster.

53

No one wants to read a report.

54

Research Advantages

55

¿Faster decisions

56

CLower risk

57

Saves money$58

JIncreased value

59

Continuing returnsC60

Everyone is more effective and

efficient

61

62

How to Do Research

63

Dogma

64

Design-Led

Research-Led

Expert Mindset

Participatory Mindset

Users seen as subjects Users seen as partners

Design-led with

expert mindset

Design-led with

participatory mindset

Research-led with expert

mindset

Research-led with participatory

mindset

Dubberly Design Office

65

Goal Driven

Skeptical Mindset

Increase chance of success Reduce risk

Willing to question the value of any approach

66

One Simple Process

67

Form Questions Analyze Data

Gather Data

68

Form Questions Analyze DataThink Critically

69

Form Questions Analyze DataObserve

70

Form Questions Analyze DataInterview

71

Form Questions Analyze DataRead

72


Read

Experiment

Interview

Observe

Think

73

Your Process Success!!!!Insights

74

You need a plan!

75

76


Gather Data

77

Questions determine results.

78

Questions give research meaning.

79

Research high-priority questions.

80

Good Questions

Specific Actionable Practical

81

A Bad Question

“How do we get Millennials to like us?”

82

Better Question

“How do recent college graduates living in cities decide what to have for dinner?”

83

A Bad Question

“What do people do around here all day?”

84

A Better Question

“How do editors and designers work together?”

85

The Best Question

The unknown that carries the most risk.

86

Project Risks

•Target customers don’t value product

•Customers need something else

•Business model doesn’t support it

•Organization can’t produce it

•Someone else is doing it better

87

Practice!!88

Practice Exercise: Writing questions Time:10 minutes

• I will give you a project scenario • Discuss potential research questions • Write down the 3–5 highest priority

89

What research questions might serve this project?

A e-commerce startup wants to create an app to

help people give gifts.

90

Time is up!691

Let’s talk about your questions.

92

How to answer those questions

93


Gather Data

94

Research activities are simply ways to answer questions.

95

Questions About

Users

ProductOrg

Competition

InterviewsInterviews

Usability Testing

A/B Testing

Contextual Inquiry

Literature Review

SWOT Analysis

Brand Audit

Usability Testing

Competitive Analysis

Heuristic Analysis

Descriptive

Evaluative

Evaluative

Evaluative

Analytic

Analytic

Generative

Descriptive

96

Research Activity

Topic Purpose

Time

Money

97

Phone Interviews

What do we need to know

about?What kind of

decision will it inform?

How long do we have?

What is our budget?

In-Person Interviews Contextual

Inquiry

Usability Testing

Competitive Analysis

98

Why not just make a prototype?

99

100

“If we only test bottle openers, we may never realize customers prefer screw-top bottles.”

– Victor Lombardi, Why We Fail

101

There is no one right method or activity.

102

How to get the most out of any method •Clarify goals •Enumerate assumptions •Identify questions •Prioritize questions •Work collaboratively •Present strategically

103

Don’t focus on the method

104

Tell the story

105

Critical Thinking

106

Critical Thinking

•Disciplined •Self-correcting •Clear •Logical

107

Uncritical Thinking

“I hate yellow, so a yellow website won’t succeed.”

108

Critical Thinking

“I hate yellow, but based on the evidence, it might work for our audience.”

109

Uncritical Thinking

“The information on this website is too dumbed-down for me.”

110

Critical Thinking

“Our target audience needs clear, simple information.”

111

Critical Thinking

“I don’t know.”

112

Bias

113

Bias: Something that causes an influence or

prejudice

114

Confirmation Bias: You selectively weight the information that confirms what you already believe.

115

Sampling Bias: Your sample of research subjects isn’t sufficiently

representative.

116

Interviewer Bias: You insert your opinion

into interviews.

117

Social Desirability Bias People don’t say the true things that they worry will

make them look bad.

118

Hawthorne Effect Observation changes the behavior being observed.

119

Ease

Clear Display

Related Experience

Primed Idea

Good Mood

Feels True

Feels Familiar

Feels Good

Feels Effortless

Daniel Kahneman

120

Feeling confident? It’s not a good sign.

121

You might have a bad case of Dunning-Kruger.

122

Research Topics

123

Organizational Research

124

Real World

Context

Organization Users

125

All organizations have baggage

126

Organizational research helps you understand

•Requirements •Politics •Workflow •Capabilities •Goodwill

127

Requirements

What are the top business priorities for this project/product?

128

Politics

What does success mean for our individual roles?

129

Workflow

(How) do we have to change how we work together to be successful?

130

Capabilities

What are the strengths and weaknesses of our team?

131

Capabilities

Where is the internal expertise?

132

Goodwill

How might product decisions make someone’s job easier (or harder)?

133

Stakeholders

134

How to extract knowledge from these people?

135

Get them alone.

136

Butter them up.

137

Basic Stakeholder Questions What is your title? How long have you been in this role?

What are your essential duties and responsibilities?

What does a typical day look like?

Who are the people you work most closely with?

How is that going?

138

Project Specific Questions What does success mean from your perspective?

What will have changed for the better once this project is complete?

Do you have any concerns about this project?

What do you think the greatest challenges to success are?

Internal and external?

139

Stakeholder Power Moves

“Why are you asking me this?”

“I don’t understand that question. It doesn’t make any sense.”

“I don’t feel comfortable talking to you about that.”

“No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.”

“How much more time is this going to take?”

140

Practice!!141

Practice Exercise: Stakeholder interviews Time:10 minutes x 2

• Find a partner. This is a pair exercise. • We will show you a set of questions • Interview your partner • Really listen • No need to take notes

142

Stakeholder Questions

•What is your title? How long have you had this job?

•What are your essential duties and responsibilities?

•What is a typical day like?

•Who are the people you work most closely with?

•How is that going?

•What do you think the greatest challenges to your success are?

•Internal and external?

143

Switch!∞144

Stakeholder Questions

•What is your title? How long have you had this job?

•What are your essential duties and responsibilities?

•What is a typical day like?

•Who are the people you work most closely with?

•How is that going?

•What do you think the greatest challenges to your success are?

•Internal and external?

145

Time is up!6146

What did you learn?

147

Empathy

148

In summary

• A shared reality is as important as “new” facts. • Research is a simple process. • Be goal-oriented, not dogmatic. • Questions threaten authority. • Facts alone won’t convince those who feel threatened. • Use what you learn to fit your findings into the project story.

149

User Research

150

Real World

Context

UsersOrganization

151

Photo: Flickr/theloushe

Ethnography

152

How to do bad user research: Ask people what they want.

153

How to do bad user research: Ask people what they like.

154

Never ask users what they want or like.

155

The Four Ds of Design

Ethnography

156

Deep Dive Daily Life

Data Analysis Drama

157

“...true ethnography reveals not just what people say they do, but what they actually do.”

–PARC

158

159

Photo: Flickr/lintmachine

160

The Art of The Interview

161

Interviewing is not talking.

162

Interviewing is listening.

163

Good Interviewers 1.Know Your Question 2.Warm Up 3.Shut Up

164

Interview Structure

1 Intro

2 Body

3 Conclusion

165

Introduction:

Smile

Express gratitude

Describe the process

Ask to record

Warm up questions

Body:

Ask open-ended questions

Probe for more

Allow silence

Use questions as checklist

Conclusion:

Transition to wrap-up

Ask if there is anything else

Thank for time

Interview Structure

166

Introduction:

Smile

Express gratitude


Ask to record

Warm up questions

Body:


Probe for more

Allow silence


Conclusion:



Thank for time

Interview Structure

167

Introduction:

Smile

Express gratitude


Ask to record

Warm up questions

Body:


Probe for more

Allow silence


Conclusion:



Thank for time

Interview Structure

168

You are the host You are the student

169

Out of your comfort zone,

and into theirs.

170

Interview Checklist • Create a welcoming atmosphere.

• Always listen more than you speak.

• Take responsibility to accurately convey the thoughts and behaviors of the people you are studying.

• Start each interview with a general description of the goal, but be careful of focusing responses too narrowly.

• Avoid leading questions and closed yes/no questions. Ask follow-up questions.

• Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it.

• Also note the exact phrases and vocabulary.

171

Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment

Make a note of

172

RolesInterviewer Notetaker Observer Participant

173

Practice!!174

Exercise: User research interview Time: 15 min x 3

•We will give you a research scenario and an interview script

•In your groups, assign roles of participant, interviewer, note taker, and observer (optional)

•Conduct the interview in 15 min, then switch roles. •Hang on to your notes. We’ll need them.

175

Interview Scenario

You are working on a new service to help people give gifts.

The goal of the research is to identify unmet needs people might have with regard to giving gifts.

176

Listen for:


177

!Ready? Go!

178

Switch!∞179

Remember: This is a research interview, not a friendly conversation.

Listen for:


180

Switch!∞181

Clear your mind.

Listen for:


182

Time is up!6183

How did that go?

184

How about a focus group?

185

186

“Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, invented focus groups

187

Everybody Lies

188

Anything else about activities?

189

Analysis

190

Analysis turns data into useful insights

191

192

Analysis isn’t complicated

Compile data Look for patterns

Identify insights

Create models

…but it is challenging

193

You need to create meaning from data

194

Data > Meaning > Actionable Inputs

195

Observation Observation

Observation


Observation


Observation

196

Observations are

Verbatim quotes Reported behaviors

Observed behavior

197


Observation


Observation


Observation

Collaborates on purchases

Uses several devices Needs affirmation

198

Make product information sharable

Save state in purchase process

Users collaborate on purchases

Insight Mandate/Action

199

Ground rules The goal of this exercise is to better understand the context and needs of the user.

Wait to identify larger patterns until you’ve gone through the data.

Clearly differentiate observations from interpretations (what happened versus what it means).

No specific solutions until after you’ve gone through insights and principles. Solutions come next.

200

Practice!!201

Practice Exercise: Analysis Part 1 Time: 15 minutes.

1.Gather all notes 2.Scan notes for interesting observations 3.Write observations on notes 4.Put the notes in a pile

202

Look for quotes and observations that indicate

Goals (what the

participant wants to

accomplish)

Priorities (what is most important to

the participant)

Tasks (actions the participant

takes to meet their goal)

Barriers (what prevents accomplishing

the goal)

Motivators (the situation or

event that starts down the

task path)

Habits (things the participant does on a

regular basis)

Relationships (who the

participant interacts with

doing the tasks)

203

You have 15 minutes.6

204

Get ready to finish!6

205

Time is up!6206

207

Practice Exercise: Analysis Part 2 Time: 15 minutes.

1.Put notes on the board 2.Group notes into patterns 3.Label the group 4.Negotiate and advocate for your perspective

208

1

23

209

Do this part last

210

You have 15 minutes.6

211

Get ready to finish!6

212

Time is up!6213

How did that go?

214

Creating a Model

215

Personal View

Personal View

Personal View

216

Shared Reality

217

Thinking is not useful until shared

218

A model is thinking made visible

219

Some examples

220

221

222

223

224

225

226

227

228

229

230

In summary •A model simplifies and clarifies complex ideas.

•Thinking isn’t useful until it’s out where people can see it.

• An effective model makes it easy to incorporate new information.

•A model diagram is a tool, not an end in itself.

•Avoid the temptation to make diagrams more pretty than useful.

•An effective model communicates without needing an audio guide.

231

Personas

232

A persona is “the user” made visible

233

Busy mom is the ur-persona.

234

I’ve never seen a persona called “Married woman, no kids, with pristine hardwood.” God, how I aspire to see that persona.

-Steve Portigal

235

They can still be useful

tools

236

Personas

• Distill ethnographic research • Document representative groups of needs and

behaviors as archetypes • Represent relationships among user types • Allow team to advocate for user needs • Act as a reference point for decision-making • Maintain empathy throughout design process

237

Personas must be based on actual research data

238

Fun, but not useful

239

A good model represents and simplifies knowledge

240

Practice!!241

MAKE A SIMPLE JOURNEY

Experience Journey Map | Task: Eating Lunch

242

Practice Exercise: User Journey Time: 15 minutes.

1.As a group, review your user interview notes to identify steps in gift purchase journey

2.Decide whether that step is positive or negative 3.Decide how much of a positive or negative 4.Fill in diagram

243

Time is up!6244

How did that go?

245

Reporting

246

2A useful report supports

Clear goals Shared values Access to information Confident decisions

247

2You decide the purpose

Informing? Inspiring? Focusing? Remembering? Recording? Deciding?

248

Research ReportStudy Title

Date Completed

Research Goal

Activities

Related Decisions

Key Insights

Supporting Observations

Recommended Actions

Questions for Further Study

Keep it as brief as you can.

249

Building a research-driven

culture250

Capture the value of research

251

Use your report to tell a story

252

How most people do itMethods

(what we did, usually in vast

detail)

Findings (what we found,

often disconnected

from biz goals)

Meaning (the

implications for our design

work)

Framing (How it

connects to the project story…

maybe)

zzzz

253

How you shouldMethods (a brief

summary up front, most as an appendix)

Findings (what we found, leading with the interesting bits)

Meaning (the

implications for your business)

Framing (Setting the stage with

context and a good story)

$$

$

254

Finish line!

255

In summary • Research creates a shared understanding of reality.

• Asking questions is uncomfortable. Embrace that feeling.

• A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun.

• Clear goals and good questions are required.

• Choose only the research activities that answer real questions and inform your top priority design and development decisions.

• Practice! Observe and listen every day.

• Document! Report! Share! It’s easy to lose what you learn.

256

Any questions?

257

Thank you!

258