Upload
darren-kall
View
1.954
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Every person in your company that comes in contact with your customers is a potential valuable source of customer insight, or they could be misinforming your business decisions with biased information without knowing it. As user experience thinking becomes part of corporate culture, everyone is getting on the “talk with the customer” bandwagon and they need to be aware of the pitfalls, biases, and risks to information gathering.In this talk Darren covers some of the common biases and effects that can impact customer interviews and touches on techniques to mitigate and avoid them from the Kall Consulting workshop on Customer Interviews. You don’t need to be a trained interviewer to reduce the biases in your customer contact. Benefits include simple techniques you can start using today. You can not be bias-free but you can manage the biases to increase the value of the customer insights you bring back. Audience: Any employee who has customer contact; marketing, sales, customer support, executives, designers, customer researchers, user researchers, etc.
Citation preview
UX Review1 Unbiased Customer Interviews
1© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
Unbiased Customer Interviews: What insight can we trust?
KALL Consultingcustomer and user experience design and strategy
full version: 16Sep2011
UX Review2 Unbiased Customer Interviews
2
What’s the problem?
1. Informed business decisions need accurate customer
insight.
2. Customer interviews for insight are different than other
interactions with customers.
3. People accidentally introduce biases in customer
interactions.
4. These biases taint the information they bring back.
5. Business decisions are made on this tainted information.
6. Tainted decisions increase business costs unnecessarily,
and result in lost revenue.© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review3 Unbiased Customer Interviews
3
Every customer interaction is not an interviewSelling Effect
Your approach can influence responsesThat’s not it! Effect
How information is gathered makes a differenceNumber BiasDemand Characteristics Effect
Volunteers Needed
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review4 Unbiased Customer Interviews
4
Customer Interview Workshop
Recording your Interviews
An Interview Coach
Interview Logistics
Biases in Interviewing
Picking an Interview Methodology
Question Design
Using Persona to Pre-categorize Interviewees
Data Analysis and Reporting
Learning to share: Customer insight database
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review5 Unbiased Customer Interviews
5
1. Selling Effect2. “That’s not it!” Effect3. Number Bias4. Demand
Characteristic Effect5. Scaling and Anchor
Biases6. Consistency Bias
7. Confirmation Bias8. Framing Effect Bias9. Encouragement Bias10. Correlation and
Causality Bias11. Ingrained Biases12. Status Quo Bias
12 Biases and Effects
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review6 Unbiased Customer Interviews
6
Not Remaining Neutral EffectScript Bias 1Script Bias 2White Lie EffectNot Listening EffectValue Bias
Recency EffectAnchor BiasImpact BiasNot Invented Here BiasBandwagon Effect
Extra Biases and Effects
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review7 Unbiased Customer Interviews
7
Ambiguity effectBlack swans (attention)Choice supportive biasClustering illusionContrast effectDenomination effectDetection biasEndowment effectFunding biasHalo effectHindsight effectHyperbolic discounting
Information biasLoss aversion biasMisunderstanding probabilityNegativity biasOmission biasOverconfidence effectReporting biasScope neglect biasUnit biasZero-risk biasEtc. The list goes on.
22 More Biases not Covered Today
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review8 Unbiased Customer Interviews
8
Bias or Effect Name
How the bias gets introduced and/or
what happens when it does.
What you should do instead.
Rationale for why people behave this
way or why you should avoid this bias.
The behavior, perception, or preference that
causes the problem.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review9 Unbiased Customer Interviews
9
Selling Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review10 Unbiased Customer Interviews
10
Selling Effect
Customers respond to pitch, not to the
interview. Your insight is limited.
Don’t sell. Save the selling for another
interaction.
Selling your product or your idea instead of
interviewing.
You’re not there to sell. You're trying to understand who the
customer is in order to get insight into their world.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review11 Unbiased Customer Interviews
11
“That’s not it!” Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review12 Unbiased Customer Interviews
12
“That’s not it!” Effect
Treating the customer this way gets them
defensive or shuts them down.
Correcting, demeaning, blaming, etc. the
customer.
Don't be corrective or pedantic. Reward the customer for
telling you things even if you disagree with them. Clarify
your understanding, capture the ideas, and move on.
The customer has their opinions and ideas. A customer
interview is not the place to be correcting them, even if
the idea is something the company will never do or that
you disagree with.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review13 Unbiased Customer Interviews
13
Number Bias
13© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review14 Unbiased Customer Interviews
14
Number Bias
Thinking and reporting of numbers is highly
influenced without awareness.
People avoid some numbers. They “round” numbers to
align with their base system. Different cultures have
different biases. If asked for a number, people will make
up answers.
Be aware that the numbers you get back are not absolute
but are shaped. They are not data.
Culture, base 10, 6, 8 or 60 numbering systems, whole
numbers over fractions, having 24 hours, 7 days, all frame
the number bias.
13
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review15 Unbiased Customer Interviews
15
Demand Characteristics Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review16 Unbiased Customer Interviews
16
Demand Characteristics Effect
You provide cues to your purpose and
interviewees give you a
“right” answer.
You don’t get a true answer, you get the biased answer you
“ask” for.
Remove demand characteristics, encourage open and
difficult responses.
Only by eliminating the expectation of a “right” answer can
you get to unbiased answers.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review17 Unbiased Customer Interviews
17
Scaling and Anchors Biases
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review18 Unbiased Customer Interviews
18
Scaling and Anchors Biases
Introduces biases in customer answers,
make interpretation impossible.
Making up scaling anchors, making up
scales, changing scales.
Get an expert to build your scales and set anchors. Don’t
make them up, change them, or try to be funny.
Inappropriate scales, changing them, etc. makes them
meaningless and impossible to compare.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review19 Unbiased Customer Interviews
19
Consistency Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review20 Unbiased Customer Interviews
20
Consistency Bias
Conclusions drawn won’t reflect the
reality of the customer’s variety.
Filtering out information accidentally
to make things consistent.
Extrapolating from one example to all
cases.
Report what you observe not what you want to see. Report
all that you observe.
Forcing consistency where it does not exist is biasing the
information.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review21 Unbiased Customer Interviews
21
Confirmation Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review22 Unbiased Customer Interviews
22
Confirmation Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review23 Unbiased Customer Interviews
23
Confirmation Bias
You only report supporting information
and you don’t SEE what you filtered
out with your bias.
Find only supportive information, interpret
or avoid contradictions to fit your idea.
Attempt to capture everything and make fair assessments.
Get someone else to review your interview recordings.
Confirmation bias is ingrained and hard to undo without
conscious attention and practice.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review24 Unbiased Customer Interviews
24
Framing Effect Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review25 Unbiased Customer Interviews
25
Framing Effect Bias
600 People with a fatal disease
Drug A saves 200 people's lives
Drug B has a 33% chance of saving all 600
people and a 66% possibility of saving no
one
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review26 Unbiased Customer Interviews
26
Framing Effect Bias
600 People with a fatal disease
If Drug C is taken, then 400 people die
If Drug D is taken, then there is a 33%
chance that no people will die and a 66%
probability that all 600 will die
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review27 Unbiased Customer Interviews
27
Framing Effect Bias
Framing a question or choice in a way that
manipulates its perceived value.
How a question or choice is presented biases the listener’s
answers.
Get an expert question designer to review your question
script. Explore different ways to present the question to
see if it influences the answer.
Framing effects work because of human nature preferring
gain over loss, certainty over doubt, positive over
negative.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review28 Unbiased Customer Interviews
28
Encouragement Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review29 Unbiased Customer Interviews
29
Encouragement Bias
Responses can be biased and shaped in
one direction or another.
Don’t reveal desired goal, don’t reward
positive or punish negative responses;
or vice versa. Remain consistent
customer to customer.
Encouraging a particular result, giving
“good answer” cues.
Encouragement sends information to the user as to the
desired outcome and they may respond without even
being aware.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review30 Unbiased Customer Interviews
30
Correlation and Causality Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review31 Unbiased Customer Interviews
31
Correlation and Causality Bias
When you ask interviewees for root
causes they will report correlations
which may not be the true root cause.
Assuming that because things occur
together, or close in time, that one causes
the other.
Correlations do not imply causality. Consider but don’t
blindly accept the rationale without triangulation.
General rules (heuristics) are built from experience and
applied incorrectly to all occasions.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review32 Unbiased Customer Interviews
32
Ingrained Biases
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review33 Unbiased Customer Interviews
33
Ingrained Biases
Both interviewers and interviewees
respond to each other filtered with
these biases.
Cultural, gender, national, geographic and
other backgrounds bias behavior,
preferences, and perceptions.
Mitigate your own ingrained biases as much as possible.
Counter balance interviewers. Use interviewers from the
same culture and country.
You can’t get away from your accent or other culture you
wear. You cannot control how others see you.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review34 Unbiased Customer Interviews
34
Status Quo Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review35 Unbiased Customer Interviews
35
Status Quo Bias
This biases the response away from new
things that will introduce change even
if they are better options giving them
false values.
Some people you will interview will have a
preference to keep things as they are
regardless of what you are proposing.
Understand the motivations and loss for this persona and
address them.
Study on rats, cheese, hunger and shock grid: status quo
tipping point.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review36 Unbiased Customer Interviews
36
Not Remaining Neutral EffectScript Bias 1Script Bias 2White Lie EffectNot Listening EffectValue Bias
Recency EffectAnchor BiasImpact BiasNot Invented Here BiasBandwagon Effect
Extra Biases and Effects
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review37 Unbiased Customer Interviews
37
Script Bias 1
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review38 Unbiased Customer Interviews
38
Script Bias 1
Skipping questions and significant
improvisation force biases in
analysis when combining
information.
Skipping questions in the script and
improvising question variations.
Hit all the questions. Don't improvise too much or you won't
have fair comparisons.
These are parts of normal conversation but interviews are
not normal conversations.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review39 Unbiased Customer Interviews
39
Script Bias 2
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review40 Unbiased Customer Interviews
40
Script Bias 2
You can miss gems of insight. You are
blind-sided by your focus/belief in your
survey and your bias limits the scope
of information.
Ignoring or cutting off customer attempts to
roam away from script.
Allow enough time to let the customer lead you “off track”.
Explore, evaluate, capture, then get them back on track.
The best stuff may be in what the customer knows that you
did not think to ask about.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review41 Unbiased Customer Interviews
41
White Lie Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review42 Unbiased Customer Interviews
42
White Lie Effect
Lies introduce biases in future
responses since based on
falsehoods.
Telling white lies for approval,
support, encouragement, etc..
Don't lie. Don't make false promises. Don’t confirm just
because they ask.
White lies seem like a kindness or an easy avoidance but
they have a devastating effect on the value of the
information. And you have to remember your lie.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review43 Unbiased Customer Interviews
43
Not Listening Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review44 Unbiased Customer Interviews
44
Not Listening Effect
This biases the customer against
you and changes their
responses.
Allowing interruptions such as
email, or thinking of next
appointment, to shift your focus
from the customer.
Stay focused. Take break if need one. Reschedule if needed.
Actively listen.
If you’re not listening, you’re missing things.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review45 Unbiased Customer Interviews
45
Value Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review46 Unbiased Customer Interviews
46
Value Bias
This blinds you to alternatives, gives
you false confidence in partial
information, & shuts down other
people.
Using customer information as a weapon.
It and your conclusions are complete
perfect answers.
Do interviews, bring back impressions, draw conclusions
and innovate ideas. But don’t treat them as data and
facts.
They are only impressions; treat them that way.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review47 Unbiased Customer Interviews
47
Not Remaining Neutral Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review48 Unbiased Customer Interviews
48
Not Remaining Neutral Effect
Giving feedback teaches the customer
too much about your views, and
biases their responses.
Show positive or negative reactions to
customer responses, opinions, or actions.
Remain neutral and indifferent to their answers, opinions,
etc. Remain calm and non-committal. Show no value
reactions.
It is human nature to give feedback but doing so taints
responses.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review49 Unbiased Customer Interviews
49
Recency Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review50 Unbiased Customer Interviews
50
Recency Effect
The most recent interview or last
comments will influence your
conclusions or reporting of
findings more.
Human memory biases toward the last item
in lists being remembered better.
Take notes on all interviews, review all materials when
doing analysis.
This is a core way human memory works along with the
Primacy Effect.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review51 Unbiased Customer Interviews
51
Anchor Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review52 Unbiased Customer Interviews
52
Anchor Bias
People are biased by this one aspect
skewing all their responses.
Interviewees focus on one aspect (speed,
color, size, etc.) and give it more importance
making it a reference or anchor for all other
comparisons.
Discover and dismiss the anchor “Ignoring color…”. Use
paired comparisons. Define the anchor instead.
First exposed or obvious aspects often become the
comparison for all other judgments.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review53 Unbiased Customer Interviews
53
Impact Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review54 Unbiased Customer Interviews
54
Impact Bias
The estimates you get back are biased to
overestimation in both directions.
When people make judgments of future
impacts they exaggerate both the positive
and negative aspects.
This is not data, it is perspective and opinion information
that you should note on questions where you asked for
judgments of future impact.
This is a natural tendency without awareness. People
believe the numbers they give you.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review55 Unbiased Customer Interviews
55
Not Invented Here Bias
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review56 Unbiased Customer Interviews
56
Not Invented Here Bias
Interviewees give false feedback on
ideas that are not theirs. Interviewers
miss better solutions that customers
may create.
Bias in interviewees and interviewers that
what we create is better than what someone
else creates.
Be aware of biases on both sides and keep consideration
open.
There are many reasons: fear of unknown, unwilling to
value others, lack of control, etc.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review57 Unbiased Customer Interviews
57
Bandwagon Effect
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review58 Unbiased Customer Interviews
58
Bandwagon Effect
Strong personalities can influence
people interviewed in groups.
Some people influence group opinion and
others adopt it.
Not always a bad thing. If you want unbiased opinions,
interview alone. If you want to generate conversation to
get more information, use a group.
Group dynamics influence, behaviors, perceptions and
preferences.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review59 Unbiased Customer Interviews
59
KALL Consulting Specialties
Product and Service Development and Improvement: Applying user experience techniques to the end-to-end customer and user experience design
Product Integration:Integrating the functionality of multiple products, combining several products into one product, and integrating acquired products into a product family
Merger and Acquisition User Experience: User experience evaluations in due diligence for buyers, user experience strategy as part of long term exit plan strategy for sellers, integration of products post M&A
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review60 Unbiased Customer Interviews
60
Caution
What I’ve told you about today can also be used for
eviLPlease don’t.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review61 Unbiased Customer Interviews
61
Thank You
Darren Kall• [email protected]• http://www.linkedin.com/in/darrenkall• @darrenkall• +1 (937) 648-4966
I’m glad to help your company become more
customer and user-centric.
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]
UX Review62 Unbiased Customer Interviews
62
Photo Creditsblood splat http://www.centrepatriotes.com
clock http://www.armyproperty.com/Resources/NSN-Listings/Clocks.htm
salesman http://www.expressionsofexcellence.com/GIFs/creative_selling.gif
buttons http://dragonlily.com.au/shop/popup_image.php?pID=154
script http://www.hearingreview.com/issues/articles/2003-10_02.asp
wrangler round up http://www.thebigquestions.com/wp-content/uploads/2010/04/roundup.jpg
liar http://www.mediabistro.com/fishbowldc/files/original/art_lie.jpg
scolding teacher http://2.bp.blogspot.com/_NjXJS2YMaYw/SyBiwnntCkI/AAAAAAAAAGY/hZjiRbF4TZg/s200/scold.gif
ear bud yeller http://quickbase.intuit.com/blog/wp-content/uploads/2010/08/92038203_5d8d68f920.jpg
knowledge is power cartoon http://llenrock.com/blog/wp-content/uploads/2009/12/knowledge-is-power.gif
Home Depot survey http://www.homedepotopinion.com/
neutral shirt http://www.zazzle.com/
Rosie riveter http://en.wikipedia.org/wiki/Rosie_the_Riveter
confirmation bias http://www.agenarisk.com/resources/probability_puzzles/confirmation_bias.shtml
matrix choices http://imo-networker.com/two-roads-to-take/
brick framed http://twistedphysics.typepad.com/.a/6a00d8341c9c1053ef0148c85e67e9970c-pi
brain http://www.internettime.com/wp-content/uploads/2009/12/brain-300x226.png
anchor http://www.boatweb.com.au/images/admiralty.gif
fear status quo http://writingcreativenonfiction.files.wordpress.com/2011/07/fear-of-change.jpg
bingo balls http://www.bingo-balls.com/
family circus cartoon http://www.oxfamblogs.org/fp2p/wp-content/uploads/causation-v-correlation-cartoon-246x300.gif
cultural bias faces http://news.brown.edu/node/10388?size=large
flying car http://www.metro.co.uk/weird/58822-flying-cars-are-here-at-last
bridge jumpers http://diyblogger.net/wp-content/uploads/2010/12/jumping_bridge.jpg
© Kall Consulting 2011 --- +1 937-648-4966 --- [email protected]