The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in researchUX New Zealand 2016Ruth Ellison, Principal User Researcher, PwC’s Experience Centre@RuthEllison

@RuthEllison

Hi:)

2

@RuthEllison

Free!

$10

$7

$20

@RuthEllison

$1

$10

$8

$20

@RuthEllison

The Zero Price Effect is a Cognitive Bias

@RuthEllison

What is a cognitive bias?Mental shortcut

@RuthEllison

You can buy a copy of this poster from: https://www.designhacks.co/products/cognitive-bias-

codex-poster

@RuthEllison 8

We are all humanPhoto available under a CC by 2.0 licence https://www.flickr.com/photos/yellowbookltd/14907364551/

@RuthEllison Image source: http://www.businessinsider.com.au/w hat-new-coke-looked-like-in-1985-2013- 3

@RuthEllison

Applying it to design research

PreparingRunning

Analysing

@RuthEllison

Preparing for research

@RuthEllison

@RuthEllison

Selection bias

@RuthEllison

Multiple channels - mix of recruitment companies, social media, trusted networks

Avoid professional respondents

Careful screeners

Behavioural based recruitment

@RuthEllison

Running research sessions

Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153

@RuthEllison

Observer-Expectancy effect

@RuthEllison 17

Source: https://medium.com/@userfocus/the-7-deadly-sins-of-user-r esearch-22857c5a971b#.xb38ukdtr

@RuthEllison

The way we frame our questions matter

@RuthEllison

Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 1974, 13, 585-589.

McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html

About how fast were the cars going when they smashed into each

other?

@RuthEllison

About how fast were the cars going when they collided into each other?

About how fast were the cars going when they bumped into each other?

About how fast were the cars going when they contacted into each other?

About how fast were the cars going when they smashed into each other?

@RuthEllison

Triangulate research

Use observational methods

Keeping positive-neutral body language, watch the tone of your voice

Avoid leading questions

@RuthEllison 22

I need a couple of volunteers please

23

248

What’s going on?

@RuthEllison

Confirmation bias

@RuthEllison 26

27

Take the opposing view

28

Don’t just confirm your hypothesis, see if you can prove it wrong

Analysis of Competing Hypotheses (ACH)

@RuthEllison

List assumptions

Be skeptical, especially if everyone agrees with you

Remain open

Consider all evidence equally

Multiple user researchers

Leave your ego by the door.

@RuthEllison from PwC’s Digital Services

@RuthEllison

Groupthink & the Bandwagon effect

@RuthEllison 31

@RuthEllison

Triangulate with other research methods (e.g. observational)

Use a mixture of individual and group exercises

Avoid stating preferences and expectations at start

Give someone the devil’s advocate role to question assumptions

@RuthEllison

Anchoring bias

@RuthEllison 34

35https://www.flickr.com/photos/naotakem/17105160150/

@RuthEllison

Consider the order of questions and designs carefully

Open ended questions

Alternate order in which participants are shown concept or design versions

Show version A first1 3 5

2 4 6 Show version B first

@RuthEllison

Analysing

@RuthEllison 38

@RuthEllison

Apophenia

39Source: https://au.pinterest.com/pin/249738741807276910/

@RuthEllison

Clustering illusion & reporting bias

@RuthEllison

Sample sizes – it’s about the WHY

Consider evidence equally – not just the ones that confirm your belief/assumption

Collaborative analysis sessions

Strive for objectivity

@RuthEllison 42

How vulnerable are you to these biases?

@RuthEllison

What percentage of the population will be more biased

than the average person?

@RuthEllison

Are you more biased than the average person?

@RuthEllison

Blind spot bias

@RuthEllison

Listen with an open mind

Become more rational, but less rationalising

Continuous learning

46

Always assess your method, your analysis and yourself for

bias

@RuthEllison 47

We are all human

@RuthEllison Image source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions _1954.JPG

Thank you:)

Say hello @RuthEllison

@RuthEllison

Further reading

List of cognitive biases

You are not so smart: a celebration of self delusion

9 Biases In Usability Testing

50