How to Combine Quantitative Data with Qualitative Insights for CRO

Robert Mulder & Sven Cune

“The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.”

Stephen Hawking

User Feedback

showsyou why.

Analyticsshows

youwhat is

happening

User Feedback

showsyou why.

PERFECT FOR:

• Randomized site

experience surveys

• Content based surveys

• Shorter surveys with

fewer pages TYPES OF FEEDBACK

PASSIVE FEEDBACK

ACTIVE FEEDBACK

PERFECT FOR:

• Randomized site

experience surveys

• Content based surveys

• Shorter surveys with

fewer pages The humble feedback button

@usabilla

Avoid expensive assumptions

Key Objective:Improve booking process to increase conversion

Results:• Quickly uncovered what was really happening• reduced drop-off rates by 10%

UK AIRLINE

@usabilla

See what users want and detect issues early

Key Objective:Prioritise changes based on user input

Results:• Validated new design & reprioritised UX changes• Uncovered a serious bug quickly to save over £50K

UK RETAILER

@usabilla

Improve your reputation

Key Objective:Provide a continues channel for customer feedback

Results:• Increased App store rating

HUMAN

“An App store review should never be the easiest way to give feedback”

Paul Veugen, founder @ Human

Targeted Surveys

FRICTIONDEFINITION

The psychological resistance that your visitors experience when trying to complete an action. Friction is a conversion killer usually caused by unclear messaging, lack of information

FRICTIONDEFINITION

The psychological resistance that your visitors experience when trying to complete an action. Friction is a conversion killer usually caused by unclear messaging, lack of information

FRICTIONDEFINITION

The psychological resistance that your visitors experience when trying to complete an action. Friction is a conversion killer usually caused by unclear messaging, lack of information

17

18

19

20

21

WhileatHome

WhileattheOffice

Whilecommu0ng

Whileshopping(instore)

Whilebeingoutside

other

VoC enhanced testing

@usabilla

Feedback

Segment feedback based on variations

Make UX informed changes

Surveys

A/B testing

Target surveys at variations

@usabilla

Fake door tests

Quick Guide to Effective Surveys

Simple steps to getting actionable insight from your users

WHERE ARE YOUR USERS GETTING STUCK?

FRICTIONDEFINITION

The psychological resistance that your visitors experience when trying to complete an action. Friction is a conversion killer usually caused by unclear messaging, lack of information

SURVEY PLAN CHECKLIST

DEFINE

• Why you ask — Define your objectives

• Who you ask — Target very specific segments

• What to ask — Decide on friction areas and survey questions

• When to ask — Decide on behaviours that trigger the survey

• Where to ask — Target certain urls and pages of the website

Qualitative Surveys

KEEPING IT SIMPLE

• Keep it short - 3 questions per page maximum

• Keep it real - brand & humanize your language

• Keep it relevant - Conditional questions

31

32

WINNER!!

CONTROL VARIANT VARIANT B

–

No Introduction

15 seconds

–

With Introduction

15 seconds

WHICH VARIANT WON?

PHILIPS - SURVEY A/B TEST

6"

TEST$NUMBER$ B$VARIANT$ CHANGE$IN$RESPONDENTS$ COUNTRY$

1" A%er"30"seconds"" 49.2%$ China"

6a" No"Intro,"15"seconds" +131%$ US"6b" No"Intro,"15"seconds" +194%$ China"6c" No"Intro,"15"seconds" +113%$ Brazil"

2" A%er"4"Page"Views" 449.7%$ Netherlands"

3" Upon"Exit" 450.2%$ US"

4" A%er"5"Page"Views" 439%$ Netherlands"

5" No"Intro,"15"seconds" +60%$ France"

Industry Conversion

Travel 7-15%

Electronics 1-2%

Telecom 3-24%

Healthcare 5-11%

Transport 1-3%

Travel 2-3%

Publishing 6-14%

SURVEY CONVERSION PER INDUSTRY

• Why did you visit our website today?

• Did you find what you were looking for?

• What was your goal today and were you able to achieve it?

• Do you have any questions that you could not find the answer to?

• What was the biggest challenge in finding what you were looking for?

• How can we improve our X page?

• What made you not complete your purchase today?

• What almost prevented you form purchasing today?

MAGIC QUESTIONS

Thank you!Questions?

PS: Drop by the Usabilla stand to play our balls of fire game!