The Secret to Successful Survey Projects

Better Decision Making Fueled by Data AMA Iowa NOV 2013

2

Hi, I’m Brent

• VP, GM of SurveyMonkey Audience

• @SurveyMonkey since Apr. ’09

@bchudoba brentchudoba.com in/brentchudoba

Why do marketers LOVE survey data?

3

To try and understand the public's way of thinking

Often proves a point or opens up a new market

Able to quickly provide feedback to industry decision makers to demonstrate we stay informed and keep them informed

Concrete data to tell us how our consumers think/feel

Narrows down a wide variety of a group's opinions, habits, and preferences to help give you better direction in making marketing decisions and taking action

I like statistics and analytics so when I see numbers, it helps me to make, what I think, are better informed decisions

I love the research and statistics that come out of surveys

It’s fact. You can make decisions

Lets us know what the customer wants, otherwise we are making decisions based off of stereotypes, gut reactions, or the past

SurveyMonkey mission: Help people make better decisions

4

So let’s talk about how we can help

5

Question: What is the most challenging aspect of running a survey project?

6

Let’s see what marketers said…

7

The challenges

8

The challenges

9

•  Sign off takes much longer than expected

The challenges

10

•  Sign off takes much longer than expected

•  Projects can be costly

The challenges

11

•  Sign off takes much longer than expected

•  Projects can be costly • Creating questions that

will produce reliable data is challenging

The challenges

12

•  Sign off takes much longer than expected

•  Projects can be costly • Creating questions that

will produce reliable data is challenging

•  Takes weeks (sometimes months) to get data back

The Secret

13

The Secret: Write your conclusions BEFORE you create your survey

14

Write your conclusions first?"

15"

• How do you write a conclusion without any data?"

Write your conclusions first?

16

• How do you write a conclusion without any data?

• Couldn’t writing conclusions before questions create bias?

A survey is a research project

17

Think back to the the “scientific method”

Scientific method

18

• Make an observation • Propose a hypothesis • Design an experiment • Test hypothesis • Accept or reject the hypothesis • Draw conclusions and/or revise hypothesis • Rinse and repeat…

What usually happens during a meeting…

19

1.  Business question is identified"

What usually happens during a meeting…

20

1.  Business question is identified"

2.  Someone says, “We should do a survey”"

What usually happens during a meeting…

21

1.  Business question is identified"

2.  Someone says, “We should do a survey”"

3.  Project owner is assigned"

What usually happens during a meeting…

22

1.  Business question is identified"

2.  Someone says, “We should do a survey”"

3.  Project owner is assigned"

4.  People start writing survey questions"

Now, what SHOULD happen after a meeting…

23

Observation Identified a trend for study

Hypothesis Outline a draft of the “presentation” (e.g., blog post, research report, memo, PowerPoint deck) and include hypotheses

Design Design a survey with questions that ONLY provides the data required to to prove or disprove they hypotheses

Test Find respondents and field survey

Accept/Reject

hypothesis Review findings, update draft presentation and determine if additional research

is required

Revise/Accept

hypothesis Make an informed business decision

If you start with a hypothesis…

24

• Your survey projects will be significantly faster

If you start with a hypothesis…

25

• You will have powerful data in your hands to answer business questions sooner

If you start with a hypothesis…

26

• You will have more time to iterate and refine your projects

If you start with a hypothesis…

27

•  Your data analysis stage will be simpler -  Succinct surveys = fewer

data points -  Questions designed with

analysis and presentation in mind provide immediate insights and inherently more effective question structures

If you start with a hypothesis…

28

• The post survey presentation creation stage will be lightning fast since you’ve already built your framework

If you start with a hypothesis…

29

•  You may be sought out by colleagues and superiors who want your assistance solving challenging business problems -  Promotions, increased social

standing, fame and fortune have been observed

-  Disclaimer: Results may vary, and SurveyMonkey does not guarantee any of the above results

Let’s give it a try…

30

Netflix

31

•  In early 2013, Netflix debuted ‘House of Cards” as a blockbuster original series

• How would subscribers react? Would it start a new chapter in content creation and distribution?

• Our customer found out way before everyone else

Observation & Hypothesis

32

•  Observation -  House of Cards, and original content may be a big deal and no

one knows the impact yet -  An analysis of House of Cards viewer opinions could provide

very interesting, publishable research content

•  Hypothesis -  People are consuming House of cards differently (binge

watching) -  Viewers appreciate this content and it will impact how they

value their Netflix subscription

Drafting our blog post/memo/research report

33

Netflix launched ‘House of Cards’. Netflix will provide metrics on viewership in several months. In anticipation, we talked to [XXX] Netflix subscribers last week to predict ‘House of Cards’ viewership and how important original content will be to Netflix subscribers. We found that [XXX] of Netflix customers surveyed had watched ‘House of Cards’. Of those who had watched the show, viewers reported watching [XXX] episodes on average. How will this impact Netflix? [XXX] of the ‘House of Cards” viewers indicated that original programming was [XXX] important as a subscription retention driver for Netflix.

The questions

34

1 2 3

Collecting respondents

35

Hi, we can help with that…

SurveyMonkey Audience

36

•  Millions of people ready to share their opinions

•  Recruited from SurveyMonkey •  Rewarded with charitable

donations •  Trustworthy opinions •  Dozens of targeting options •  Global respondents

Example Audience project summary

37

38

1

39

2

40

3

Let’s revisit our draft…

41

Filling in the blanks

42

Netflix launched ‘House of Cards’. Netflix will provide metrics on viewership in several months. In anticipation, we talked to over 150 Netflix subscribers last week to predict ‘House of Cards’ viewership and how important original content will be to Netflix subscribers.

We found that 20% of Netflix customers surveyed had watched ‘House of Cards’. Of those who had watched the show, viewers reported watching nine episodes on average.

How will this impact Netflix? 26% of the ‘House of Cards” viewers indicated that original programming was extremely or very important as a subscription retention driver for Netflix.

Making better decisions

43

• Does this replace the need for a research department? Or your research agency? -  No, but this process empowers everyone

• Test out concepts, campaigns, and use data to make more decisions be great decisions

• At today’s speed of business, you can’t afford not to incorporate data into your decision making

Top use cases for marketers

44

• Message testing • Creative testing •  Product research •  Brand research •  Segmentation studies •  Persona development • Competitive research • Content generation

Give it a try

45

•  Pay only for the survey respondents •  We do the rest

-  We’ll help you create an amazing survey for free ($999 value)

Email: Visit:

AMA@surveymonkey.com audience.surveymonkey.com

*Conditions: Offer expires on December 31, 2013. Offer available to customers who reference the AMA Iowa presentation. Limit of 1 discount per customer. Survey respondent purchase minimum of $1,500 to qualify for the discount. You must apply for the offer by emailing ama@surveymonkey.com and mentioning this offer.

Thank you!