• What’s good?– Clear wording.– A “typical week”

eliminates confusion as to weekday vs. weekend

• What’s bad?– The “buckets” are not evenly

distributed. (normative and informational bias towards low answers)

– The behavior is poorly defined. • E.g., “What part of my internet activity

fits their definition?”

• What would be better?– Clarify what “visiting

websites” means– Perhaps ask separate

questions about weekdays or weekends.

– Use open-ended reporting or same-sized buckets.

• What’s bad?– There is not a lot to learn from the

question.– It is really obvious that MAZDA3 is

the target of the survey.– Arrangement of response options

indicates that “Not Sure” does not fall between the first two.

– Ambiguates response

• What’s good?– Clear wording.– Allows for a non-response

(i.e., “not sure”).

• What would be better?– If you want to gauge actual

familiarity, perhaps ask people to match photos with labels.

– If you want subjective familiarity, allow people to report that on a graded scale.

– If you want relative subjective familiarity, ask people to rank the options.

– Choose comparison options that are similar in knowledge/familiarity.

• What’s bad?– Extraordinarily difficult to answer

correctly (“not sure” is very tempting).– Again, “not sure” is in a peculiar

physical position.– 1. “I am not sure what a MAZDA3 is”– 2. “I am not sure if I have seen it

advertised”– 3. “I am not sure if those

advertisements were online.”– 4. “I am not sure if it was within 30

days.”

• What’s good?– Clear wording.– Clearly identifies which

media channel under consideration.

• What would be better?– Implicit Goal: Has the

respondent seen any online ads for our product?

– Perhaps it is better to simply estimate based on knowledge of website and exposures paid for.

– Alternatively, you could ask about something precise and see if respondents identify it. (e.g., an image or a slogan)

• What’s bad?– Ambiguates indifference and

ambivalence.– Who has a “very unfavorable” view of a

Sentra? (the scale implies too much range).

• What’s good?– Generally clear.– Easy to use 5-point

bipolar scale.

• What would be better?– Other than conflation of

brand ignorance with brand indifference, this question seems easy to answer and face valid.

– Yay!

• What’s bad?– Implies that people would (should)

have different strategies for different brands.

– How do you differentiate someone who would consider a Civic (but not the others) from someone who searches differently by brand?

– What is Consumer Reports? Does it matter if I read it in print or online?

• What’s good?– Allows for many

responses– Conceptually captures

meaningful variation in consumer search

• What would be better?– Perhaps ask people to

identify where they would seek automotive advice in general, and provide a semi-exhaustive list.

– If you want to know which brand people are considering, ask that directly.

• What’s bad?– What is “neutral” consideration of

purchase? – Is the respondent meant to interpret

these as “relative likelihood” or “absolute likelihood”?

– What constitutes consideration (have I already reached the minimum threshold)?

• What’s good?– Simple, generally

straightforward.

• What would be better?– Ask two questions: (a) Would

you consider buying one of these vehicles? (b) Relative to the listed alternatives, how much would you consider this option?

– Perhaps allow people to express likelihood on a 100-point scale.

– Remove silly neutral option.

• What’s bad?– We already know that this is a survey

for MAZDA3, that is probably the answer.

– The extent to which I am uncertain, that answer is more likely (because the other brands are better known).

– On the other hand, “don’t know” is incredibly appealing.

• What’s good?– Objective method for

assessing knowledge of advertising.

• What would be better?– Ask about a number of

slogans and look for the correct pairing.

– Include fake slogans.– Exclude the “don’t know”

option.

• What’s bad?– If MAZDA3 is unfamiliar, that is a lot of room for

non-response.– Scale midpoint responses are identical for people

who have no knowledge and for people with neutral opinions.

– People feel like they are doing something wrong if they give the same answer over and over.

– False variation… and undifferentiable from random responding.

• What’s good?– Directly asks about

associations with the product.

• What would be better?– Use a previous question to

identify sufficient product knowledge.

– Allow people to respond “don’t know” for specific items.

• What’s bad?– Very difficult to express uncertainty.– Mini-van? – What counts as a cross-over SUV?– If I want a sedan that is sporty, but not

luxurious, which box do I check?

• What’s good?– Gives a necessarily

complete list (i.e., it includes “other”).

– Most entries are understandable.

• What would be better?– Perhaps ask about specific

parameters and build the concept.

– How many doors? Approximate price? How fast? Mileage?

– Ask a separate question about how certain people are in their preferences.

• What’s bad?– Buckets range in size. Are they

proportional to the population? – Because the lower end is over

represented, the survey has now made it clear that the MAZDA3 is not for wealthier people.

• What’s good?– A complete list of options.– Income is generally well

understood.– Small buckets allow for

sensitivity at the smaller end of the scale.

• What would be better?– Actually, this is challenging.

An open-ended item would be fine in concept, but people may feel lazy or uncomfortable answering it.

– Even buckets might be nice, but that would provide less subtlety at the low end.

– Perhaps ask two questions: large buckets, followed by small buckets.