Measuring the Effectiveness of the

Promotional Program

Measuring the Effectiveness of the

Promotional Program

19

McGraw-Hill/Irwin Copyright © 2009 by The McGraw-Hill Companies, Inc. All rights reserved.

Pros and Cons of Measuring Effectiveness

Objections of creativesObjections of creatives

Evaluate alternative strategies

Evaluate alternative strategies

Avoid costly mistakesAvoid costly mistakes

Increase efficiency in general

Increase efficiency in general

Disagreement on what to test

Disagreement on what to test

Research problemsResearch problems

Cost of measurementCost of measurement

Determine if objectives are achieved

Determine if objectives are achieved

TimeTime

Advantages Disadvantages

Evaluating Alternative Strategies

Measuring Effectiveness: Marketers’ Views

Issue Agree

MPM is a key priority for today’s technology companies.

90%

I am dissatisfied with our ability to demonstrate marketing programs’ business impact and value.

80%

It is important to define, measure, and take concrete steps in the area of advertising accountability.

61.5%

I can forecast the impact on sales of a 10% cut in marketing spending.

37%

How to test• Testing guidelines• Appropriate tests

How to test• Testing guidelines• Appropriate tests

Measuring Advertising Effectiveness

When to test• Pretesting• Posttesting

When to test• Pretesting• Posttesting

Where to test• Laboratory tests• Field tests

Where to test• Laboratory tests• Field tests

What to test• Source factors• Message variables • Media strategies• Budget decisions

What to test• Source factors• Message variables • Media strategies• Budget decisions

Pretesting Methods

On-air Tests

Dummy Ad VehiclesConsumer Juries

Portfolio Tests

Physiological Measures

Theater Tests

Rough Tests

Concept Tests

Readability Tests

Comprehension and Reaction Tests

Laboratory Field

Posttesting Methods

Recall TestsRecall Tests

Inquiry TestsInquiry Tests

Association Measures

Association Measures

Single-Source

Systems

Single-Source

Systems

Tracking Studies

Tracking Studies

Recognition Tests

Recognition Tests

MethodsMethods

Posttesting the Value of a Sponsorship

Test Your Knowledge

Laboratory methods of testing for advertisement effectiveness generally offer high control but low _____.

A) realism

B) external validity

C) costs

D) generalization

E) value

Where to Test

In the Field

In the Lab

Positioning Advertising Copy Test (PACT)

1. Provide measurements relevant to objectives of advertising

2. Require agreement on how results will be used

3. Provide multiple measures

4. Be based on a model of human response to communications

5. Consider multiple versus single exposure to the stimulus

6. Require alternative executions to have same degree of finish

7. Provide controls to avoid biasing effects of exposure context

8. Take into account basic considerations of sample definition

9. Demonstrate reliability and validity

Test Points

3.Finished art or commercial pretesting

3.Finished art or commercial pretesting

1.Concept Testing1.Concept Testing

2.Rough Testing2.Rough Testing

4.Market testing (posttesting)

4.Market testing (posttesting)

Occurs at Various Stages

Occurs at Various Stages

Concept Testing

Explores consumers’ responses to ad concepts expressed in words, pictures, or symbols

ObjectiveObjective

Alternatives are exposed to consumers who match the target audience

Reactions & evaluations sought through focus groups, direct questioning, surveys, etc.

Sample sizes depend on the number of concepts and the consensus of responses

MethodMethod

Qualitative and/or quantitative data evaluating and comparing alternative concepts

OutputOutput

Rough Art, Copy, and Commercial Testing

Number of ads that can be evaluated is limited

Preference for ad types may overshadow objectivity

Consumer may become a self-appointed expert

A halo effect is possible

Cost effectiveness

Endorsements by independent third parties

Achievement of credibility

Comprehension and Reaction Tests

Consumer Juries

Control

Advantages Disadvantages

Rough Testing Terms

Photomatic RoughPhotomatic Rough

Animatic RoughAnimatic Rough

Live-action RoughLive-action Rough

TermsTerms

An Animatic Example from Airwalk

*Click outside of the video screen to advance to the next slide

Spot Resulting from Animatic Rough

*Click outside of the video screen to advance to the next slide

Pretesting Finished Print Ads

Based on syllables per 100 words

Other factors also considered

Readability Tests

Readability Tests

A laboratory method

Includes test and control ads

Portfolio test have problems

Portfolio Tests

Portfolio Tests

Distributed to random sample homes

Product interest may still bias results

Dummy Advertising

Vehicles

Dummy Advertising

Vehicles

Test Your Knowledge

_____ is a method of testing ads by placing them randomly in certain copies of regularly distributed magazines.

A) Vehicle source testing

B) Burke's reflection test

C) A Flesch test

D) Dummy advertising vehicle testing

E) A contextual test

Pretesting Finished Broadcast Ads

Theater Tests

•Measures changes in product preferences

•May also measure . . .•Interest in and reaction

to the commercial•Reaction from an

adjective checklist•Recall of various

aspects included•Interest in the brand

presented•Continuous reactions

On-Air Tests

•Insertion in TV programs in specific markets

•Limitations are imposed by “day-after recall”

•Physiological Measures

Physiological Measures

Eye trackingEye tracking

Pupil dilationPupil dilation Galvanic skin response

Galvanic skin response

Brain wavesBrain waves

TestingTesting

Market Testing Print Ads

Recall TestsRecall Tests

Inquiry TestsInquiry Tests Recognition Tests

Recognition Tests

Tracking StudiesTracking Studies

TestingTesting

Starch-Scored Sports Illustrated Ad

Posttests of Broadcast Commercials

Test marketing

Day after recall tests

Persuasive measures

Diagnostics

Comprehensive measures

Single-source tracking

Tracking studies

Testing

Comprehensive Testing by Ipsos-ASI

Problems With Current Research Methods

Essentials of Effective Testing

Use a consumer response modelUse a consumer response model

Use pretests and posttests

Use pretests and posttests

Use multiple measures

Use multiple measures

Understand and implement

proper research

Understand and implement

proper research

Establish communications

objectives

Establish communications

objectives TestingTesting

Test Your Knowledge

Good tests of advertising effectiveness must address the nine principles established by PACT. One of the easiest ways to do this is to follow a decision sequence model. The first step in the model is to:

A) Understand the appropriate research

B) Create a model that uses multiple measures

C) Establish communication objectives

D) Decide whether to use posttests or pretests

E) Develop a consumer response model

Measuring Effectiveness of Other Programs

Shopping cart signageShopping cart signage

Ski resort-based mediaSki resort-based media

In-store radio and televisionIn-store radio and television

Other mediaOther media

Non-traditional

media

Non-traditional

media

Sales promotions

Sales promotions

Sponsor-ships

Sponsor-ships

Exposure methodsExposure methods

Tracking measuresTracking measures

Measuring Effectiveness + Efficiency