Communications Effects Pyramid

20% Trial

40% Liking

90% Awareness

5% Use

70% Knowledge/Comprehension

25% Preference

The DAGMAR Approach

Define

Advertising

Goals for

Measuring

Advertising

Results Action

Awareness

Conviction

Comprehension

AD Effectiveness• Advertising is of no use if the defined objectives of

communication is not achieved.

• So, it is necessary to evaluate the effectiveness ofadvertisement at different level, starting from creationof ad-copy to running of ad on media, and also afterexecution of ad to know to what extent the objectivesare achieved.

Types of tests to measure effectivenessFollowing are the types of test applied in advertisement evaluation:

• Pre-Testing

• Concurrent Testing

• Post Testing

Developing an Advertising Campaign (cont’d)

• Evaluating Advertising Effectiveness

Evaluation Assessment

Pretest Evaluation of ads performed before a campaign begins

Consumer Jury A panel of a product’s actual or potential buyers who

pretest ads

Posttest Evaluation of advertising effectiveness after the

campaign

Recognition Test A posttest in which individuals are shown the actual

ad and asked if they recognize it

Unaided Recall Test A posttest in which respondents identify ads they have

recently seen but are given no recall clues

Aided Recall Test A posttest that asks respondents to identify recent ads

and provides clues to jog their memories

1.Pre-Testing

• Pre-Testing follows the universal law "Prevention isbetter than cure". Advertising can be pretested atseveral points in the creative development process.

• Pre-Testing helps the advertiser to make a final go orno go decision about finished or nearly finishedadvertisement. Pre-Testing method refer to testing thepotentiality of a communication message or ad-copybefore printing, broadcasting, or telecasting

Qualitative Methods of Pre-Testing

• Focus Group: Focus group involve exposing the ad to a group of8 to 12 respondents. Focus groups are used with surprisingfrequency for making final go or no go decision. A moderatorfacilitates the discussion and walk s the group through a seriesof issues that are outlined in discussion guide.

• In-depth Interview: In-depth interview involve one on onediscussion with respondents. Interviews are very effectivewhen a researcher has a good idea of critical issues but doesnot have a sense of the kind of responses one will get. Thismethod can be effectively used to generate new ad conceptsand ideas.

• Projective Techniques: In this technique the respondent isinstructed to project himself into the situation and verbalizethe thoughts. projective technique can be very effective forevaluating ad concepts and for generating new ad concepts.But, it cannot be used for making final decisions.

Quantitative Methods of Pre-Testing

• Checklist Method: Checklist method is used to test the effectiveness of ad-copy. The purpose of this method is to ensure that all elements of the ad-copyare included with due importance in the advertisement. As it is a pre-testmethod any omitted element of ad can be included in the copy before releaseof the advertisement.

• Consumer Jury Method: This method involves the exposure of alternativeadvertisements to a sample of jury or prospects. This test is designed to learnfrom a typical group of prospective customers. Advertisements which areunpublished are presented before the consumer jury either in personalinterviews or group interviews and their reactions are observed and responsesare recorded.

• Sales Area Test: Under this method advertising campaign is run in the marketsselected for testing purposes. The impact of the campaign is evaluated byactual sales in the selected markets. The market with high sales is consideredthe best market for effective sales campaign. In other markets suitable changesare made in the advertising campaign.

• Questionnaire Method: It is a list of questions related to an experiment. Thedraft of an advertisement along with some relevant questions is to be sent to agroup of target consumers or advertising experts. Their opinions are collectedand analyzed to find out whether the proposed advertisement is satisfactory ornot.

Contd…

• Recall Test: Under this method, advertising copies are shown toa group of prospects. After few minutes they are asked to recalland reproduce them. This method is used to find out how farthe advertisements are impressive.

• Reaction Test: The potential effect of an advertisement is judgedwith the help of certain instruments, which measure heartbeats,blood pressure, pupil dilution etc. Their reactions reveal thepsychological or nervous effects of advertising.

• Readability Test: All the listeners of advertisements cannot readit equally. So respondents are drawn from different socioeconomic and geographical backgrounds. This method is used tofind out the level of effectiveness when and advertisement isread.

• Eye Movement Test: The movements of eyes of the respondentsare recorded by using eye observation camera whenadvertisements are shown to them in a screen. This helps to findout the attention value of advertisement.

2.Concurrent Testing

• Concurrent testing is evaluated throughout the wholeadvertisement execution process. Tests are conducted whileaudience is exposed to different type of media. Following are thetypes of concurrent testing methods:

• Consumer Diaries: Diaries are provided to a selected customers.They are also informed to record the details of advertisementsthey watch, listen or read. The diaries are collected periodically.The result obtained from such a survey reveals the effectivenessof advertisement.

• Co-incidental Surveys: This method is also called as co-incidentaltelephone method. Under this method, samples of customersare selected and calls are made at the time of broadcast of theadvertisement programme. The data obtained and analyzed willgive a picture about the effectiveness of an advertisement.

• Electronic Devices: Now day’s electronic devices are widely usedto measure the effectiveness of an advertisement. They aremainly used in broadcast media. These are auto meters, trackelectronic units etc.

3. Post Testing

• Post testing is done to know- to what extent the advertisingobjectives are achieved. Following are the types of post testingmethods:

• Recognition Test: Recognition test involves the ability of viewers tocorrectly identify ad, brand, or message they previously exposedto. The types of recognition test are:

• Starch Test - The Starch test is applied only to print ads that havealready run. The interviewer shows each respondent a magazine ornewspaper containing the ads being tested. For each ad theinterviewer asks the respondents to reply to ad related questions.

• Bruzzone Test - The Bruzzone test is conducted through mailsurveys. Questionnaires containing frames and audio scripts fromtelevision commercials are sent to respondents and respondentsare asked whether they recognize the ad and brand.

• Recall or Impact Test: The recall test is designed to measure the impression of readers or viewers of the advertisement. If a reader has a favorable impression of the advertisement, he will certainly retain something of the advertisement. The measures of interest would be obtained by interviewing the readers or viewers or listeners, days after the advertisement or commercial is appeared in the newspaper, or on T.V. Interviewer asks the readers or viewers to answer some ad related questions, and in response to the question asked, the reader reveals the accuracy and depth of his impression.

Copyright © Houghton Mifflin Company. All rights reserved.

19 | 13

Developing an Advertising Campaign (cont’d)

• Evaluating Advertising Effectiveness

Evaluation Assessment

Pretest Evaluation of ads performed before a campaign begins

Consumer Jury A panel of a product’s actual or potential buyers who

pretest ads

Posttest Evaluation of advertising effectiveness after the

campaign

Recognition Test A posttest in which individuals are shown the actual

ad and asked if they recognize it

Unaided Recall Test A posttest in which respondents identify ads they have

recently seen but are given no recall clues

Aided Recall Test A posttest that asks respondents to identify recent ads

and provides clues to jog their memories

3.Finished Art or Commercial Testing

3.Finished Art or Commercial Testing

1.Concept Testing

2.Rough Testing

4.Market Testing (Post-testing)

2.Rough Testing

1.Concept Testing

The Testing Process

Occurs at Various Stages

On-air Tests

Dummy Ad VehiclesConsumer Juries

Portfolio Tests

Physiological Measures

Theater Tests

Rough Tests

Concept Tests

Reliability Tests

Comprehension and Reaction Tests

Pretesting MethodsLaboratory Field

Dummy Ad VehiclesConsumer Juries

Portfolio Tests

Physiological Measures

Theater Tests

Rough Tests

Concept Tests

Reliability Tests

Comprehension and Reaction Tests

Qualitative and/or quantitative data evaluating and comparing alternative concepts

Explores consumers’ responses to ad concepts expressed in words, pictures, or symbols

Alternatives are exposed to consumers who match the target audience

Reactions and evaluations are sought through focus groups, direct questioning, surveys, etc.

Sample sizes depend on the number of concepts and the consensus of responses

Explores consumers’ responses to ad concepts expressed in words, pictures, or symbols

Alternatives are exposed to consumers who match the target audience profile.

Reactions and evaluations are sought through focus groups, direct questioning, surveys, etc.

Sample sizes depend on the number of concepts being tested and the “consensus level” of initial responses.

Concept Testing

Objective

Method

Output

Recall Tests

Inquiry Tests

Association Measures

Single-Source Systems

Tracking Studies

Recognition TestsRecognition Tests

Inquiry Tests

Single-Source Systems

Association Measures

Recall Tests

Field Post testing Methods

Methods

Positioning Advertising Copy Testing (PACT)

1. Provide measurements relevant to objectives of advertising

2. Require agreement on how results will be used before each test

3. Provide multiple measures (Single measures aren't adequate)

4. Be based on a model of human response to communications

5. Consider multiple versus single exposure to the stimulus

6. Require alternative executions to have the same degree of finish

7. Provide controls to avoid the biasing effects of exposure context

8. Take into account basic considerations of sample definition

9. Demonstrate reliability and validity

Based on syllables per 100 words

Other factors also considered

A laboratory method

Includes test and control ads

Portfolio test have problems

Based on # syllables per 100 words

Measures comprehension efficiency

A laboratory method

Includes test and control ads

Portfolio test problems: Halo Effects

Pretesting Finished Print Ads

Readability Tests

Portfolio Tests

Dummy Advertising Vehicles

Distributed to random sample of homes

Product interest may still bias results

Distributed to random sample of homes

Theater Tests

Measures:• Interest in and reaction to the

commercial• Changes in product preferences• Emotional reactions and

engagement using an adjective checklist

• Recall of content, brand name, information, etc.

• Interest in the brand presented• Continuous (frame-by-frame)

reactions

On-Air Tests

• Insertion in TV programs in specific markets

• Typically measure “Day After Recall” (DAR) as main variable of interest

• Split Broadcasts

Theater Tests

Pre-testing Finished Broadcast Ads

Eye tracking

Pupil dilationGalvanic skin response

Brain waves (EEGs) & FMRI

Eye tracking

Galvanic skin response

Pupil dilation

Physiological Measures (Lab)

Testing

Test marketing

Day after recall tests Persuasive measures

Diagnostics

Comprehensive measuresSingle-source tracking

Tracking studies

Single-source tracking

Test marketing

Comprehensive measures

Diagnostics

Persuasive measuresDay after recall tests

Post-Testing Broadcast Commercials

Testing

23

Conclusion 1: All Commercials Are Not Created Equal: Ad Copy Must Be Distinctive

• Commercials having strong selling propositions are distinctive and tend to achieve higher ARS Persuasion scores (e.g. Mentadent toothbrush with a strong USP (flexible handle for gentle brushing) scored 11.2 on ARS persuasion)

• Commercials for new brands tend to be most persuasive, but commercials for established brands can be made persuasive via brand differentiation.

24

Conclusion 2—More is Not Necessarily Better: Weight Is Not Enough

• Ad weight means the number of gross rating points (GRP) that support an advertising campaign.

• An ineffective ad (not distinctive or persuasive) has no likelihood of increasing sales even if the ad weight is doubled or tripled.

25

Conclusion 3—All Good Things Must End: Advertising Eventually Wears Out

• Advertising ultimately wears out and must be refreshed to maintain or increase brand sales.

• Familiar brands have been shown to wear out more

slowly than unfamiliar brands.

26

Conclusion 4—Don’t Be Stubborn: Advertising Works Quickly or Not at All

• Some advertisers tend to “hang in there” and wait for an ad to increase sales.

• Most of the sales impact occurs in the first three months of a new ad.

• “Sunk costs” are an issue to consider, but if an ad is not working at first, it probably never will.