Chapter Eight Experimentation in Marketing Research

Chapter Eight

Experimentation in Marketing

Research

Copyright © Houghton Mifflin Company. All rights reserved. 8 | 2

Eddie Bauer's Electronic Windows

• Eddie Bauer, a leading tri-channel specialty retailer was looking for a way to draw more shoppers into their stores

• Indiana University students conducted an in-store advertising experiment using electronic window posters (images displayed on plasma screens) on 3 selected stores


Electronic Windows (Cont’d)

• RESULTS– The number of passersby who entered the

control stores went up 7 percent

– Sales soared 56 percent compared to the weeks before the installation of digital windows


Advertising Experiment

• Will replacing commercial A with commercial B lead to a marked increase in consumer preference for a company’s brand?


Pricing Experiment

• Can a company improve the profitability of its fashion clothing line by increasing its price by 10 percent?


Sales Productivity Experiment

• Will an increase in the average number of sales calls per customer from six to eight per year significantly improve sales?


Shelf Space Experiment

• Will decreasing the shelf space allocated to brand X detergent by 25 percent significantly lower its sales?


Direct Mail Experiment

• Will it be worthwhile to mail last year's donors an attractive (but expensive) brochure describing the company’s activities and soliciting higher contributions for this year?


Experiment

• An experiment is a procedure in which a company manipulates one (or sometimes more than one) independent or cause variable and collects data on the dependent or effect variable while controlling for other variables that may influence the dependent variable


Descriptive Research

• This research asks consumers whether they would buy more of a product if its price were lowered

• Descriptive survey data will merely suggest causation


Experimental Research

• Manipulates the independent variable or variables before measuring the effect on the dependent variable– The effect of price changes on sales volume of

a particular product can be examined by actually varying the price of the product

• The very basis of experimental research lies in the manipulation of independent variables


Conditions For Inferring Causality

• Temporal ordering of variables – X Y not Y X

• Evidence of association– X and Y are related ; presence of X

presence of Y; absence of X absence of Y

• Control of other causal factors– X Y, Z Y


Laboratory vs. Field Experiments

• A laboratory experiment is a research study conducted in a contrived setting in which the effect of all, or nearly all, influential but irrelevant independent variables is kept to a minimum

• A field experiment is a research study conducted in a natural setting in which the experimenter manipulates one or more independent variables under conditions controlled as carefully as the situation will permit


Internal Validity

• Internal validity is the extent to which observed results are solely due to the experimental manipulation

• Laboratory experiments are generally high on internal validity

• Field experiments are generally low on internal validity


External Validity

• External validity is the extent to which observed results are likely to hold beyond the experimental setting

• Laboratory experiments are generally low on external validity

• Field experiments are generally high on external validity


Deciding Which Type of Experiment to Use

• Practical Considerations – Time

– Cost

– Exposure to competition

– Nature of the manipulation


Test Marketing

• Dunkin Donuts and Baskin-Robbins are now offered in “combo” stores

• KaBloom is testing kiosk flower sales in a variety of locations

• Utilities companies are experimenting with providing Internet services via existing power lines


McDonald's Tests McPizza

• McDonald's test-marketed McPizza to strengthen the after-4pm adult market– Introduced McPizza with heavy advertising,

emphasizing speedy service for pizza

– McPizza received favorable nods in some test markets and had partial rollout nationally


McDonald's Tests McPizza (Cont’d)

• Pizza Hut, a leading competitor, reacted aggressively to McDonald's move by running a buy-one-get-one-free promotion wherever McPizza was introduced

• The sales performance of McPizza did not meet management's expectations


Simulated Test Marketingwww.harrisinteractive.com/advantages/marketingsciences.asp

• Step #1 Pre-recruitment• Step #2 Background: habits and practices• Step #3 Exposure to real advertising in a

competitive context• Step #4 Simulated store purchase• Step #5 Post: purchase inquiry• Step #6 Respondents take product home for

usage• Step #7 Post: usage evaluation


Virtual Test Markets-- Ray Burke, Professor of Business Administration at Indiana University

• Created a virtual store to determine how products catch a consumer's eye

• Computer 3-D graphics create a feeling of being in a store, walking past shelves of grocery items just as in a real store

• Consumers can pick items off the virtual shelves to examine them as in real store and can select items they would buy


Virtual Test Markets

• Virtual simulated marketing tests will enable companies to examine consumers' reactions to new products, product line extensions, prices, packaging, and merchandising


Scanner Data Analysis

• Electronic scanners at the checkouts capture the product sales

• Marketers of packaged goods conduct sophisticated field experiments

• The data from the stores are transmitted electronically to central computers for analysis and interpretation

• Information Resources Inc. (IRI) and ACNielsen offer marketers a variety of services through their information system called BehaviorScan and Scantrack


Web-Based Experiments

• Web-based experimentation will enable companies to test a wide range of possible marketing mix changes and statistically model consumer responses to these changes


Web-based Experiments Conducted to Test the Effectiveness of Banner Advertising

• Random people were selected while visiting the company’s website

• The questionnaire asked them to complete a short questionnaire while on the site

• Participants are shown random test banner ads

• Participants fill out a second survey, answering questions about the impact of the banner ads on their impressions of the brand


Web-based Experiments Conducted to Test the Effectiveness of Banner Advertising (Cont’d)

• What are the independent variables?• What is the dependent variable?• What are some validity threats?


Internal Validity

• The presence of any condition or occurrence (other than the independent variable manipulation) that can offer a compete explanation for the experimental results is a threat to internal validity


Threats to Internal Validity

• History - external events• Maturation - physiological or psychological

changes that occur over time• Pretesting - Early responses impact later

responses• Instrument Variation - questionairre changes• Selection - groups differ on characteristics• Mortality - participants drop put


Threats To External Validity

• External validity of experimental results relates to their generalizability

• The various internal validity threats also indirectly affect external validity

• Biases that stand in the way of generalizing experimental results: – Reactive bias

– Pretest-manipulation interaction bias

– Non-representative-sample bias



Hyundai India’s Experiment

• Hyundai wanted to test and increase its brand awareness in rural India

• Hyundai attempted to identify and establish relationships with village leaders– The leaders are the only ones offered a test

drive of the vehicle

– A van with a video screen is brought to the village


Hyundai India’s Experiment (Cont’d)

• Advertisements are shown to the assembled villagers via the vans’ screens

• In some cases, the leaders take this opportunity to announce they have decided to purchase the vehicle

• These types of field experiments are not conventional field experiments, they are intended to gauge the effectiveness of a novel marketing technique


Hyundai India’s Experiment (Cont’d)

• What do you think of the Hyundai experiment?

• What causal inference is implied in this scenario?

• What is your evaluation of the validity of the implied causal inference?


Pre-Experimental Designs

• Pre-experimental designs exert little or no control over the influence of extraneous factors

• These studies are not much better than descriptive studies when it comes to making causal inferences

• Pre-experimental: emphasizes the fact that these studies are more exploratory than conclusive as far as causal inferences are concerned


Reasons for Studying Pre-Experimental Designs

• Studies employing pre-experimental designs often form the basis for causal inferences in the real world, and we need to be aware of their pitfalls to avoid interpreting their findings at face value

• Comparisons with pre-experimental designs can help highlight the merits of true experimental designs


One-Group, After-Only Design

• Situation A. A company introduces a new brand of margarine in four test market areas and employs a unique and revolutionary promotional campaign for it– The brand captures at least a 10 percent share in each

market within two months after introduction

– The company's management concludes that the revolutionary promotional campaign played a major role in the market share achieved by the brand


One-Group, After-Only Design (Cont’d)

• Situation B. The president of the United States makes a television speech soliciting public support for legislation favoring prayer in public schools– A telephone survey of those who viewed the

presidential speech indicates that 70 percent favor such legislation

– The president's speech is therefore considered to have had a significant impact on the U.S. public


One-Group, After-Only Design (Cont’d)

• Casual inference from a one-group, after-only design cannot be trusted entirely


True Experimental Designs

• The presence of one or more control groups • The random assignment of units to various

experimental and control groups• Random assignment distributes the sample

units chosen for a study to various groups on a strictly objective basis so that the group compositions can be equivalent before an experiment is started


2004 Ford F-150 Launch

• Ford launched the redesigned F-150 with the biggest communication blitz in its history

• 90% of the spending went to television, and it was estimated that males 25 – 49 saw the F-150 ad 30 times during the 60 day launch

• The Internet was also extensively used for this campaign

• An online marketing research companies was hired to track the effectiveness of this campaign


2004 Ford F-150 Launch (Cont’d)

• The researchers used a variety of experimental designs to isolate the effects of the various media

• The campaign was shown to be highly successful, and overall brand recall rose 26 percent over the course of the campaign


Experimental Designs

EG X O

EG O1 X O2

EG

CG

X O1

O2

EG

CG

O1

O3

X O2

O3

Documents

Chapter Eight Experimentation in Marketing Research