Dr. Michael R. Hyman, NMSU

Experiments in Marketing Research

2

Criteria for Establishing Causal Relationships

• Concomitant variation

• Temporal ordering of variables

• Control over other possible causal factors

3

Preliminary Definitions: Independent Variable

• Experimenter controlled variable

• Value can be manipulated by experimenters to whatever they wish it to be

4

Dependent Variable

• Value expected to be dependent on experimenter’s manipulation

• Criterion or standard by which results are judged

5

Test Units

Subjects or entities whose response to the experimental treatment are measured or observed

6

What is an Experiment?

• Research investigation under controlled conditions

• At least one independent variable is manipulated

• Its effect on at least one dependent variable is measured

• Purpose: To test hypothesis

7

Unit Pricing Experiment

8

Basic Experimental Design Issues

• Manipulation of independent variable

• Selection of dependent variable

• Assignment of subjects (or other test units) to treatments/conditions

• Control over extraneous variables

9

How Can Experimenters Control for Extraneous Variation?

• Eliminate Extraneous Variables

• Hold Conditions Constant

• Randomization

• Matching Subjects

• Presentation Order

• Blinding

10

Control Groups

Isolate extraneous variation

11

Erroneous Inferences Caused by Lack of Control

12

Field vs. Laboratory Experiment

13

Types of Research Environments

14

Artificiality of Laboratory vs. Field Experiments

15

Laboratory Experiment Field Experiment

High Internal Validity

Few ExtraneousVariables

High Control

Low Cost

Short Duration

Subjects Aware ofParticipation

High External Validity

Many ExtraneousVariables

Low Control

High Cost

Long Duration

Subjects Unaware ofParticipation

Demand Artifacts High Demand Artifacts Low

Few Participants Many Participants

Easier to Do Harder to Do

16

• Guinea pig effect

• Hawthorne effect

Demand Characteristics

Experimental procedures that intentionally hint to subjects something about experimenter’s hypothesis

17

Internal Validity

• Ability of experiment to determine if the experimental treatment was the sole cause of changes in a dependent variable

• Did the manipulation do what it was supposed to do?

18

Threats to Internal Validity

• History

• Maturation

• Testing

• Instrumentation

• Selection bias

• Mortality

19

Type of Extraneous Variable Example

History - Specific events in theenvironment between the Beforeand After measurement that are beyond the experimenter’s control

Maturation - Subjects changeduring the course of the experiment

Testing - The Before measure alertsor sensitizes subject to nature of experiment or second measure.

A major employercloses its plant intest market area

Subjects become tired

Questionnaireabout the traditionalrole of women triggers enhanced awareness of womenin an experiment.

20

Instrument - Changes ininstrument result in response bias

Selection - Sample selectionerror because of differentialselection comparison groups

Mortality - Sample attrition; some subjects withdraw from experiment

New questions aboutwomen are interpreteddifferently from earlierquestions.

Control group and experimental group isself-selected groupbased on preference forsoft drinks

Subjects in one groupof a hair dying study marry rich widows and move to Florida

Type of Extraneous Variable Example

21

22

Types of Experimental Designs

23

Sample Marketing Experiments Using Different Experimental

Designs

24

25

26

27

28

29

30

31

32

33

34

Test Marketing

35

Test Market

Field test of a new product or marketing mix element(s) of a current product using experimental or quasi-experimental designs

36

Uses and Objectives

• Evaluate proposed natural programs

• Estimate market shares and volumes

• Estimate cannibalization rate on existing product line

• Collect classification data on potential customers

• Analyze competitor reactions

37

Benefits and DisadvantagesBenefits

• Estimate sales potential under realistic conditions

• Find and correct weaknesses in marketing mix

Disadvantages

• Costs

• Tipping hand to competitors

38

What to Consider

• Tradeoff of costs and risk of product failure versus potential profits and probability of success

• Speed of competitor response

• Cost to produce product for test market

• Impact of failure on company reputation

39

Cost of Two-Market Test

Direct costs: $500,000

• POP materials, coupons, samples, syndicated research, media time/space, ad agency payments, initially high trade allowances

Indirect costs

• Management time, diversion from current products, negative internal/external impact of test failure, etc.

40

Steps in Test Market Study

1. Define objectives• Estimate market share, purchase frequency,

cannibalization rates• Discover who buys product and where

2. Select testing approach• Standard• Simulated (mall testing facility)• Controlled test market (e.g., BehaviorScan)

41

Steps in Test Market Study

3. Develop marketing plan

4. Select test markets• Should reflect regional differences if important,

but have similar demographics and little media spillover from/to other markets

• Avoid over-tested and idiosyncratic markets

42

43

Steps in Test Market Study

5. Run test markets• Sales estimates take 6-12 months because

initial estimates of repeat purchases high

6. Analyze results• Check purchase data (i.e., trial versus

repeat purchases), awareness data, competitive response, and sales of all similar products