Business Research Method

Business Research Method

Experimentation

Experiment

. Experiment: A research method in which conditions are controlled so that one or more independent variables can be manipulated and their effect on dependent variable is measured .

To study impact of a sports coach on cricket team’s performance an experiment can be conducted

Experiment

.-A research investigation in which conditions are controlled

-One independent variable is manipulated (sometimes more than one)

-Its effect on a dependent variable is measured

-To test a hypothesis

Some Definitions

• Absolute Experiment: If we want to know impact of a sports coach on cricket team’s performance it is an absolute experiment

• Comparative experiment :If we want to know impact of one sports coach on cricket team’s performance compared to impact of another sports coach on cricket team’s performance it is a comparative experiment

• Experimental units: Subjects or entities subjected to various treatments are called experimental units

Some Definitions

- Experimental group: The group of subjects exposed to an experimental treatment.

Group of students provided with a sport coach

- Control Group :A group of subjects who are exposed to the control conditions in an experiment, that is they are subjects not exposed to the experimental treatment

Group of students not provided with a sports coach

Treatment

- Treatment: It refers to conditions to which experimental groups are subjected and control groups are not subjected

Provision of sports coach

Extraneous Variable

• Extraneous variable: Independent variables that are not directly linked with the study but may influence dependent variable:

• If the study wants to test a hypothesis that a relationship exists between academic achievement and self concept of children then academic achievement is dependent and self concept is independent variable.

• Intelligence may also affect academic achievement but it is not related to study’s purpose and hence it is an extraneous variable.

Experimental Design• Experimental Design is a set of

procedures specifying

--Manipulation of the Independent Variable

--Selection of Dependent Variable

--Experimental groups & Control groups

--Control Over Extraneous Variables

Manipulation of independent Variable

• The experimenter has some degree of control over the independent variable

• The variable is independent because its value can be manipulated by the experimenter.

• To test relationship between package design & sales• Expose customers to three package designs A,B &C.

These packs are placed on shelves of select outlets• Consumers’ response is measured• Package design is independent variable ,which is

manipulated & there are 3 treatment levels A,B & C

Selection of Dependent Variable

• Selection of dependent variable is another important decision

• Sales volume of the product is considered as dependent variable in package design example

• Selecting dependent variable may not be easy in all cases

• To evaluate effectiveness of various ad programmes dependent variable can be brand image, brand awareness & product sales

• Purpose of the research helps in selection of appropriate dependent variable

Experimental & Control Groups• Experimental group is a group of test units exposed to

change in independent variable• Control group is a group of test units not exposed to

change in independent variable• In package design example a group of supermarkets

(Experimental group) is selected & each package design is displayed for a month

• Another group of supermarkets (Control group) continue to carry regular package design for that period

• Sales of product are measured for both groups & difference in sales is analysed to find out impact of design on sales

Controlling Extraneous Variables

• Extraneous (Confounding) variables need to be controlled otherwise it is difficult to find out whether the change in dependent variable is due to change in independent variable or change in extraneous variable

Ways to control extraneous variable • Randomization

• Hold Conditions Constant

• Matching Subjects

• Design Control

• Statistical Control

Randomisation• Process of assigning test units randomly to

experimental treatments and assigning experimental treatments randomly to test units

• It helps to spread effects of extraneous variable equally over the test units

• Most popular method • Not effective for small samples ----Department stores assigned randomly to

Experimental & Control groups ----If 3 versions of test commercials are to be

assigned to 3 groups then one commercial is selected at random & assigned to a randomly selected group & process is repeated

Physical Control of Extraneous Variables • Hold Conditions Constant: The

level of extraneous variables is kept constant throughout the experiment

-- If price is the extraneous variable prices are not changed during experimental period

Physical Control of Extraneous Variables • Matching Subjects: Judgmental sampling is

used to assign test units to both experimental group & control group so that both groups are matched in terms of characteristics of test units

• ---Department stores are matched on the basis of annual sales, size and location. One store from each matched pair would be assigned to each experimental group & other to control group

Statistical Control• Extraneous variables affecting dependent

variable are identified & measured using appropriate statistical tools like ANOVA. Then effects of extraneous variable on dependent variable are removed by statistical adjustments

• If 5 departmental stores are used to study sales of products before & after in- house sales promotion then impact of the characteristics of the stores can be removed through use of ANOVA-randomised block design using department stores as blocks

Design Control• Selecting appropriate experimental

designs to conduct experiments helps in controlling specific extraneous variables .

• Experimental Designs are of 4 types -Pre Experimental -True Experimental -Quasi experimental -Statistical

Experimental Validity• Validity: Extent to which a research process

is accurate & reflects actual market conditions

• Internal Validity: It measure the extent of change in dependent variable in an experiment caused by manipulation of independent variable .( Not vitiated by extraneous variables)

• External Validity :It measures to what extent inferences derived from experiment can be generalised to real environment

Internal Validity• Control of extraneous variable is a

necessary condition for establishing internal validity

• Internal validity is the basic minimum requirement of an experiment

• Six major types of extraneous variables are threats to internal validity

Factors Influencing Internal Validity

• History

• Maturation

• Testing

• Instrumentation

• Selection

• Mortality

Type of Extraneous Variable Example

History - Specific events in theenvironment between the Beforeand After measurement that are historic in nature (rare occurrence)

Maturation - Subjects change during the course of the experiment due to passage of time. Subjects become tiredTesting - The Before measure alertsor sensitizes subject to nature of experiment or second measure.

While test marketing new farm equipment area gets affected by floods Impact of promotionalcampaign for a car getsaffected by steep price risein petrol Drug trials carried over longer period of time results in some effect on account ofphysiological changesQuestionnaire about the traditional role of women triggers enhanced awareness of women in an experiment.

Instrument - Changes ininstrument to minmise testing effect result in response bias

Selection – test units selected for experimental group differ from those assigned to control group

Mortality - Sample attrition; some subjects withdraw from experiment

New questions aboutwomen are interpreteddifferently from earlierquestions.

Heavy users in experimental group & moderate & lowusers in control group

Subjects in TV viewing study drop out before completion

External Validity• Experiments conducted in natural

settings offer more external validity as compared to experiments in controlled conditions.

• Since generalisation is easier with field conditions

Experimental Environment• Laboratory Environment: Experiments conducted in controlled

conditions ---Showing ads or products to select consumers in controlled

conditions ----blind taste tests• Advantages-• Effects of extraneous variables can be minimised• -History effects eliminated• -Provides more internal validity• -time & cost effective• -reduces the risk of information about products & ideas being passed

on to competitors• Disadvantages• -Expts conducted in artificial conditions &results may not hold well

in actual conditions• -less external validity• -results influenced by testing effects ,Test units aware of being

tested & may not respond naturally

Field Environment• Experiments conducted in natural settings

--Launching products in select regions

--Observing consumer behaviour regarding a POP display in supermarkets

--analysing customer response to trial offers• Advantages• -high degree of external validity• Disadvantages• -Low degree of internal validity• -Require greater time& efforts and are expensive

Laboratory Experiment Field Experiment

Artificial-Low Realism

Few ExtraneousVariables

High control

Low Cost

Short Duration

Subjects Aware ofParticipation

Natural-High Realism

Many ExtraneousVariables

Low control

High Cost

Long Duration

Subjects Unaware ofParticipation

Different Basic Experimental Designs

• Pre-Experimental Designs

• True Experimental Designs

• Quasi Experimental Designs

• Statistical Designs

Notations • X: Treatment or manipulation of independent

variable• O: Observation or measurement of dependent

variable• R: Random assignment of test units to

experimental groups• EG : Experimental group exposed to experimental

treatment• CG :Control group not exposed to experimental

treatment• Notations in the diagram are read from left to right

Conventions• Movement from left to right indicates

movement through time

• Horizontal alignment of symbols implies that all those symbols refer to specific treatment group

• Vertical alignment of symbol implies that those symbols refer to activities or events that occur simultaneously

Examples • X O1 O2 means that a given group of test

units was exposed to treatment variable X and the response was measured at two different points in time ,O1 and O2

• R X1 O1

R X2 O2 means that two groups of test units were randomly assigned to two different treatment groups at the same time and the dependent variable was measured in two groups simultaneously

Pre-Experimental Designs

---They lack proper control mechanism to deal with effects of extraneous variables

--- Randomization is absent

One Shot Design (After Only)

One Group Pretest-Posttest

Static Group Design

One Shot Design (After Only)• It involves exposing experimental group to treatment X

after which the measurement (O1) of dependent variable is taken .

• EG : X O1• To measure effectiveness of a test commercial• TV programme containing test Ad (X) is selected• Telephone interviews are conducted of respondents who

report having watched the TV programme previous night• Respondents are asked whether they recall having seen the

Ad. Measurements of Dependent variable are recalls (O1)• Recalls are compared with norm scores for interpretation.

One Shot Design (After Only)

Drawbacks

• No random assignment of test units

• Extraneous variables are not controlled

• Internal validity low

• There is no other measurement against which O1 can be compared (what would happen if X was absent)

One Group Pretest -Posttest• Exposing an experimental group to experimental treatment (X) and

taking measurements of dependent variable before(O1) and after(O2) the treatment

• EG: O1 X O2• Difference between O1 and O2 measures the impact of treatment on

dependent variable• To measure effectiveness of a test commercial • Respondents are recruited on the basis of convenience• Administered a personal interview to measure attitude towards the

product• Then they watch a TV programme containing Commercial• After watching TV programme they are again administered a personal

interview to measure attitude towards product• Effectiveness of test commercial is measured as O2-O1• Influenced by maturation,testing ,mortality, selection bias affecting

IV

Static Group Design• Two groups of test units are involved EG &CG• EG is exposed to treatment CG is not exposed• Measurements are taken for both groups after Expt.• EG: X O1• CG: O2• Difference between O1& O2 is effect of treatment• To test effectiveness of TV commercial• Two groups of respondents are recruited • EG is exposed to TV programme containing Ad• Attitude towards product of both EG &CG are measured• Effectiveness of test Ad is measured as O1-O2• Selection bias ,mortality affects results with less IV

True Experimental Designs

---Use randomisation & one or more control groups to reduce effect of extraneous variables

Pre test – Post test Control Group Design

Post –test only Control Group Design

Solomon Four Group design

Pretest-Posttest Control Group Design

• Two groups of test units EG &CG are considered• Test units are assigned to these 2 groups randomly• Pre test measurements of dependent variable are

taken for both groups• EG is exposed to treatment• Post test measurements of dependent variable are

taken for both groups• EG: R O1 X O2• CG: R O3 O4 • Treatment Effect (TE) is (O2-O1)-(O4-O3)

Pre Test Post Test Control Group Design• To measure effectiveness of a TV commercial• Sample of respondents is selected at random• Half of these would be randomly assigned to EG & other half

would form CG• Both groups would be administered a questionnaire to obtain pre-

test measurement on attitude towards the product• Only EG would be exposed to TV programme containing

commercial• Then respondents in both groups would be administered a

questionnaire to obtain post test measurements on attitude towards product

• Effectiveness measure is (O2-O1)-(O4-O3) • This design provides accurate results & addresses most of

extraneous variables accept testing effects

Posttest test only Control Group Design

• Both EG & CG are formed by random assignment of test units

• EG is exposed to experimental treatment CG is not

• Post test measurement of dependent variable is taken for both the groups

• EG: R X O1

• CG: R O2

• Treatment Effect (TE) equals O1-O2

Posttest Only Control Group Design

• To measure effectiveness of a TV commercial• Sample of respondents is selected at random• Half of these would be randomly assigned to EG & other

half would form CG• Only EG would be exposed to TV programme containing

commercial• Then respondents in both groups would be administered a

questionnaire to obtain post test measurements on attitude towards product

• Difference in attitudes of EG &CG is used as effectiveness measure

Posttest Only Control Group Design

• Involves only 2 groups & one measurement

• Significant advantage in terms of time ,cost & sample size

• Selection bias & mortality which are the major drawbacks can be controlled through carefully designed experimental procedures

• Because of its simplicity it is most popular in marketing design

Solomon Four Group Design• It involves 4 groups ,2 EGs & 2 CGs• Six measurements are taken 2 pretest &4 posttest• Also known as 4 group 6 study design• EG: R O1 X O2• CG: R O3 O4• EG: R X O5• CG: R O6 • Design provides various observations which can be

analysed• (O2-O1) –(O4-O3) ; O5-O6• Addresses all extraneous variables but is expensive

& consumes time & efforts

Quasi Experimental Designs

• Are used when randomisation is not possible & timing of test presentation lacks control of researcher

• Not as effective as true designs but better than pre-design• Prominent is Time Series Design• A series of measurements are taken on dependent variable for

a group of test units before & after the experimental treatment• O1,O2,O3 X O4, O5, O6• Trends before treatment are compared with trends after

treatment to determine effectiveness of treatment• Aids in identifying permanent change & temporary change• Threats on account of history & instrumentation

•

Quasi Experimental Designs

• To test effectiveness of a test commercial• Broadcasting test commercial a predetermined number of

times & examining data from a pre-existing test panel• Scheduling of test commercial can be controlled but it is

uncertain when or whether panel members are exposed to it

• Purchases of panel members before ,during and after the campaign are examined to find out whether test commercial has a short term effect ,long term effect or no effect

• To find out impact of price change on sales of a product, a series of observations are taken before the price change &after the price change. Trends before treatment are compared with trends after treatment to determine change

Multiple Time Series Design

• Similar to time series except that another group of test units is added to serve as control group

• EG:O1,O2,O3 X O4,O5,O6

• CG:O7,O8,O9 O10,O11,O12

• Test commercial would be shown only in a few of test cities

• Panel members in these cities would form EG

• Panel members in the cities where commercial was not shown would constitute control group

One-Shot DesignInternal Validity Problems

• History– weak

• Maturation– weak

• Testing– not relevant

• Instrumentation– not relevant

• Selection– weak

• Mortality– weak

One-Group Pretest-PosttestInternal Validity Problems

• History– weak

• Maturation– weak

• Testing– weak

• Instrumentation– weak

• Selection– controlled

• Mortality– controlled

Static-Group DesignInternal Validity Problems

• History– controlled

• Maturation– possible source of

concern

• Testing– controlled

• Instrumentation– controlled

• Selection– weak

• Mortality– weak

Pretest-Posttest ControlInternal Validity Problems• History

– controlled

• Maturation– controlled





Posttest-Only ControlInternal Validity Problems

• History– controlled






Solomon Four-Group DesignInternal Validity Problems• History

– controlled






Statistical Designs

• Aids in measuring effect of more than one independent variable

• Helps in isolating effects of most extraneous variables

• Four prominent designs in this category

• -Completely Randomised

• -Randomised Block

• -Latin Square

• -Factorial

Advanced Experimental Designs are More Complex

• Completely randomized

• Randomized block design

• Latin square

• Factorial

Completely Randomized Design• An experimental design that uses a random process

to assign subjects (test units) and treatments to investigate the effects of only one independent variable (factor).

• There are n test units & k treatments (levels of factor)

• The n test units are assigned to k treatments randomly.

• Post test measurements are evaluated

Completely Randomized Design• 3 test cities are randomly exposed to 3 test

commercials

• Then measurements are taken on sales in these 3 cities

• Differences in sales in 3 cities are analyzed to see which AD is effective

Example: Home Products, Inc.Home Products, Inc. is considering marketing a long-lasting car wax. Three different waxes (Type 1, Type 2,and Type 3) have been developed.In order to test the durability of these waxes, 5 new cars were waxed with Type 1, 5 with Type 2, and 5 with Type 3. Each car was then repeatedly run through an automatic carwash until the wax coating showed signs of deterioration. The number of times each car went through the carwash is shown on the next slide.Home Products, Inc. must decide which wax to

market. Are the three waxes equally effective?

Example: Home Products, Inc.WAX WAX WAX

Observation Type 1 Type 2 Type 3

1 48 73 51

2 54 63 63

3 57 66 61

4 54 64 54

5 62 74 56

Sample Mean 55 68 57

Sample Variance 26.0 26.5 24.5

• Completely Randomized Design– ANOVA Table

Source of Sum of Degrees of Mean

Variation Squares Freedom Squares F

Treatments 490 2 245 9.55

Error 308 12 25.667

Total 798 14

Example: Home Products, Inc.

Example: Eastern Oil Co.• Completely Randomized Design

– Rejection Rule

Assuming = .05, F.05 = 3.88 (2 d.f. numerator and 12 d.f. denominator). Reject H0 if F > 3.88.

– Test Statistic

F = MSTR/MSE = 245/25.67= 9.55

– Conclusion

Since 9.55 > 3.88, we reject H0 and conclude that the miles per gallon ratings differ for the three gasoline blends.

Completely Randomized Design• To evaluate efficacy of a weight loss drug• A sample of 40 consumers is selected• Assigned randomly to two treatment levels• 25 to treatment 1 & 15 to treatment 2• Consumers in group 1 are asked to take drug for one

month• Consumers in group 2 are not given any drug• After experiment measurements are taken for both groups

& differences if any are analysed to find out effectiveness of the drug

• Applicable when test units are homogeneous• Extraneous variables can be controlled • A single variable is evaluated

Randomized Block Design

• An extension of the completely randomized design in which a single extraneous variable that might affect test units’ response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects.

Example Randomised Block Design

• A consumer product company wants to examine impact of price change on sales of its newly developed health drink

• Store type is extraneous variable which would also affect product sales

• Company has decided to conduct experiment using 3 price levels & 3 store types

• Nine retail outlets are segregated according to store type

• Treatment levels (prices) are assigned randomly to each test unit ( retail Outlet)

Example Randomised Block Design

Price Change

Drug Store Supermarkets Malls

Rs 10

Rs 20

Rs 30

Example: Eastern Oil Co.Eastern Oil has developed three new blends

of gasoline and must decide which blend or blends to produce and distribute. A study of the miles per gallon ratings of the three blends is being conducted to determine if the mean ratings are the same for the three blends.

Five automobiles have been tested using each of the three gasoline blends and the miles per gallon ratings are shown on the next slide.

Example: Eastern Oil Co.

Automobile Type of Gasoline (Treatment) Blocks (Block) Blend X Blend Y Blend Z Means

1 31 30 30 30.333 2 30 29 29 29.333 3 29 29 28 28.667 4 33 31 29 31.000 5 26 25 26 25.667

Treatment Means 29.8 28.8 28.4

• The ANOVA procedure for the randomized block design requires us to partition the sum of squares total (SST) into three groups: sum of squares due to treatments, sum of squares due to blocks, and sum of squares due to error.

• The formula for this partitioning is

SST = SSTR + SSBL + SSE

• The total degrees of freedom, n - 1, are partitioned such that k - 1 degrees of freedom go to treatments,

b - 1 go to blocks, and (k - 1)(b - 1) go to the error term.

The ANOVA Procedure

Example: Eastern Oil Co.

• Randomized Block Design– Mean Square Due to Treatments

The overall sample mean is 29. Thus,

SSTR = 5[(29.8 - 29)2 + (28.8 - 29)2 + (28.4 - 29)2] = 5.2

MSTR = 5.2/(3 - 1) = 2.6– Mean Square Due to Blocks

SSBL = 3[(30.333 - 29)2 + . . . + (25.667 - 29)2] = 51.33

MSBL = 51.33/(5 - 1) = 12.8 – Mean Square Due to Error

SSE = 62 - 5.2 - 51.33 = 5.47

MSE = 5.47/[(3 - 1)(5 - 1)] = .68

• Randomized Block Design– ANOVA Table

Source of Sum of Degrees of Mean

Variation Squares Freedom Squares F

Treatments 5.2 2 2.6 3.82

Block 51.3 4 12.8

Error 5.5 8 0.68

Total 62 14

Example: Home Products, Inc.

Example: Eastern Oil Co.• Randomized Block Design

– Rejection Rule

Assuming = .05, F.05 = 4.46 (2 d.f. numerator and 8 d.f. denominator). Reject H0 if F > 4.46.

– Test Statistic

F = MSTR/MSE = 2.6/.68 = 3.82– Conclusion

Since 3.82 < 4.46, we cannot reject H0. There is not sufficient evidence to conclude that the miles per gallon ratings differ for the three gasoline blends.

Factorial Design

• An experiment that investigates the interaction of two or more independent variables on a single dependent variable.

Factorial Design

• A 3x2 design has two factors first with 3 levels & second with 2 levels

• A 3x3 design has 2 factors each with 3 levels

• A 2x2x2 design has 3 factors each with 2 levels

• A 3x2x4 design has 3 factors with 3,2 7 4 levels respectively

Factorial Design• Researcher is investigating believability of Ads on 0-100

scale• Two magazine Ads A & B are to be compared• Gender of the reader is another factor• This is a 2x2 factorial experiment• Permits to test 3 hypothesis• Which Ad is more believable (Main)• Which gender tends to believe magazine Ads more(Main)• Which gender finds which ad more

believable(Interaction)

Men

Women

Ad A Ad B

65

65

70 60

Main Effectsof Gender

Main Effects of Ad

>

>

2 x 2 Factorial Design

Price Red Gold

$25 Cell 1 Cell 4$30 Cell 2 Cell 5$35 Cell 3 Cell 6

Package Design

Factorial Design -- Roller Skates

Latin Square Design

• A balanced, two-way classification scheme that attempts to control or block out the effect of two extraneous factors by restricting randomization with respect to the row and column effects.

Latin Square Example• To find out impact of three different Ads on sales• Pricing & income levels of consumers are EV• In latin square levels of all the 3 variables must be same

( 2EV and 1 INDEP)• A table is then developed with levels of one EV

representing rows ,levels of second EV representing columns .

• Levels of independent variable (treatments) are exposed to each cell on a random basis so that each row has all treatments & each column has all treatments

• Treatment effect is determined for each cell• Analysis shows which treatment level influences dependent

variable more

Latin Square Example

Pricing Levels

Low Income

Middle Income

High Income

Rs 10000 Ad-B Ad-A Ad-C

Rs 12000 Ad-C Ad-B Ad-A

Rs 14000 Ad-A Ad-C Ad-B

Latin Square

• A, B & C identify three treatments• Rows & Columns of the table identify

confounding factors (extraneous variables)• A taste test may be confounded by order of testing• A taste test may also be confounded by individual

test preference• To control for these 2 factors each subject is

exposed to every treatment & order in which subjects test is randomised.

1 2 3

1 A B C2 B C A3 C A B

Order of UsageS

UB

JEC

T

Latin Square

• Interaction effects are assumed to be minimal or absent

• Number of treatment levels for both confounding factors must be equal

• Suppose a retail grocery chain wishes to control for shelf space & city where product is sold

• It markets in only 3 cities & wishes to experiment with 4 levels of shelf space

• Having unequal number of levels for each factor will eliminate latin square as a possibility

TEST MARKETING

Controlled experimentationControlled experimentation

Not just tryingNot just tryingsomethingsomethingoutout

But scientificBut scientifictestingtesting

Test Marketing

• An experimental procedure that provides an opportunity to test a new product or a new marketing plan under realistic market conditions to measure sales or profit potential.

Selecting a Test Market

• Population size

• Demographic composition

• Lifestyle considerations

• Competitive situation

• Media

• Self-contained trading area

• Secrecy

Control Method of Test Marketing

• Small city

• Low chance of being detected

• Distribution is forced (guaranteed)

Documents

Business Research Method