Class 13

Marketing Analytics

CBC, Sarah, COPD, and CBC

Stevens and

Darden09

Charles

book club

How to do Response Modeling

1. Test something using n names. Keep track

of Xs and RESPONSE (1/0).

2. Use the n names to build a model that

predicts RESPONSE.

3. Use that model to score new names (for

which you know the Xs).

4. Mail to the top scoring names.

Where to draw the cut depends on the

economics.

1. Test something using n names

2. Use the n names to build a

model to predict response.

We tested our

mailing on

4,000 names.

We have

several X

variables.

We will use

regression to

forecast

FLORENCE.

Regression of Florence on

Related Purchase

SUMMARY OUTPUT

Regression Statistics

Multiple R 0.107248523

R Square 0.011502246

Adjusted R Square 0.011172527

Standard Error 0.266713758

Observations 3000

ANOVA

df SS MS F

Regression 1 2.481586485 2.481586485 34.88498803

Residual 2998 213.2664135 0.071136229

Total 2999 215.748

Coefficients Standard Error t Stat P-value

Intercept 0.057089558 0.006020467 9.482579519 4.87592E-21

Related Purchase 0.023486083 0.003976411 5.906351499 3.89041E-09

High t

and low

p!

Forecast Score =

0.057 +

0.0235*Related

Purchase

3. Use the model to score the

new names.

High scores

mean likely

to buy

Florence.

4. Mail to top scoring names.

Mail if score > .1 because we need at least a

.1 response rate to make money given cost

=$1 and response is worth $10

Because the model is so simple, this is the

same as mailing to all those with related

purchases > =2.

Mailing to

Related

Purchases >=2

achieves $214

Cap One Product Design

Lets here what you did!

How you Did

TEST all cells with 4K $21,000.00 $375,000.00 $397,600.00 22380 $1,159,555.00 $761,955.00

TEST all cells with 3K $21,000.00 $375,000.00 $397,600.00 22904 $1,188,169.00 $790,569.00

TEST all cells with 2K $21,000.00 $375,000.00 $397,600.00 17376 $990,948.00 $593,348.00

TEST all cells with 1K $21,000.00 $375,000.00 $397,600.00 19542 $965,827.00 $568,227.00

Click on the team names below to view more detailed results.

Team Name: Rounds

Solicit.& Devlop Cost:

Cost of Pieces Mailed:

Total Mailing Costs:

Total No. of Responses

Total Response

Value Total Profit: Score

27-May 2 $15,000.00 $375,000.00 $391,600.00 12936 $756,404.00 $364,804.00 100

Team Eldrick 2 $21,000.00 $218,600.00 $241,200.00 17867 $522,874.00 $281,674.00 77.2

Pepe Nepveux 2 $18,000.00 $375,000.00 $394,600.00 9270 $652,714.00 $258,114.00 70.8

Nanners 2 $21,000.00 $47,500.00 $70,100.00 2949 $93,608.00 $23,508.00 60.0

Tiger 1 $16,000.00 $6.00 $16,806.00 0 $0.00 ($16,806.00) 60.0

Cap One Product Design

Dont rely on regression of exhibit 2 data.

Things have changed.

BK score is an average

Test most cells and roll out the HIGHEST

VALUE cell in each column.

Total value is responses*their value

You should not ignore the fact that the value of response

depends on the cell.

If you tested all cells equally, roll out the cell in each

column that created the most value.

TESTING STRATEGIES

Test only the best Very Risky

Case Data are not that relevant the environment has changed

only AVERAGE BK score is available

Design an Experiment Test a carefully selected subset of cells

Use the results to build a model to forecast all 36 cells

Roll out the cells with best forecasted profit

TEST all 36 Cells--roll out the best testing cells the safest strategy.

Death Wish Marketing

The failure to develop and test several

marketing options is a form of death

wish marketing Clancey and Krieg, Counter-Intuitive Marketing, NY: Free Press, 2000 (quoted in Lynn

and Lynn, Experiments and Quasi-Experiments: Tools for Evaluating Marketing

Options, WP No. 03-18-03, The center for Hospitality Research.)

In his memoirs, David Ogilvy says he succeeding in

advertising because he was always ready to run a few

ads he deemed to be losers. Invariably, some were

big hits, leading him to revise his theories. (Russo and Schoemaker, Decision Traps)

Change the mindset

Ask How would we test this?

Ask, why not test this?

Get excited about testing it!

They test, why dont you?Dance with Chance, Makridakis, Hogarth, and Gaba

In 90s Swedish doctors implanted 81 pace

makers...but only turned half of them on!

Every patient experienced improvement.

1,103 heart attack victims given the potent

drug..2,789 given a placebo

20% death rate for drug, 21% for placebo.

In both groups, those who were diligent with

their meds lived longer than those who did

not.

Government by chanceSupercrunchers, Ian Ayers

Piggyback on other Random Processes

NH kids applying to magnet schools were

chosen by lottery to attend.

Thats all we need to test the efficacy of magnet

schools! (p 73)

Since 1998 in India, 1/3 of villages were

assigned a female chief (Pradhan) at random

Sarah Gets a Diamond Exercise

6,000 Diamonds

in the

training

set

3,142

Diamonds

in the

test set

Advice for Sarah

Use ln(price) as your dependent variable

Create a column labeled lnprice using =ln(). Thereafter think of

this new variable as your dependent variable.

Convert your forecasts of ln(Price) back to prices by using =exp().

Be sure you do this before sending me your price forecasts

Use ln(carat weight) as a predictor variable

Use either numbers (1 to 5?) or sets of dummy variables

for the other characteristics.

Consider using several regression models..not just one for

all diamonds.

What we did

Pepe and Nanners both used the ln ln model

and got a mape of 20.7

Team Eldrick included numerical values for

the other Cs and did much better?

Team EDI is a professional data mining

firm.

Regression Accounts for correlated

XsAverage Price Count

CUTSignature-Ideal $11,541.53 253

Ideal $13,127.33 2,482

Very Good $11,484.70 2,428

Good $9,326.66 708

Fair $5,886.18 129

COLORD $15,255.78 661

E $11,539.19 778

F $12,712.24 1,013

G $12,520.05 1,501

H $10,487.35 1,079

I $8,989.64 968

CLARITYFL $63,776.00 4

IF $22,105.84 219

VVS1 $16,845.68 285

VVS2 $14,142.18 666

VS1 $13,694.11 1,192

VS2 $11,809.05 1,575

SI1 $8,018.86 2,059

TOTAL $11,791.58 6,000

Why do SI

diamonds

have lower

avg price than

Ideal cut

diamonds?

Because sig ideal are

likely to be smaller.

Regression can

handle this?

Stevens Sarah Results

TEAM Pepe "May27"Team

Eldrick TIGER Nanners

3142 3142 3142 3142 3142

MAPE 20.07% 33.17% 9.41% 21.94% 20.07%

SCORE 80 75 100 80 80

Eldrick gets the 100. Pepe, Tiger,

nanners all did about the same.

May27 had a technical error.

Colonial Broadcasting Company

Please read the case

Any questions about the case?

Use the regression results to

answer these questions

Which of the three networks had the highest

rated TV movies in 1992?

Regression 1 tells us that ABN had an average

rating of 13.363+1.397 = 14.76

What was the 1992 average rating of TV

movies from CBC?

Regression 1 says 13.363!

Regression with dummy

variables goes thru the group

averages.

Use the regression results to

answer these questions

Conventional Wisdom says that FACT based

movies do better. What do the data tell us?

Regression 2 tells us that FACT movies beat

FICTION movies by 1.4 points (on average) in

1992.

How strong is the evidence?

The result is statistically significant. The t was 2.6

and the p was .01. It did not happen by chance.

Use the regression results to

answer these questions If we expect 1993 results to be similar to those

in 1992, what are the chances that a randomly chosen CBC TV movie will get a rating greater than 15?

Regression 1 says a CBC rating will have mean 13.363 and standard deviation of 2.42.

Probability the rating will be less than 15 is NORMDIST(15,13.363,2.42,true) = 0.750.

The probability the rating will greater than 15 is 0.25.

Use the regression results to

answer these questions Regression 2 says that FACT based movies are rated

higher by 1.4 points (on average).

Regression 3 says that FACT movies are rated 1.8 points higher (on average).

What the heck is going on? FACT and STARS are correlated in our data.

FACT movies had either more or fewer STARS (on average) than FICTION movies.

Since STARS improve the rating, then FACT based movies must have had fewer STARS..that explains why FACT beat FICTION by only 1.4. For a given number of STARS, FACT beats fiction by 1.8.

Use the regression results to

answer these questions

If we know whether the movie is FACT and how many STARS it has, does it also help to know (if we are trying to predict the rating) the competition rating?

YES. The t for COMPETION in regression 4 is -2.3 and the p is 0.03.

The negative sign just means that the higher the competition the lower is the rating expected to be. That makes total sense.

Final QUIZ

930 to 1130. Thursday, April 29.

Open book and notes.

No searching the internet or each other for

help with specific questions.

All material used in the course is usable.

Ill gladly give a help sessionjust let me

know when and where and how many.