Media Mix Modeling - Predictive Analytics · PDF fileMedia Mix Modeling Objectives ... Lasso Variable Selection Summary Analysis of Variance Source DF F Value Model 4 2.61132 0.65283

Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011

Predictive Analytics World 2011

Media Mix Modeling

Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011

Media Mix Modeling

Objectives

Demonstrate some of the commonly used techniques and methodologies used to estimate the impacts of media spend.

Illustrate some of the most frequently encountered problems.

Reference some of the newer Econometric techniques incorporated into SAS/ETS and Base Stat.

Caveats

It is not possible to provide an extensive catalog in the time provided.

There are far more techniques and challenges than those listed here.



1 3 5 7 9 111315171921232527293133353739414345474951 1 3 5 7 9 111315171921232527293133353739414345474951Year 1 2

$0

$500,000

$1,000,000

$1,500,000

$2,000,000

$2,500,000

$3,000,000

$3,500,000

Simulated Media Spend Data

Television_Spend Radio_Spend Newspaper_Spend Direct_Mail_Spend Digital_Spend

A Number Of Data Concerns Exist



Data Concerns

Stationarity !   A simple “working definition” of stationarity is a process whose mean, variance and autocorrelation structures do not vary over time.

!   If variables in a regression model are not stationary, standard asymptotic assumptions are not valid (e.g. t-statistics will not follow a t distribution).

!   Generally, regression models with non-stationary predictors (that are not differenced) yield spurious (even nonsensical) results.

Cointegration !   If a stationary linear combination of non-stationary regressors exists, these regressors are said to be cointegrated. !   “Long run” and “short run” dynamic relationships exist amongst cointegrated regressors. !   Generally, regression models that properly account for cointegrated predictors will not yield spurious results



Data Concerns

Exogeneity !   Currently exogeneity is defined in terms of weak, strong and super exogeneity.

!   A regressor is said to be weakly exogenous if inference on the regression parameter estimates conditional upon the regressor involves no loss of information. If weak exogeneity does not hold the model's dynamic parameter estimates are inefficient. !   A regressor is said to be super exogenous if it is weakly exogenous and the regression parameter estimates do not change when changes in the regressor's distribution occur.

!   A regressor is said to be strongly exogenous if it is weakly exogenous and the regressor is not preceded by an endogenous variable (in the model formulation).



Data Concerns

GARCH !   GARCH – Generalized Autoregressive Conditional Heteroscedasticity.

!   The variance of the current error term (or innovation) is a function of the size of the previous period's error term (or innovation).

!   Primarily used in variance modeling and may not necessarily improve forecasts.

Multicollinearity !   Largely a question of degree or severity.

!   If severe multicollinearity exist, the variance estimates are inflated and the following may be observed: imprecise (or implausible) and unstable parameter estimates, a very high r-squared with statistically insignificant predictors, incorrect coefficient signs.



Data Concerns

Autocorrelation !   When model residuals are correlated, parameter estimates are inefficient, t-statistics and r-squared values are upwardly biased.

!   Autocorrelation can be positive or negative.

!   First order autocorrelation is the most common variant.

!   Common causes for autocorrelation include observations being present in multiple time periods and omitted variables.



Multicollinearity



Multicollinearity

proc reg data=mme.simulated_base;

Title1 'Main and Interaction Effects -- Multicollinearity Demonstration';

where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital

LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif;

output out=p1 p=py r=residual;

run;

quit;

Year 1 sales are regressed against Direct Mail, Television, Radio Newspaper and Digital Spend levels.

A log-log functional form was assumed to enable easy elasticity estimates.



Multicollinearity

Parameter Estimates

Variable DF t Value Pr > |t|Intercept 1 -1.33934 163.23107 -0.01 0.9935 0holiday 1 3.61428 10.18642 0.35 0.7245 81880

Log_DM 1 0.00484 0.00279 1.73 0.0904 1.45394Log_TV 1 1.11802 11.78288 0.09 0.9249 51846

Log_Radio 1 0.20417 0.14413 1.42 0.1642 13.56417Log_Paper 1 -1.48058 13.72226 -0.11 0.9146 68041Log_Digital 1 2.73806 5.83569 0.47 0.6414 22776LogTVPaper 1 0.09406 0.9969 0.09 0.9253 253746LogTVDigital 1 -0.21443 0.43017 -0.5 0.6208 55934LogTVHoldy 1 -0.53949 0.6456 -0.84 0.4082 65908

LogRadioHoldy 1 0.36448 0.38295 0.95 0.3468 19923

ParameterEstimate

StandardError

VarianceInflation

Even though none of the regressors are statistically significant at the 5% confidence level, the Adjusted R-square is .8453.

Only Direct Mail had a variance inflation value less than 10.

Many of the coefficient signs are reversed.



Multicollinearity

proc reg data=mme.simulated_base outvif

outest=b ridge=0 to 0.40 by 0.02;


Title2 'Ridge Regression';

where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif noprint;

plot /ridgeplot;

output out=p1 p=py r=residual;

run;

quit;

proc print data=b;run;

Ridge Regression Code.



Multicollinearity

Ridge Plots



Multicollinearity

Regression Plots



Multicollinearity

proc score data=mme.simulated_base score=b(where=(_RIDGE_=0.04)) out=p2

type=RIDGE;

var Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy;

run;

proc print data=p1;run;

proc print data=p2;run;

Ridge Regression Code Continued.

A variable selection mechanism is missing.

Each regressor is included (“considered statistically significant”).



Multicollinearity

GLM Select (Lasso).

proc glmselect data=mme.simulated_base plots=all;


Title2 'GLM Select -- Lasso';

where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital

LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy

/details=all stats=all

selection=lasso;

*modelAverage nsamples=1000 subset(best=1);

run;



Multicollinearity

Step SBC ASE F Value Pr > F0 Intercept 1 0 0 -144.3294 0.0578 0 11 LogRadioHoldy 2 0.5747 0.5662 -184.8367 0.0246 67.57 <.00012 Log_Radio 3 0.8251 0.8179 -227.0783 0.0101 70.12 <.00013 Log_DM 4 0.8569 0.8479 -233.5702 0.0083 10.68 0.0024 Log_Digital 5 0.8695 0.8584* -234.4167* 0.0075 4.54 0.0383

EffectEntered

EffectRemoved

NumberEffects In

ModelR-Square

Adjusted R-Square

Lasso Variable Selection Summary

Analysis of Variance

Source DF F ValueModel 4 2.61132 0.65283 78.29Error 47 0.39192 0.00834

51 3.00324

Parameter EstimatesParameter DF EstimateIntercept 1 10.424565Log_DM 1 0.003916

Log_Radio 1 0.228549Log_Digital 1 -0.126448

LogRadioHoldy 1 0.037262

Sum ofSquares

MeanSquare

Corrected Total

Lasso Anova and parameter estimates



Multicollinearity




Multicollinearity




Multicollinearity




Stationarity



Stationarity

proc varmax data=mme.simulated_base plots=(impulse) outest=est outstat=stat;

where year = 1;

nloptions tech=newrap maxiter=5000000000 maxfunc=5000000000;

model Log_Sales = Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 cointtest=(sw) /*dify=(1) difx=(1)*/ p=1 q=1;

output out=out lead=5;

causal group1=(Log_Sales) group2=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM);

causal group1=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM) group2=(Log_Sales);

run;

Vector Autoregression Code (model assumed to be ARMAX(1,1,0))

This model is testing for the need to difference the data.



Stationarity


Dickey-Fuller Unit Root TestsVariable Type Rho Pr < Rho Tau Pr < Tau

Log_Sales Zero Mean 0.04 0.6873 0.76 0.8742Single Mean -2.55 0.7028 -0.88 0.7866

Trend -6.49 0.6818 -1.74 0.7176

Model Parameter Estimates

Equation Parameter Estimate t Value Pr > |t| VariableLog_Sales CONST1 -0.93697 0.90437 -1.04 0.3054 1

XL0_1_1 0.40894 0.08564 4.78 0.0001 Log_TV(t)XL0_1_2 0.0546 0.05846 0.93 0.355 Log_Digital(t)XL0_1_3 0.00444 0.00217 2.05 0.0462 Log_DM(t)XL0_1_4 0.10212 0.1133 0.9 0.3719 L1_Radio(t)XL0_1_5 0.35952 0.14147 2.54 0.0143 L3_Paper(t)AR1_1_1 0.06687 0.24856 0.27 0.7891 Log_Sales(t-1)MA1_1_1 0.14939 0.33238 0.45 0.6551 e1(t-1)

StandardError

Dickey-Fuller Tests indicated model should be differenced

The model has an R-square value of .9027



Stationarity


where year = 1;


model Log_Sales = Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 cointtest=(sw) dify=(1) difx=(1) p=1 q=1;




run;


This model is estimated in first differences



Stationarity


Dickey-Fuller Tests indicated the model is “fully differenced



Log_Sales Zero Mean -75.28 <.0001 -5.44 <.0001Single Mean -81.28 0.0004 -5.51 0.0001

Trend -89.13 <.0001 -5.46 0.0003


Equation Parameter Estimate t Value Pr > |t| VariableLog_Sales CONST1 0.00224 0.00096 2.33 0.0241 1

XL0_1_1 0.35504 0.08953 3.97 0.0002 Log_TV(t)XL0_1_2 0.07366 0.05414 1.36 0.1802 Log_Digital(t)XL0_1_3 0.00527 0.00198 2.66 0.0107 Log_DM(t)XL0_1_4 0.15012 0.08453 1.78 0.0822 L1_Radio(t)XL0_1_5 0.40703 0.09935 4.1 0.0002 L3_Paper(t)AR1_1_1 -0.10757 0.1264 -0.85 0.3991 Log_Sales(t-1)MA1_1_1 0.98452 0.05045 19.52 0.0001 e1(t-1)

StandardError



Model Diagnostics Durbin Watson does not indicate autocorrelations

The model order does not appear to have an autoregressive error

Univariate Model White Noise Diagnostics

VariableNormality ARCH

Chi-Square Pr > ChiSq F Value Pr > FLog_Sales 2.06684 3.35 0.1872 1.54 0.2212

DurbinWatson

ARCH test statistic does not indicate heteroscedasticity

Univariate Model AR Diagnostics

VariableAR1 AR2 AR3 AR4

F Value Pr > F F Value Pr > F F Value Pr > F F Value Pr > FLog_Sales 0.25 0.6185 0.51 0.6043 0.59 0.6249 1.59 0.1962

Model residuals appear to be normally distributed



Stationarity

Vector Autoregression Code (model assumed to be MAX(1,0))

Dickey-Fuller Tests results are unchanged



Log_Sales Zero Mean -75.28 <.0001 -5.44 <.0001Single Mean -81.28 0.0004 -5.51 0.0001

Trend -89.13 <.0001 -5.46 0.0003


Equation Parameter Estimate t Value Pr > |t| VariableLog_Sales CONST1 0.00188 0.00082 2.29 0.0267 1

XL0_1_1 0.36226 0.09098 3.98 0.0002 Log_TV(t)XL0_1_2 0.06322 0.05588 1.13 0.2636 Log_Digital(t)XL0_1_3 0.00491 0.00199 2.47 0.0173 Log_DM(t)XL0_1_4 0.11312 0.07101 1.59 0.1179 L1_Radio(t)XL0_1_5 0.36666 0.08721 4.2 0.0001 L3_Paper(t)MA1_1_1 0.99856 0.05334 18.72 0.0001 e1(t-1)

StandardError



Model Diagnostics Durbin Watson does not indicate autocorrelations

The model order does not appear to have an autoregressive error

ARCH test statistic does not indicate heteroscedasticity

Model residuals appear to be normally distributed


VariableNormality ARCH

Chi-Square Pr > ChiSq F Value Pr > FLog_Sales 2.27232 1.6 0.4499 1.03 0.3152

DurbinWatson

Univariate Model AR Diagnostics

VariableAR1 AR2 AR3 AR4

F Value Pr > F F Value Pr > F F Value Pr > F F Value Pr > FLog_Sales 1.08 0.3035 0.99 0.3784 0.76 0.5207 1.58 0.1983



Forecasts



Forecasts

Error=(1-(Forecast/Sales))



Cointegration



Cointegration



model Log_Sales Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 p=4 cointtest=(johansen=(normalize=Log_Sales));

cointeg rank=4 normalize=Log_TV exogeneity;




run;

Vector Error Correction Model (VECM) Code

This model is estimated in levels.

Estimating a VECM with differenced data results in a lost of information.



Cointegration

Granger Causality

Sales drives (Granger causes) advertising.

Test 1: Group 1 Variables: Log_SalesGroup 2 Variables: Log_TV L1_Radio L3_Paper Log_Digital Log_DM

Test 2: Group 1 Variables: Log_TV L1_Radio L3_Paper Log_Digital Log_DMGroup 2 Variables: Log_Sales

Advertising drives (Granger causes) sales.

Granger-Causality Wald Test

Test DF1 20 54.61 <.00012 20 32.01 <.0432

Chi-Square

Pr > ChiSq



Cointegration

Johansen Rank Test

Cointegration Rank Test Using Trace

Trace0 0 0.4891 188.8512 93.92 Constant Linear1 1 0.3978 123.7144 68.682 2 0.271 74.5222 47.213 3 0.2289 43.8567 29.384 4 0.113 18.6465 15.345 5 0.0697 7.012 3.84

H0: Rank=r

H1: Rank>r

Eigenvalue

5% Critical Value

Drift in ECM

Drift in Process

Cointegration Rank Test Using Trace Under Restriction

Trace0 0 0.4892 189.2263 101.84 Constant Constant1 1 0.3984 124.0615 75.742 2 0.2712 74.7639 53.423 3 0.2293 44.0806 34.84 4 0.1138 18.8118 19.995 5 0.0705 7.0963 9.13

H0: Rank=r

H1: Rank>r

Eigenvalue

5% Critical Value

Drift in ECM

Drift in Process



Cointegration

Johansen Rank Test

Hypothesis of the Restriction

HypothesisH0(Case 2) Constant ConstantH1(Case 3) Constant Linear

Drift in ECM

Drift in Process

Hypothesis Test of the Restriction

Rank Eigenvalue DF0 0.4891 0.4892 6 0.38 0.9991 0.3978 0.3984 5 0.35 0.99672 0.271 0.2712 4 0.24 0.99333 0.2289 0.2293 3 0.22 0.97364 0.113 0.1138 2 0.17 0.92065 0.0697 0.0705 1 0.08 0.7715

RestrictedEigenvalue

Chi-Square

Pr > ChiSq



Cointegration

Model Diagnostics


Variable

Normality ARCH

Chi-Square F Value Pr > FLog_Sales 2.02311 3.86 0.145 0.04 0.8432Log_TV 2.027 76.5 <.0001 0.01 0.9315

Log_Digital 2.00815 13.3 0.0013 0.06 0.8026Log_DM 1.89245 1.9 0.387 0.6 0.441L1_Radio 2.05207 1.51 0.4708 0.02 0.8891L3_Paper 2.01872 0.74 0.6907 0.46 0.4999

DurbinWatson

Pr > ChiSq

There is no evidence of heteroscedasticity

Only the residuals for Television and Digital Spend are not normally distributed



Cointegration

Model Diagnostics

Univariate Model ANOVA Diagnostics

Variable R-Square F Value Pr > FLog_Sales 0.3979 0.11948 1.98 0.0139Log_TV 0.2892 0.16767 1.22 0.2547

Log_Digital 0.499 0.1971 2.99 0.0002Log_DM 0.4996 3.33832 3 0.0002L1_Radio 0.6721 0.10855 6.15 <.0001L3_Paper 0.7234 0.06408 7.85 <.0001

StandardDeviation

Except for television spend, each of the models is statistically significant.



Cointegration

Weak Exogeneity

Amongst the six regressors, only Television Spend appears to be weakly exogenous

Testing Weak Exogeneity of Each Variables

Variable DF Chi-SquareLog_Sales 4 11.01 0.0265Log_TV 4 5.72 0.2212

Log_Digital 4 23.18 0.0001Log_DM 4 44.98 <.0001L1_Radio 4 14.83 0.0051L3_Paper 4 20.18 0.0005

Pr > ChiSq



VECM Forecasts



VECM Forecasts

Error=(1-(Forecast/Sales))



Impulse Response Functions




































Thank You


Feel free to contact me at [email protected].

Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011


Documents

Media Mix Modeling - Predictive Analytics · PDF fileMedia Mix Modeling Objectives ... Lasso Variable Selection Summary Analysis of Variance Source DF F Value Model 4 2.61132 0.65283