Media Mix Modeling

Predictive Analytics World 2011

Objectives

Demonstrate some of the commonly used techniques and methodologies used to estimate the impacts of media spend. Illustrate some of the most frequently encountered problems. Reference some of the newer Econometric techniques incorporated into SAS/ETS and Base Stat. Caveats

It is not possible to provide an extensive catalog in the time provided.

There are far more techniques and challenges than those listed here.

Predictive Analytics World 2011

Simulated Media Spend Data

$3,500,000

$3,000,000

$2,500,000

$2,000,000

$1,500,000

$1,000,000

$500,000

$0 1 3 5 7 9 111315171921232527293133353739414345474951 1 3 5 7 9 111315171921232527293133353739414345474951 Year 1 2

Television_Spend Radio_Spend Newspaper_Spend Direct_Mail_Spend Digital_Spend

Predictive Analytics World 2011

A simple “working definition” of stationarity is a Stationarity process whose mean, variance and autocorrelation structures do not vary over time.

If variables in a regression model are not stationary, standard asymptotic assumptions are not valid (e.g. t-statistics will not follow a t distribution).

Generally, regression models with non-stationary predictors (that are not differenced) yield spurious (even nonsensical) results.

If a stationary linear combination of non-stationary Cointegration regressors exists, these regressors are said to be cointegrated. “Long run” and “short run” dynamic relationships exist amongst cointegrated regressors. Generally, regression models that properly account for cointegrated predictors will not yield spurious results Predictive Analytics World 2011

Currently exogeneity is defined in terms of weak, Exogeneity strong and super exogeneity.

A regressor is said to be weakly exogenous if inference on the regression parameter estimates conditional upon the regressor involves no loss of information. If weak exogeneity does not hold the model's dynamic parameter estimates are inefficient. A regressor is said to be super exogenous if it is weakly exogenous and the regression parameter estimates do not change when changes in the regressor's distribution occur.

A regressor is said to be strongly exogenous if it is weakly exogenous and the regressor is not preceded by an endogenous variable (in the model formulation).

Predictive Analytics World 2011

GARCH – Generalized Autoregressive GARCH Conditional Heteroscedasticity.

The variance of the current error term (or innovation) is a function of the size of the previous period's error term (or innovation).

Primarily used in variance modeling and may not necessarily improve forecasts. Multicollinearity Largely a question of degree or severity. If severe multicollinearity exist, the variance estimates are inflated and the following may be observed: imprecise (or implausible) and unstable parameter estimates, a very high r-squared with statistically insignificant predictors, incorrect coefficient signs.

Predictive Analytics World 2011

When model residuals are correlated, parameter Autocorrelation estimates are inefficient, t-statistics and r-squared values are upwardly biased.

Autocorrelation can be positive or negative.

First order autocorrelation is the most common variant.

Common causes for autocorrelation include observations being present in multiple time periods and omitted variables.

Predictive Analytics World 2011

proc reg data=mme.simulated_base; Title1 'Main and Interaction Effects -- Multicollinearity Demonstration'; where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif; output out=p1 p=py r=residual;

run; quit;

Year 1 sales are regressed against Direct Mail, Television, Radio Newspaper and Digital Spend levels.

A log-log functional form was assumed to enable easy elasticity estimates.

Predictive Analytics World 2011

Parameter Estimates Parameter Standard Variance Variable DF Es ti mate Er r or t Value Pr > |t| Inflation Intercept 1 -1.33934 163.23107 -0.01 0.9935 0 holiday 1 3.61428 10.18642 0.35 0.7245 81880 Log_DM 1 0.00484 0.00279 1.73 0.0904 1.45394 Log_TV 1 1.11802 11.78288 0.09 0.9249 51846 Log_Radio 1 0.20417 0.14413 1.42 0.1642 13.56417 Log_Paper 1 -1.48058 13.72226 -0.11 0.9146 68041 Log_Digital 1 2.73806 5.83569 0.47 0.6414 22776 LogTVPaper 1 0.09406 0.9969 0.09 0.9253 253746 LogTVDigital 1 -0.21443 0.43017 -0.5 0.6208 55934 LogTVHoldy 1 -0.53949 0.6456 -0.84 0.4082 65908 LogRadioHoldy 1 0.36448 0.38295 0.95 0.3468 19923

Even though none of the regressors are statistically significant at the 5% confidence level, the Adjusted R-square is .8453.

Only Direct Mail had a variance inflation value less than 10.

Many of the coefficient signs are reversed. Predictive Analytics World 2011

Ridge Regression Code.

proc reg data=mme.simulated_base outvif

outest=b ridge=0 to 0.40 by 0.02;

Title1 'Main and Interaction Effects -- Multicollinearity Demonstration';

Title2 'Ridge Regression';

where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif noprint;

plot /ridgeplot;

output out=p1 p=py r=residual;

run;

quit;

proc print data=b;run;

Predictive Analytics World 2011

Ridge Plots

Predictive Analytics World 2011

Regression Plots

Predictive Analytics World 2011

Ridge Regression Code Continued.

proc score data=mme.simulated_base score=b(where=(_RIDGE_=0.04)) out=p2

type=RIDGE;

var Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy;

run;

proc print data=p1;run;

proc print data=p2;run;

A variable selection mechanism is missing.

Each regressor is included (“considered statistically significant”).

Predictive Analytics World 2011

GLM Select (Lasso).

proc glmselect data=mme.simulated_base plots=all;

Title1 'Main and Interaction Effects -- Multicollinearity Demonstration';

Title2 'GLM Select -- Lasso';

where year = 1;

model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital

LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy

/details=all stats=all

selection=lasso;

*modelAverage nsamples=1000 subset(best=1);

run;

Predictive Analytics World 2011

Lasso Variable Selection Summary

Effect Effect Number Model Adjusted Step Entered Removed Effects In R-Square R-Square SBC ASE F Value Pr > F 0 Intercept 1 0 0 -144.3294 0.0578 0 1 1 LogRadioHoldy 2 0.5747 0.5662 -184.8367 0.0246 67.57 <.0001 2 Log_Radio 3 0.8251 0.8179 -227.0783 0.0101 70.12 <.0001 3 Log_DM 4 0.8569 0.8479 -233.5702 0.0083 10.68 0.002 4 Log_Digital 5 0.8695 0.8584* -234.4167* 0.0075 4.54 0.0383

Lasso Anova and parameter estimates

Analysis of Variance Sum of Mean Source DF Squares Square F Value Model 4 2.61132 0.65283 78.29 Er r or 47 0.39192 0.00834 Corrected Total 51 3.00324

Parameter Estimates Parameter DF Es ti mate Intercept 1 10.424565 Log_DM 1 0.003916 Log_Radio 1 0.228549 Log_Digital 1 -0.126448 LogRadioHoldy 1 0.037262 Predictive Analytics World 2011

Lasso Variable Selection Summary

Predictive Analytics World 2011

Lasso Variable Selection Summary

Predictive Analytics World 2011

Lasso Variable Selection Summary

Predictive Analytics World 2011

Vector Autoregression Code (model assumed to be ARMAX(1,1,0))

proc varmax data=mme.simulated_base plots=(impulse) outest=est outstat=stat;

where year = 1;

nloptions tech=newrap maxiter=5000000000 maxfunc=5000000000;

model Log_Sales = Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 cointtest=(sw) /*dify=(1) difx=(1)*/ p=1 q=1;

output out=out lead=5;

causal group1=(Log_Sales) group2=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM);

causal group1=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM) group2=(Log_Sales);

run;

This model is testing for the need to difference the data.

Predictive Analytics World 2011

Vector Autoregression Code (model assumed to be ARMAX(1,1,0))

Dickey-Fuller Unit Root Tests Variable Type Rho Pr < Rho Tau Pr < Tau Log_Sales Zero Mean 0.04 0.6873 0.76 0.8742 Single Mean -2.55 0.7028 -0.88 0.7866 Trend -6.49 0.6818 -1.74 0.7176

Dickey-Fuller Tests indicated model should be differenced

Model Parameter Estimates Standard Equati on Parameter Es ti mate Er r or t Value Pr > |t| Variable Log_Sales CONST1 -0.93697 0.90437 -1.04 0.3054 1 XL0_1_1 0.40894 0.08564 4.78 0.0001 Log_TV(t) XL0_1_2 0.0546 0.05846 0.93 0.355 Log_Digital(t) XL0_1_3 0.00444 0.00217 2.05 0.0462 Log_DM(t) XL0_1_4 0.10212 0.1133 0.9 0.3719 L1_Radio(t) XL0_1_5 0.35952 0.14147 2.54 0.0143 L3_Paper(t) AR1_1_1 0.06687 0.24856 0.27 0.7891 Log_Sales (t-1) MA1 _ 1 _ 1 0.14939 0.33238 0.45 0.6551 e1(t-1)

The model has an R-square value of .9027

Predictive Analytics World 2011

Vector Autoregression Code (model assumed to be ARMAX(1,1,0))

proc varmax data=mme.simulated_base plots=(impulse) outest=est outstat=stat;

where year = 1;

nloptions tech=newrap maxiter=5000000000 maxfunc=5000000000;

model Log_Sales = Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 cointtest=(sw) dify=(1) difx=(1) p=1 q=1;

output out=out lead=5;

causal group1=(Log_Sales) group2=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM);

causal group1=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM) group2=(Log_Sales);

run;

This model is estimated in first differences

Predictive Analytics World 2011

Vector Autoregression Code (model assumed to be ARMAX(1,1,0))

Dickey-Fuller Unit Root Tests Variable Type Rho Pr < Rho Tau Pr < Tau Log_Sales Zero Mean -75.28 <.0001 -5.44 <.0001 Single Mean -81.28 0.0004 -5.51 0.0001 Trend -89.13 <.0001 -5.46 0.0003

Dickey-Fuller Tests indicated the model is “fully differenced

Model Parameter Estimates Standard Equati on Parameter Es ti mate Er r or t Value Pr > |t| Variable Log_Sales CONST1 0.00224 0.00096 2.33 0.0241 1 XL0_1_1 0.35504 0.08953 3.97 0.0002 Log_TV(t) XL0_1_2 0.07366 0.05414 1.36 0.1802 Log_Digital(t) XL0_1_3 0.00527 0.00198 2.66 0.0107 Log_DM(t) XL0_1_4 0.15012 0.08453 1.78 0.0822 L1_Radio(t) XL0_1_5 0.40703 0.09935 4.1 0.0002 L3_Paper(t) AR1_1_1 -0.10757 0.1264 -0.85 0.3991 Log_Sales (t-1) MA1 _ 1 _ 1 0.98452 0.05045 19.52 0.0001 e1(t-1)

The model has an R-square value of .6882

Predictive Analytics World 2011

Durbin Watson does not indicate autocorrelations ARCH test statistic does not indicate heteroscedasticity Model residuals appear to be normally distributed

Univariate Model White Noise Diagnostics Durbin Normality ARCH Variable Watson Chi-Square Pr > ChiSq F Value Pr > F Log_Sales 2.06684 3.35 0.1872 1.54 0.2212

The model order does not appear to have an autoregressive error

Univariate Model AR Diagnostics AR1 AR2 AR3 AR4 Variable F Value Pr > F F Value Pr > F F Value Pr > F F Value Pr > F Log_Sales 0.25 0.6185 0.51 0.6043 0.59 0.6249 1.59 0.1962

Predictive Analytics World 2011

Vector Autoregression Code (model assumed to be MAX(1,0))

Dickey-Fuller Unit Root Tests Variable Type Rho Pr < Rho Tau Pr < Tau Log_Sales Zero Mean -75.28 <.0001 -5.44 <.0001 Single Mean -81.28 0.0004 -5.51 0.0001 Trend -89.13 <.0001 -5.46 0.0003

Dickey-Fuller Tests results are unchanged

Model Parameter Estimates Standard Equati on Parameter Es ti mate Er r or t Value Pr > |t| Variable Log_Sales CONST1 0.00188 0.00082 2.29 0.0267 1 XL0_1_1 0.36226 0.09098 3.98 0.0002 Log_TV(t) XL0_1_2 0.06322 0.05588 1.13 0.2636 Log_Digital(t) XL0_1_3 0.00491 0.00199 2.47 0.0173 Log_DM(t) XL0_1_4 0.11312 0.07101 1.59 0.1179 L1_Radio(t) XL0_1_5 0.36666 0.08721 4.2 0.0001 L3_Paper(t) MA1 _ 1 _ 1 0.99856 0.05334 18.72 0.0001 e1(t-1)

The model has an R-square value of .6917

Predictive Analytics World 2011

Durbin Watson does not indicate autocorrelations ARCH test statistic does not indicate heteroscedasticity Model residuals appear to be normally distributed

Univariate Model White Noise Diagnostics Durbin Normality ARCH Variable Watson Chi-Square Pr > ChiSq F Value Pr > F Log_Sales 2.27232 1.6 0.4499 1.03 0.3152

The model order does not appear to have an autoregressive error

Univariate Model AR Diagnostics AR1 AR2 AR3 AR4 Variable F Value Pr > F F Value Pr > F F Value Pr > F F Value Pr > F Log_Sales 1.08 0.3035 0.99 0.3784 0.76 0.5207 1.58 0.1983

Predictive Analytics World 2011

Error=(1-(Forecast/Sales)) Predictive Analytics World 2011

Predictive Analytics World 2011

Vector Error Correction Model (VECM) Code

proc varmax data=mme.simulated_base plots=(impulse) outest=est outstat=stat;

nloptions tech=newrap maxiter=5000000000 maxfunc=5000000000;

model Log_Sales Log_TV Log_Digital Log_DM L1_Radio L3_Paper

/print=(all) lagmax = 10 p=4 cointtest=(johansen=(normalize=Log_Sales));

cointeg rank=4 normalize=Log_TV exogeneity;

output out=out lead=5;

causal group1=(Log_Sales) group2=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM);

causal group1=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM) group2=(Log_Sales);

run;

This model is estimated in levels.

Estimating a VECM with differenced data results in a lost of information.

Predictive Analytics World 2011

Granger Causality

Granger-Causality Wald Test Chi- Pr > Test DF Square ChiSq 1 20 54.61 <.0001 2 20 32.01 <.0432

Test 1: Group 1 Variables: Log_Sales Group 2 Variables: Log_TV L1_Radio L3_Paper Log_Digital Log_DM

Test 2: Group 1 Variables: Log_TV L1_Radio L3_Paper Log_Digital Log_DM Group 2 Variables: Log_Sales

Sales drives (Granger causes) advertising.

Advertising drives (Granger causes) sales.

Predictive Analytics World 2011

Johansen Rank Test

Cointegration Rank Test Using Trace 5% H0: H1: Eigenvalu Critical Drift in Drift in Rank=r Rank>r e Trace Value ECM Process 0 0 0.4891 188.8512 93.92 Constant Linear 1 1 0.3978 123.7144 68.68 2 2 0.271 74.5222 47.21 3 3 0.2289 43.8567 29.38 4 4 0.113 18.6465 15.34 5 5 0.0697 7.012 3.84

Cointegration Rank Test Using Trace Under Restriction 5% H0: H1: Eigenvalu Critical Drift in Drift in Rank=r Rank>r e Trace Value ECM Process 0 0 0.4892 189.2263 101.84 Constant Constant 1 1 0.3984 124.0615 75.74 2 2 0.2712 74.7639 53.42 3 3 0.2293 44.0806 34.8 4 4 0.1138 18.8118 19.99 5 5 0.0705 7.0963 9.13 Predictive Analytics World 2011

Johansen Rank Test Hypothesis of the Restriction Drift in Drift in Hypothesis ECM Process H0(Case 2) Constant Constant H1(Case 3) Constant Linear

Hypothesis Test of the Restriction Restricted Chi- Pr > Rank Eigenvalue Eigenvalue DF Square ChiSq 0 0.4891 0.4892 6 0.38 0.999 1 0.3978 0.3984 5 0.35 0.9967 2 0.271 0.2712 4 0.24 0.9933 3 0.2289 0.2293 3 0.22 0.9736 4 0.113 0.1138 2 0.17 0.9206 5 0.0697 0.0705 1 0.08 0.7715

Predictive Analytics World 2011

Model Diagnostics

Univariate Model White Noise Diagnostics Normality ARCH Durbin Pr > Variable Watson Chi-Square ChiSq F Value Pr > F Log_Sales 2.02311 3.86 0.145 0.04 0.8432 Log_TV 2.027 76.5 <.0001 0.01 0.9315 Log_Digital 2.00815 13.3 0.0013 0.06 0.8026 Log_DM 1.89245 1.9 0.387 0.6 0.441 L1_Radio 2.05207 1.51 0.4708 0.02 0.8891 L3_Paper 2.01872 0.74 0.6907 0.46 0.4999

There is no evidence of heteroscedasticity

Only the residuals for Television and Digital Spend are not normally distributed

Predictive Analytics World 2011

Model Diagnostics

Univariate Model ANOVA Diagnostics Standard Variable R-Square Deviation F Value Pr > F Log_Sales 0.3979 0.11948 1.98 0.0139 Log_TV 0.2892 0.16767 1.22 0.2547 Log_Digital 0.499 0.1971 2.99 0.0002 Log_DM 0.4996 3.33832 3 0.0002 L1_Radio 0.6721 0.10855 6.15 <.0001 L3_Paper 0.7234 0.06408 7.85 <.0001

Except for television spend, each of the models is statistically significant.

Predictive Analytics World 2011

Weak Exogeneity Testing Weak Exogeneity of Each Variables Pr > Variable DF Chi-Square ChiSq Log_Sales 4 11.01 0.0265 Log_TV 4 5.72 0.2212 Log_Digital 4 23.18 0.0001 Log_DM 4 44.98 <.0001 L1_Radio 4 14.83 0.0051 L3_Paper 4 20.18 0.0005

Amongst the six regressors, only Television Spend appears to be weakly exogenous

Predictive Analytics World 2011

Error=(1-(Forecast/Sales)) Predictive Analytics World 2011

Predictive Analytics World 2011

Feel free to contact me at [email protected].

Predictive Analytics World 2011