Media Mix Modeling
Total Page:16
File Type:pdf, Size:1020Kb
Media Mix Modeling Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Media Mix Modeling Objectives Demonstrate some of the commonly used techniques and methodologies used to estimate the impacts of media spend. Illustrate some of the most frequently encountered problems. Reference some of the newer Econometric techniques incorporated into SAS/ETS and Base Stat. Caveats It is not possible to provide an extensive catalog in the time provided. There are far more techniques and challenges than those listed here. Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 A Number Of Data Concerns Exist Simulated Media Spend Data $3,500,000 $3,000,000 $2,500,000 $2,000,000 $1,500,000 $1,000,000 $500,000 $0 1 3 5 7 9 111315171921232527293133353739414345474951 1 3 5 7 9 111315171921232527293133353739414345474951 Year 1 2 Television_Spend Radio_Spend Newspaper_Spend Direct_Mail_Spend Digital_Spend Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Data Concerns ! " A simple “working definition” of stationarity is a Stationarity process whose mean, variance and autocorrelation structures do not vary over time. ! " If variables in a regression model are not stationary, standard asymptotic assumptions are not valid (e.g. t-statistics will not follow a t distribution). ! " Generally, regression models with non-stationary predictors (that are not differenced) yield spurious (even nonsensical) results. ! " If a stationary linear combination of non-stationary Cointegration regressors exists, these regressors are said to be cointegrated. ! " “Long run” and “short run” dynamic relationships exist amongst cointegrated regressors. ! " Generally, regression models that properly account for cointegrated predictors will not yield spurious results Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Data Concerns ! " Currently exogeneity is defined in terms of weak, Exogeneity strong and super exogeneity. ! " A regressor is said to be weakly exogenous if inference on the regression parameter estimates conditional upon the regressor involves no loss of information. If weak exogeneity does not hold the model's dynamic parameter estimates are inefficient. ! " A regressor is said to be super exogenous if it is weakly exogenous and the regression parameter estimates do not change when changes in the regressor's distribution occur. ! " A regressor is said to be strongly exogenous if it is weakly exogenous and the regressor is not preceded by an endogenous variable (in the model formulation). Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Data Concerns ! " GARCH – Generalized Autoregressive GARCH Conditional Heteroscedasticity. ! " The variance of the current error term (or innovation) is a function of the size of the previous period's error term (or innovation). ! " Primarily used in variance modeling and may not necessarily improve forecasts. Multicollinearity ! " Largely a question of degree or severity. ! " If severe multicollinearity exist, the variance estimates are inflated and the following may be observed: imprecise (or implausible) and unstable parameter estimates, a very high r-squared with statistically insignificant predictors, incorrect coefficient signs. Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Data Concerns ! " When model residuals are correlated, parameter Autocorrelation estimates are inefficient, t-statistics and r-squared values are upwardly biased. ! " Autocorrelation can be positive or negative. ! " First order autocorrelation is the most common variant. ! " Common causes for autocorrelation include observations being present in multiple time periods and omitted variables. Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity proc reg data=mme.simulated_base; Title1 'Main and Interaction Effects -- Multicollinearity Demonstration'; where year = 1; model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif; output out=p1 p=py r=residual; run; quit; Year 1 sales are regressed against Direct Mail, Television, Radio Newspaper and Digital Spend levels. A log-log functional form was assumed to enable easy elasticity estimates. Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Parameter Estimates Parameter Standard Variance Variable DF Es ti mate Er r or t Value Pr > |t| Inflation Intercept 1 -1.33934 163.23107 -0.01 0.9935 0 holiday 1 3.61428 10.18642 0.35 0.7245 81880 Log_DM 1 0.00484 0.00279 1.73 0.0904 1.45394 Log_TV 1 1.11802 11.78288 0.09 0.9249 51846 Log_Radio 1 0.20417 0.14413 1.42 0.1642 13.56417 Log_Paper 1 -1.48058 13.72226 -0.11 0.9146 68041 Log_Digital 1 2.73806 5.83569 0.47 0.6414 22776 LogTVPaper 1 0.09406 0.9969 0.09 0.9253 253746 LogTVDigital 1 -0.21443 0.43017 -0.5 0.6208 55934 LogTVHoldy 1 -0.53949 0.6456 -0.84 0.4082 65908 LogRadioHoldy 1 0.36448 0.38295 0.95 0.3468 19923 Even though none of the regressors are statistically significant at the 5% confidence level, the Adjusted R-square is .8453. Only Direct Mail had a variance inflation value less than 10. Many of the coefficient signs are reversed. Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Ridge Regression Code. proc reg data=mme.simulated_base outvif outest=b ridge=0 to 0.40 by 0.02; Title1 'Main and Interaction Effects -- Multicollinearity Demonstration'; Title2 'Ridge Regression'; where year = 1; model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy/vif noprint; plot /ridgeplot; output out=p1 p=py r=residual; run; quit; proc print data=b;run; Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Ridge Plots Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Regression Plots Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Ridge Regression Code Continued. proc score data=mme.simulated_base score=b(where=(_RIDGE_=0.04)) out=p2 type=RIDGE; var Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy; run; proc print data=p1;run; proc print data=p2;run; A variable selection mechanism is missing. Each regressor is included (“considered statistically significant”). Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity GLM Select (Lasso). proc glmselect data=mme.simulated_base plots=all; Title1 'Main and Interaction Effects -- Multicollinearity Demonstration'; Title2 'GLM Select -- Lasso'; where year = 1; model Log_Sales = Holiday Log_DM Log_TV Log_Radio Log_Paper Log_Digital LogTVPaper LogTVDigital LogTVHoldy LogRadioHoldy /details=all stats=all selection=lasso; *modelAverage nsamples=1000 subset(best=1); run; Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Lasso Variable Selection Summary Effect Effect Number Model Adjusted Step Entered Removed Effects In R-Square R-Square SBC ASE F Value Pr > F 0 Intercept 1 0 0 -144.3294 0.0578 0 1 1 LogRadioHoldy 2 0.5747 0.5662 -184.8367 0.0246 67.57 <.0001 2 Log_Radio 3 0.8251 0.8179 -227.0783 0.0101 70.12 <.0001 3 Log_DM 4 0.8569 0.8479 -233.5702 0.0083 10.68 0.002 4 Log_Digital 5 0.8695 0.8584* -234.4167* 0.0075 4.54 0.0383 Lasso Anova and parameter estimates Analysis of Variance Sum of Mean Source DF Squares Square F Value Model 4 2.61132 0.65283 78.29 Er r or 47 0.39192 0.00834 Corrected Total 51 3.00324 Parameter Estimates Parameter DF Es ti mate Intercept 1 10.424565 Log_DM 1 0.003916 Log_Radio 1 0.228549 Log_Digital 1 -0.126448 LogRadioHoldy 1 0.037262 Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Lasso Variable Selection Summary Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Lasso Variable Selection Summary Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Multicollinearity Lasso Variable Selection Summary Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Stationarity Predictive Analytics World 2011 Copyright © 2011, SAS Institute Inc. All rights reserved. #analytics2011 Stationarity Vector Autoregression Code (model assumed to be ARMAX(1,1,0)) proc varmax data=mme.simulated_base plots=(impulse) outest=est outstat=stat; where year = 1; nloptions tech=newrap maxiter=5000000000 maxfunc=5000000000; model Log_Sales = Log_TV Log_Digital Log_DM L1_Radio L3_Paper /print=(all) lagmax = 10 cointtest=(sw) /*dify=(1) difx=(1)*/ p=1 q=1; output out=out lead=5; causal group1=(Log_Sales) group2=(Log_TV L1_Radio L3_Paper Log_Digital Log_DM); causal group1=(Log_TV L1_Radio L3_Paper Log_Digital