Cimm Best Practices in Cross Platform Advertising Effectiveness Measurement
Total Page:16
File Type:pdf, Size:1020Kb
BEST PRACTICES IN CROSS PLATFORM ADVERTISING EFFECTIVENESS MEASUREMENT By: Michele Madansky, PHD Kathryn Koegel May 2011 CIMM BEST PRACTICES IN CROSS PLATFORM ADVERTISING EFFECTIVENESS MEASUREMENT EXECUTIVE SUMMARY Cross Platform Advertising Effectiveness Measurement is critically important in today's media environment due to the rapid changes in consumer behavior with respect to all forms of media. Mobile devices – and the consumption of video anywhere – are particularly heightening interest in this type of analysis. While initially used to help demonstrate effectiveness of digital media, these sorts of studies are now used to adjust media mix, understand the synergistic role of media and in the planning process. No constituents (marketers, agencies or media) are entirely satisfied with current solutions, which include opportunity to see methodologies (OTS), lab testing and some syndicated solutions. But, all are eager to improve upon current methodologies and push beyond to experiment with new methodologies. Single source data for passive media measurement is the Holy Grail for Cross Platform Advertising Effectiveness, but isn’t going to be available in the near term for these executives who have product/inventory to sell, marketing budgets to manage and media plans to justify each quarter. Until true single source becomes available, there is a desire to “build a better mousetrap” with what is available now, especially for various opportunity to see (OTS) methodologies. There is much innovation around solutions for well known OTS challenges which include apples to oranges comparison between media, murky experimentation (especially in respect to control group development), the current low incidence of survey response (and potential bias among responders) and high drop-out rates due to survey complexity. Research vendors are applying sophisticated modeling techniques, using persistent data tags, mining new sources for survey samples (including Facebook and mobile recruitment) and going beyond mega panels to find exposed cells in “panels of panels.” There are also vendors who are applying a back-to-basics approach such as simplifying survey design or simply synching up exposure windows for various media to improve reliability and comparability of results. The piqued interest in Cross Platform Advertising Effectiveness Measurement has created dramatic opportunity in the space that can best be explored through the following: i. Support & Test Data Mash-Up Methodologies: By finding connections between panels that offer media and ad exposure data with purchase or attitudinal data, we can make smaller strides towards single source that work for individual product categories. What we learn from these tests may help with ultimate connections between data sets such as set top box and online ad exposure. ii. Improve Current Methodologies: Work with the here and now and develop better confidence and better practices with studies being conducted today. We understand the limitations – let’s fix what can be fixed and validate these improved methodologies. iii. Validate/Establish Methodological Best Practices: Techniques like weighting and Bayesian modeling may offer relatively simple solutions to problems with current methodologies. Test their validity (through research on research) and demonstrate best practices in using these techniques. iv. Tackle Missing Data Issues: Work with new sources of data – especially from the mobile arena – to continually develop a greater understanding of how media is used now, with a goal towards preparing ourselves for the media landscape over the next decade. v. Foster Shared Learning and Best Practices: An organization like CIMM can help establish best practices by educating the community on current state of Cross Platform Advertising Effectiveness methodologies as well as best practices on integrating these results with other research programs (including marketing mix modeling, tracking studies and single media brand impact studies). This can serve as an educational forum for all that Cross Platform Advertising Effectiveness Measurement can accomplish now and will accomplish in the future. 2 CIMM BEST PRACTICES IN CROSS PLATFORM ADVERTISING EFFECTIVENESS MEASUREMENT Table of Contents EXECUTIVE SUMMARY ............................................................................................................................ 2 INTRODUCTION AND BACKGROUND: ..................................................................................................... 4 The Role of Cross Platform Advertising Effectiveness Measurement in a World of Increasing Media Choices ................................................................................................................................................ 4 Goal Of Cross Platform Advertising Effectiveness Measurement Project: ......................................... 5 I. METHODOLOGY AND LANDSCAPE OVERVIEW: .............................................................................. 6 White Paper Methodology ................................................................................................................. 6 Landscape Overview ........................................................................................................................... 7 II. STRENGTHS, OPPORTUNITIES & CHALLENGES OF EACH METHODOLOGY: .................................. 14 OTS – Opportunity to See: ................................................................................................................ 14 i. Apples to Oranges ................................................................................................................ 17 ii. Murky Experimentation ....................................................................................................... 19 iii. Freaks and Geeks .................................................................................................................. 21 iv. Dropouts ............................................................................................................................... 25 Lab Testing: ....................................................................................................................................... 27 Other Methodologies: ...................................................................................................................... 29 III. FUTURE OPPORTUNITIES FOR CROSS PLATFORM ADVERTISING EFFECTIVENESS MEASUREMENT: ................................................................................................................................... 34 i. Support Data Mash-Up Methodologies ................................................................................... 35 ii. Improve Current Methodologies .............................................................................................. 35 iii. Validate/Establish Methodological Best Practices ................................................................... 38 iv. Tackle Missing Data Issues ....................................................................................................... 38 v. Foster Shared Learning and Best Practices .............................................................................. 39 APPENDIX: ............................................................................................................................................. 42 A. Cross Platform Advertising Effectiveness Measurement Vendor List: ..................................... 42 B. Cross Platform Advertising Effectiveness Measurement Landscape: ...................................... 43 Cross Platform Advertising Effectiveness Measurement Landscape – OTS Capabilities .................... 43 Cross Platform Advertising Effectiveness Measurement Landscape – Lab Testing Capabilities ......... 48 Cross Platform Advertising Effectiveness Measurement Landscape – Other Research Capabilities... 49 C. Interviewee List: ....................................................................................................................... 50 Agencies ............................................................................................................................................ 50 Marketers.......................................................................................................................................... 51 Media ................................................................................................................................................ 51 Research Vendors ............................................................................................................................. 52 D. The Authors: ............................................................................................................................. 53 3 CIMM BEST PRACTICES IN CROSS PLATFORM ADVERTISING EFFECTIVENESS MEASUREMENT INTRODUCTION AND BACKGROUND: The Role of Cross Platform Advertising Effectiveness Measurement in a World of Increasing Media Choices The media landscape has changed more in the past 10 years than in the previous five decades altogether. The proliferation of devices and new digital media opportunities has created additional touchpoints for marketers not only to reach their audiences but also engage with them in more meaningful ways. Digital media allows the consumer to interact with a brand – whether conducting an online search, visiting a corporate website, becoming a “fan” of the brand on Facebook, or actively choosing to view which video ad to watch