The State of Digital Ad Measurement Becky Wu, Ph.D. Senior Executive Vice President, Luth Research
The ad spending figures continue to climb for total ad spending and digital ad spending. According to eMarketer, ad investments in the U.S. alone will see an average annual growth rate of over 5%, reaching $180 billion in 2018. These numbers tell the significance of getting it right in advertising. The face of advertising has morphed into diverse, often fragmented practices that offer both promises and challenges. Ad targeting and media planning are advancing with innovations on re-targeting, near-distance targeting, statistical ID targeting, serving on a plethora of devices and formats. Accordingly, it requires continuous adaptation in ad measurement methodologies to answer the ROI question for ad spending. What are the dominant and emerging approaches in measuring ad effect? How do advertisers, marketers and researchers use each based on pros and cons? What are the ideas we want to put forward for devising a better ad measurement with collective collaboration in the industry? These are the goals this whitepaper is addressing.
Back to Basics The very first ad measurement research appeared in 1923 by Daniel Starch, a Harvard professor who created and employed a recognition based method to assess ad effectiveness. Later on,
George Gallup developed a recall procedure, which has evolved to be a popular methodology. Both trailblazers have a profound influence on survey based ad measurement. Ad measurement is built upon the fundamental idea of comparing two groups of individuals who are identical or highly similar in all aspects except for their exposure to the ads. If the primary difference is ad exposure, and one group has a significantly higher propensity to buy the advertised product or brand, then we can say the advertising is working. The unexposed group is typically called Control, and the exposed group is called Test. Using survey to isolate out the impact of advertising among a multitude of potentially influencing factors is a practical solution to achieve testing results at a larger scale. There are two dominant designs for survey based ad measurement. One is a pre/post design, which draws one independent sample of respondents before and after the ad campaign (note: “independent samples” means not having the same respondents but the respondents will be highly similar in characteristics), and administers surveys to collect both perceptions, purchase intent and other usual impact related metrics. The end goal is to compare these two surveys and determine if there is any lift in the metrics from the second survey. The second method is conducting only one post campaign survey, comparing exposed (Test) and unexposed (Control) respondents. Both approaches rely on respondents’ ability to recall the ads they have seen. The post campaign only method is more likely to suffer from the risk of inaccurate recall unless Test and Control are also defined by distinct exposed and unexposed geo markets. The pre/post method is less vulnerable as it controls for the same level of potential recall bias by contrasting the overall target audience before and after advertising takes place. However, it can be less efficient in needing to execute two surveys and two separate samples. The potential inaccuracy for self-reported responses is a well recognized limitation for the survey methodology. This not only calls into question the true account of ad exposure, but also challenges the resulting impact metrics. If people cannot remember ads reliably, can we fully trust their purchase intent and brand perceptions? The drawback for recall inaccuracy is especially salient when multiple campaigns are taking place with overlapped or similar messages. However, when lacking a viable alternative, survey remains a pragmatic way to measure ad exposures and effectiveness across a large sample size.
Pros Cons Overall Survey Methodology • Does not involve • At risk for inaccurate technical setup as recall and overstated part of the ad serving impact process • Relatively cost effective
Pre/Post • Provides a relatively • Requires advance clean read on the planning to execute advertising impact pre survey before due to the ability to campaign starts compare post • Requires two campaign outcome to substantial sample baseline sizes for pre/post, which sometimes is a challenge for hard to find niche audiences Post Only – Test vs. Control • Cost and time • More vulnerable for efficient recall bias, making it even harder to accurately differentiate Control vs. Test
Who has moved my “cookies”? The popularity of cookies has been rising in recent years thanks to the convenience to set cookies for Internet users’ browsers and hence recognize certain characteristics of the users from the vantage point of any given website. Cookies are an advertising targeting means first and foremost. They allow websites (i.e., publishers) to know information about the visitors and customize the website experience for the visitors to some extent. This also enables advertisers and ad serving companies to re-target the visitors as they surf across sites before the cookies expire or are no longer valid. Given cookies’ ability to target individuals with a certain timeframe, they are becoming a common method for identifying ad exposures among consumers for research purposes. The primary approach using cookies is to leverage a process like this: Step 1: Place a pixel tag created by the research firm responsible for measurement onto the ad creative to be served Step 2: Place cookies onto the sample respondents’ browsers so these respondents can be recognized when they surf on the Internet and visit a page where a relevant ad (with the pixel tag embedded) is served; the respondents typically are from an online research panel Step 3: Have ad serving platforms or ad networks selected for the campaign serve the ads (with the pixel tag embedded) according to the established media plan Step 4: Identify respondents who have been exposed to the ads and those who have not as they surf the Internet by determining the respondents’ cookies have been “recognized” by the pixel tag during the campaign
Step 5: Deploy an online survey, typically at the end of the campaign or at a pre-established interval, to both exposed and unexposed respondents to measure perceptions, purchase intent and other metrics; the online survey is not a pop up survey, and also used to determine if the exposed and unexposed respondents qualify as the campaign’s target audience based on demographic and other criteria set by the advertiser In addition to using a research firm’s online research panel for cookie placement, it is also possible for an advertiser to directly work with a publisher such as a well-known, high traffic website to deploy pop-up surveys to site visitors who have been exposed to ads served on the publisher’s website(s). Ad exposure is detected by cookies placed by the publisher, which triggers a pop up survey on the site. This overall approach seems logical and a much better solution for identifying ad exposures at a much more accurate level that doesn’t depend on human recall. However, cookies’ increasingly apparent limitations start to outweigh its merits. The pros and cons for cookies are summarized as follows:
Pros Cons • Is not prone to human recall • Cookies do not work in mobile apps. This inaccuracy notable misfit calls for re-evaluation of • Allows for tracking for a large cookies tracking as a true digital ad number of consumers due to the measurement tool given that mobile ad ease of placing cookies onto user spending continue to climb and mobile browsers apps remain a bigger share of consumers’ mobile activities1. • Consumers are deleting cookies at a much higher rate than ever before. The heightened consumer awareness of what cookies can do for targeting adds to the spreading distrust with cookies. • Cookies are not unique. Duplicated cookies cause inaccurate attribution to the right individuals. • Certain browsers are now rejecting third party cookies (all cookies set by a research firm or ad serving platforms that are not used by the website the consumer is visiting), practically making it impossible to build the linkage between
1 eMarkter, 2015: By 2018, mobile ad spending is expected to account for approximately 25% of total ad spending
ad exposure and respondents. • Cookies expire at a certain time frame (such as two weeks). Even if new cookies are placed for the same individuals again and again, there is ample room for cookies to miss a significant part of the respondent’s Internet activities in between expiration and replacement. • The requirement of embedding a pixel tag into the ad creative before the campaign starts is not trivial. This placement process involves close technical collaboration between the research firm, the ad agency, and the selected ad serving companies/ad networks (if the ad agency does not play that role) with coordination from the advertiser. It is not uncommon to see at least two weeks needed for incorporating the pixel tag into the ad serving process. Many campaigns on a speedy rollout schedule simply do not have room for this lead time.
There are many factors that can move a person’s cookies or renders cookies ineffective in matching up individual consumers and ad exposures, which is ironically the very benefit cookies are intended to accomplish. Cookie-based tracking, the existing currency in the ad measurement world, is now under critical examination and called into question for its accuracy and value2. Admit these challenges, cookies yield efficiency when the campaign target audience is expected to be a small segment within the general population or when the advertiser works directly with a few select major publishers. All of these still need to be conditioned on that mobile apps are not part of the tracking.
Unified ad tag tracking As illustrated by the above discussion on survey and cookie tracking, the salient issues digital advertising measurement is facing are clear. There is an imperative need to continue to
2 Brian Boland, 2014, AdAge, http://adage.com/article/digitalnext/cookies-cut-anymore-online- ad-measurement/292225/
develop ad measurement approaches that combat the human recall risk, mitigate cookie mismatching, and most importantly fill the current gap of not being able to track ad exposure within mobile apps. Unified ad tag tracking has emerged to be a promising solution. Luth Research’s ZQ Intelligence™ ad tracking is a leading platform using this methodology. It is worth noting the “ad tag” here refers to the very original ad tags created for each ad on the side of the publisher, ad serving platforms/ad networks. The primary purpose of these ad tags is to allow these parties to keep track of ads as they are being served during the campaign. These ad tags are fundamentally different from the pixel tags or cookies. An anatomy of Luth’s ZQ platform illuminates how combining passive device tracking of consumer behaviors and the ad tags proves effective in overcoming the aforementioned challenges. At its very core, ZQ’s ad tracking technology consists of two components: 1) the ability to identify ad tags that are generated for ad creatives when they are being served, and 2) the ability to collect the user’s digital data across browsers, operating systems, and devices. Because each ad creative must have unique ad tags in ad serving, detecting the ad tag is the surest way to accurately pinpoint if and where the ad is being shown to the user across publishers and ad networks. This detection mechanism is not effective without ZQ’s holistic digital behavior data collection. After a user has downloaded the ZQ tracking software, ZQ captures granular http/https requests and responses from both PC and mobile devices. Specifically for mobile, ZQ enables unified data collection across iOS and Android, and across mobile web and mobile app. The ad tag identification is performed for the target ads within this digital data with tremendous depth and breadth. More importantly, ZQ makes it possible to directly correlate the individual’s ad tag exposure(s) with his/her other digital activities collected 24/7 and over time with user permission. Hence, post-click/post-impression attribution is no longer an unattainable ideal for ad measurement. This capability of unified ad tag tracking addresses the ultimate goal of ad measurement - assessing if ad exposure leads to changes in behaviors and perceptions. The changes in behaviors in this case are captured by the same passive digital tracking rather than the respondents’ self-reported responses. Unified ad tracking based on passive digital measurement has numerous distinct advantages including platform agnostic, breaking down ecosystem fragmentation, precise ad detection, deep in-app ad visibility, and providing a single-source view of the entire consumer digital journey. However, this approach is not without limitation. The process of ensuring respondents be tracked across devices with permission is a relatively new practice. This well-rounded tracking can encounter challenges with yielding a large sample size when the ad campaign has niche, hard to reach segments. As the research industry continues to advance in cross-platform digital tracking and empanelment as a research methodology, this limitation is expected to be mitigated over time.
Pros Cons • Allows for true cross platform digital • May not be able to yield a large sample ad measurement including PC and size for niche consumer segments mobile • Accuracy of detecting ad exposures; is not prone to human recall error or cookie inaccuracy • No lead time to set up pixel tagging; uses the very original ad tags used by ad serving platforms • Allows for direct linkage of ad exposures and ensuing online behaviors
Viewability – The Newest Frontier The latest ad measurement movement centers around viewability. It stems from advertisers’ discontent with knowing a high percent of impressions (as high as 50%) have no views3. The rightful request has been met with a significant amount of attention and efforts. However, the future for what standards to implement, and how to implement from the side of publishers and ad networks, and how to enable 3rd party research measurement remains less clear. The Media Rating Council (MRC) sets the standard of a display ad to be viewable when 50% of an ad’s pixels come into view for a minimum of one second. This overall standard has not been widely adopted. As a front runner in participating in the debate of enforcing viewability, Facebook has defined its own metric, which currently equates to when any part of the ad comes into view even for less than one second. Based on the analysis from the ad serving platforms, the complexity of the issue is influenced by page position, ad size, the industry/nature of the ad and other factors4. It is necessary to call out that the ad serving technology itself has not yet had a method to count actual views correctly or ensure a specific target % of viewability5. Mobile advertising also faces different challenges and considerations when applying viewability. Developing a reasonable, enforceable standard for viewability for the publishers and ad networks is only the start. The journey to establish corresponding evaluation research
3 http://adage.com/article/digital/facebook-mrc-tackling-mobile-ad-viewability-year/297192/ 4 https://think.storage.googleapis.com/docs/5-factors-of-viewability_infographics.pdf 5 http://blogs.wsj.com/cmo/2015/02/18/facebook-says-it-only-sells-viewable-ads/
methodologies to be performed by third party independent research firms outside of publishers and ad networks will not be speedy.
The Road Ahead Ad serving practices continue to benefit from many technological and engineering advancements. The market research industry’s duty to deliver on rigorous, unbiased, and comprehensive ad measurement can be best fulfilled by not only promoting innovation within the industry itself to catch up with the latest and greatest ad technology, but also forming truly forward-looking alliance with technology players in the ad serving space to create ad targeting technologies that have built-in consideration for ad measurement.