Scheduling of Dynamic In-Game Advertising

OR PRACTICE Scheduling of Dynamic In-Game Advertising

John Turner The Paul Merage School of Business, University of California, Irvine, Irvine, California 92697, [email protected] Alan Scheller-Wolf, Sridhar Tayur Tepper School of Business, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213 {[email protected], [email protected]}

Dynamic in-game advertising is a new form of advertising in which ads are served to video game consoles in real time over the Internet. We present a model for the in-game ad-scheduling problem faced by Massive Inc., a wholly owned subsidiary of Microsoft, and a leading global network provider of in-game ad space. Our model has two components: (1) a linear program (solved periodically) establishes target service rates, and (2) a real-time packing heuristic (run whenever a player enters a new level) tracks these service rates. We benchmark our model against Massive’s legacy algorithm: When tested on historical data, we observe (1) an 80%–87% reduction in make-good costs (depending on forecast accuracy), and (2) a shift in the age distribution of served ad space, leaving more premium inventory open for future sales. As a result of our work, Massive has increased the number of unique individuals that see each campaign by, on average, 26% per week and achieved 33% smoother campaign delivery as measured by standard deviation of hourly impressions served. Subject classiﬁcations: dynamic in-game advertising; video game advertising; display advertising; revenue management; linear programming; goal programming. Area of review: OR Practice. History: Received July 2008; revisions received March 2009, July 2009, December 2009; accepted February 2010. Published online in Articles in Advance February 8, 2011.

1. Introduction where ads are displayed. For this reason, and also because saturation competition Video games have incorporated static ads for decades: of additional constraints (such as , , and context, discussed below), scheduling in-game ads is for example, an old racing game may have a billboard signiﬁcantly more complicated than scheduling banner ads.

to this article and distributed this copy asthat a courtesy toalways the author(s). displays the same Sunoco ad. Recently, how- The ad server is operated by a network provider, an inter- ever, technology and business relationships have matured mediary between game publishers and advertising agen- to allow Internet-enabled consoles (e.g., Xbox, PCs) to cies. We focus on the scheduling problem faced by Massive dynamically serve ads over time, creating an entirely new Inc., a wholly owned subsidiary of Microsoft and a leading ad market: Dynamic in-game ad technology allows in-game copyright global network provider. Game publishers allow Massive to billboards to display different ads to different players based sell and serve ads in their games, and consequently receive on their demographic, the time of day, the day of week, and a portion of the generated revenues; ad agencies buy ad holds possibly other parameters. The in-game advertising indus- campaigns directly from Massive. try is growing quickly; at present, revenues are projected The operational problem network providers like Massive to reach $800 million by 2012 (Cai 2007). face is how to schedule and serve ads to players over At the heart of an in-game ad system is the ad server: time so as to make the best use of their inventory of ad INFORMS Additional information, including rights and permission policies, is available atWhen http://journals.informs.org/. a player enters a new level of a game, his console space. Campaigns purchased by an ad agency specify a tar- connects to the ad server via the Internet and requests new get number of impressions (ads seen by gamers), a rough ad graphics for billboards, stadium walls, and other loca- schedule for serving these impressions over time, and also tions where ads are shown in the level. The ad server decides a desired mix (e.g., 60% in sports games, 40% in the rest of which ads to serve to this player, functioning like a Web the games). A campaign’s delivery may also be restricted server delivering banner ads. But, unlike on the Web, where to certain geographic areas and/or times of the day. In addi- a selected ad is almost certainly seen, it is common for tion, the network provider must also manage (1) satura- only a fraction of selected in-game ads to be seen. Billable tion: it is undesirable for a single player to simultaneously ad time is thus recorded only when, as part of game play, see many copies of the same campaign; (2) competition: the player navigates through the level and passes locations campaigns of two competing brands—e.g., Coke and

Pepsi—should not be served to the same player; and (3) advertisers. In these models, the advertiser chooses among context: ads should not seem out of place within the advertising vehicles (e.g., newsprint, television, radio) to game—e.g., Coke ads belong on virtual soda machines, maximize some combination of reach (audience size), fre- Tide ads do not. quency, and campaign duration subject to a budget con- The size, scope, and complexity of Massive’s problem straint (see textbooks by Rossiter and Danaher 1998 and are such that even if there were no system uncertainty, Gensch 1973). One typical assumption is wearout (the optimization of their ad server would be difficult. But, effectiveness of an ad decreases as the number of expo- of course, uncertainty is present—there are three primary sures to the same person increases); papers in this line sources: (1) the acquisition of new games, (2) the sale of research include Thompson (1981) and Simon (1982), of new campaigns, and (3) error in inventory forecasts of which arrive at an optimal pulsing strategy of advertising ad space. This last factor, uncertainty in the amount of expenditure over time. Although some of these concepts are ad space, arises because the number of players, the types relevant, it is most instructive to compare our model with (demographics) of players, and the ad space that the play- those that take the perspective of a third party scheduling ers actually see during game play are not known when the many advertisers, such as models that plan TV commercials scheduling problem needs to be solved. Thus, campaigns and webpage banner ads. Structurally, in-game advertising sometimes fall short of their impression goals or deviate sits between TV and Web advertising: It has well-defined from the desired pattern of delivery; in that case, the net- contracts like TV, yet decisions are made at a very fine work provider offers the advertiser a make-good: the cam- granularity, as on the Web. paign is extended or the advertiser is offered a refund or TV Commercials. Advertisers purchase 60%–80% of credit for future use. the year’s ad space during a two- to three-week period in We present the first planning/scheduling model and algo- May called the up-front market; the remaining ad space rithm for dynamic in-game advertising. Our model has two is sold first-come-first-serve in the scatter market. In con- components: (1) a linear program called the Weekly Plan trast, in-game ads are sold throughout the year, so the divi- is solved periodically to establish target service rates, and sion between up-front and scatter markets is not profound. (2) a packing heuristic called the Real-Time Algorithm is Therefore, papers (e.g., Araman and Popescu 2010) that run whenever a player enters a new level, to serve impres- determine the optimal up-front versus scatter trade-off are sions in accordance with these service rates. Benchmarking not directly applicable. our model against Massive’s legacy algorithm using histor- Bollapragada et al. (2002) use goal programming to pro- ical data, we observe (1) an 80%–87% reduction in make- duce a sales plan for a single campaign that allocates com- good costs (depending on forecast accuracy), and (2) a shift mercial slots from TV shows such that the advertiser’s in the age distribution of served ad space, leaving more pre- preferences are honored as closely as possible. Inventory mium inventory open for future sales. Massive has begun a is booked for each advertiser in sequence, allowing each staged implementation of our model, and to date has bene- advertiser to request changes to their plan before it is fitted from a 26% average increase in the number of unique finalized. Audience size is assumed to be deterministic individuals that see each campaign each week, and 33% and there is no mention of make-goods allocation when to this article and distributed this copy assmoother a courtesy to the author(s). campaign delivery, as measured by standard devi- schedules are not executed as planned. This is a static up- ation of hourly impressions served. front market problem; in contrast, we consider the dynamic We proceed as follows: In §2, we review the literature problem in which new campaigns and games materialize on media planning and scheduling, and describe existing over a rolling horizon. Furthermore, our Real-Time Algo- copyright models for broadcast television and webpage banner ads. rithm is significantly different than the low-level ad-slotting We introduce the problem in §3, the Weekly Plan LP in §4, “ISCI Rotator” algorithms used for TV because in our case and the Real-Time Algorithm in §5. We benchmark our each viewer can be shown different ads. Finally, Zhang holds algorithm against Massive’s legacy algorithm in §6, and (2006) also solves the (static, deterministic, up-front mar- comment on our implementation learnings in §7. We con- ket) TV commercial scheduling problem, but in their case clude with comments and list future work in §8. they assume that client negotiations are minimal, allowing them to simultaneously schedule all campaigns via a INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. mixed-integer program. 2. Literature Review Web Advertising. The most common objective in Web To the best of our knowledge, ours is the first academic advertising is maximizing click-through (e.g., Chickering treatment of the scheduling of in-game advertisements. The and Heckerman 2003, Nakamura and Abe 2005), a con- process of dynamic in-game advertising is well documented cept that does not apply to in-game ads. However, some (Chambers 2005, Svahn 2005); however, operational prob- papers explore the essential dynamic trade-off: choosing lems, such as the scheduling problem we examine, have between serving the campaign with the highest bid versus not been studied. In the traditional media planning liter- the one farthest from achieving its contracted impression ature, the optimization problem is usually that of a sin- goal. Approaches include the online algorithm of Mehta gle advertiser and not of a network provider serving many et al. (2007) and the large LP solved by column generation Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 3

by Abrams et al. (2007). Of these, Abrams et al. (2007) receives this revenue up-front when the campaign is nego- include promising computational results on historical data; tiated. Subsequent scheduling decisions affect Massive’s however, the authors note some challenges to implementing ability to deliver the campaign; thus penalty costs (make- the full-scale algorithm in practice. In addition to tailoring goods) may be incurred should campaign k not be delivered our focus to in-game advertising, our problem definition is as promised. Nonpaying campaigns include house ads, pub- more comprehensive than either of these papers: we solve lic service announcements (PSAs), and default graphics; a multiobjective problem with many practical constraints these do not generate any revenue, do not have minimum not present in Abrams et al. (2007) or Mehta et al. (2007) impression goals, and are served as “filler” when no paying (e.g., to spread impressions over time and across various campaigns can be served. ad spaces). Campaign Targeting—Audience Demographics and Context. Paying campaigns currently specify targeting con- 3. Definitions and Problem Statement straints along three dimensions: geography, time, and context; i.e., impressions must come from specific geographic Games, Zones, Inventory Elements, and Regions. regions, time slots, and inventory elements, respectively. A video game is subdivided into many zones, each having The geographic dimension is indexed by 300+ designated one or more inventory elements (the generic name given market area (DMA) codes that allow Massive to con- to in-game billboards and other locations where ads are strain service to metropolitan areas across the United States displayed). Typically, each level of a game is a separate and/or key areas in other parts of the world. There are zone. Within each zone, inventory elements are grouped 42 combinations of {Weekday, DayPart} tuples for the time into regions based on their spacial proximity; all inventory dimension, one for each of six DayParts (contiguous groups elements visible from the same vantage point are typically of hours, e.g., Prime Time = 7 p.m.–12 a.m.) in each of the in the same region. seven weekdays. Finally, there are 1 to 50 inventory ele- Requests, Arrivals, Game Instances, and Spots. When ment classes (IECs) per game that group inventory elements a gamer enters a new zone, his console sends a request by context (e.g., “all virtual soda machines”). By targeting to the ad server to select ad campaigns for all inventory specific games, geographic regions, and times, a campaign elements in the zone. From the ad server’s perspective, an can be tailored for specific audience demographics. In con- arrival of a gamer has just occurred. This arrival spawns a trast, IEC targeting ensures proper context by integrating game instance; the gamer will continue to see the ads that ads in realistic places: Coke ads, not Tide ads, belong on were selected by the ad server for this game instance until virtual soda machines. the gamer departs from this zone. Although some inventory Competition and Saturation. Within each game in- elements can only show one ad per game instance, others stance, (1) campaigns of competing brands (e.g., Coke and (e.g., an in-game Jumbotron) cycle through a sequence of Pepsi) cannot be shown, (2) each campaign can be served spot ads; we call each element of the sequence an ad . in at most z spots, and (3) when a campaign is served in Adtime. An inventory element in one game instance multiple spots, the spots should preferably be in different

provides adtime equal to the total amount of time—not regions. The quantity z is called the saturation cap and is to this article and distributed this copy asnecessarily a courtesy to the author(s). contiguous—that the inventory element appears typically between a quarter to half the number of inventory on-screen. Adtime can be aggregated over multiple inven- elements in zone z. tory elements and over multiple game instances; thus, we Campaign Delivery Quality. In addition to a cam-

can consider quantities such as the expected adtime of Zone paign’s aggregate impression goal qk, the advertiser usually 3 of Game A on Monday. copyright speciﬁes other impression goals either explicitly or implic- Impressions. We measure billable adtime in impressions. itly. These include mix requirements (e.g., at least 40% of Massive deﬁnes an impression as 10 seconds of time—not impressions should come from sports games) and deliv- holds necessarily contiguous—in which a gamer sees the same ery schedules (e.g., uniformly spread delivery over time, or ad campaign on-screen, possibly across multiple inventory deliver more impressions the week before a new product elements in the same game instance. For example, given launch). Mix requirements are modeled with an impression

a zone with two single-spot inventory elements, if these goal qIk for certain groups of games I that campaign INFORMS Additional information, including rights and permission policies, is available atelements http://journals.informs.org/. log 7 and 8 seconds of adtime, respectively, in k desires; we call I a MixComponent. Delivery schedules t one game instance, then either: (1) the same campaign was can include weekly impression goals qk and, if needed, + = t served in both inventory elements, so 7 8/10 1 weekly impression goals for each MixComponent qIk.It impression is counted toward that campaign’s impression is usually also important to serve each campaign impres- goal; or (2) different campaigns were served in these inven- sions from many different games because part of the value tory elements, and no impressions are counted for either of contracting with a network provider such as Massive campaign (7/10=8/10=0). is derived from obtaining access to advertising space in a Paying and Nonpaying Campaigns. Paying campaigns breadth of games from different publishers.

specify an impression goal of qk and a unit price of pk, Publisher Concerns. Because game publishers receive a × providing revenue of pk qk. We can assume that Massive portion of the revenues generated by the impressions served Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 4 Operations Research 59(1), pp. 1–16, © 2011 INFORMS t t t t t in their games, the network provider must attempt to sched- • s = s¯ ∈ b ∈ b ∈ ∈ b = ik i d Dk id w Wk iw z Zi e Ezk ie ule ads across games to avoid negatively affecting the rev- estimated number of impressions provided by game i in enue stream of a particular publisher. week t that match the targeting requirements of cam- Supply. Inventory is not games, or zones, or inventory paign k; and t t t t t elements, but rather eyeballs: the total inventory available • sˆ = s¯ ∈ b ∈ b ∈ ∈ b = ik i d Dk id w Wk iw z Zi e Ezk ie to fulfill a campaign is the number of impressions generated estimated number of nonsaturated impressions provided by by gamers that match that campaign’s targeting. We use game i in week t that match the targeting requirements of point forecasts of supply in our Weekly Plan LP, which are campaign k. computed from the following point estimates of adtime: Demand. The demand for ad space arises from ad agen- • ¯t = ∈ si expected adtime of game i, week t; cies purchasing campaigns. For each week t T of the • ¯t = sid expected adtime of game i, week t, DMA d; planning horizon, Massive uses a proprietary method to • ¯t = sie expected adtime of game i, week t, inventory estimate the distributions of aggregate demand-to-come element e; and for three groups of games: premium, middle, and dis- • ¯t = siw expected adtime of game i, week t, {Week- count tiers. day, DayPart} w. Tightness of Capacity—Sell-Through. We first define: • = Defining the set of zones in game i as Zi, the set ae adtime of inventory element e in the game of inventory elements in zone z as Ez, and the sets instance started by arrival a (measured ex post); • t t = of all DMAs and all {Weekday, DayPart} tuples as D AzAzk sets of arrivals for zone z in week t, and for and W , respectively, we compute “breakouts”—proportions zone z in week t that match the targeting of campaign k, of adtime that come from a single inventory element, DMA, respectively; • = or {Weekday, DayPart}: Kae set of campaigns with targeting that matches • t =¯t ¯t = bid sid/ d∈D sid proportion of game i, week t’s both arrival a and inventory element e; and • = adtime that comes from DMA d; Ik set of games that match the targeting of cam- t t t • b =¯s / ∈ ∈ s¯ = proportion of game i, week ie ie z Zie Ez ie paign k. t = t’s adtime that comes from inventory element e; and Then, k i∈I z∈Z a∈At e∈E ae is the number • t =¯t ¯t = k i zk zk biw siw w∈W siw proportion of game i, week t’s of impressions counted ex post in week t that could have adtime that comes from {Weekday, DayPart} w. been used by campaign k (assuming we disregard competi- We use these breakouts to generate estimates of supply tion and saturation constraints). The sell-through of inven- that are used by the Weekly Plan LP. This final step requires tory element e in the game instance started by arrival a = t t the additional notation: in week t is ae k∈K qk/k, the proportion of inven- • = ae Ezk set of inventory elements in zone z that matches tory element e in arrival a that would have been allocated the targeting of campaign k; to paying campaigns if each campaign were served at the • = DkWk sets of DMAs and {Weekday, DayPart} same rate across the entire inventory space it targeted. The

tuples that match the targeting of campaign k, respectively; population mean and population standard deviation • = Pe set of spots in inventory element e; describe sell-through over aggregated blocks of inventory;

to this article and distributed this copy as a courtesy• to the author(s). = se expected adtime of inventory element e in one for example, the network sell-through over the planning game instance; horizon (i.e., computed over all games and weeks T ) has • p = = ∈ s e se/ Pe expected adtime of any spot p Pe, mean, second moment, and standard deviation: assuming each spot of Pe is equally likely to be on- ∈ ∈ ∈ ∈ t ∈ copyright screen; and t T i z Z a A e E ae ae = i z z • ∈ = 0 1 a factor that approximates the conver- ∈ ∈ ∈ ∈ t ∈ t T i z Zi a A e Ez ae sion ratio between adtime and impressions. We assume that z 2 holds ∈ ∈ ∈ ∈ t ∈ when measuring aggregate supply over many inventory ele- = t T i z Zi a Az e Ez ae ae 2 ments and/or large periods of time, adtime and impressions ∈ ∈ ∈ ∈ t ∈ t T i z Zi a Az e Ez ae are approximately equal modulo this scaling constant. Note = − 2 that if adtime was not rounded down to compute billable 2 INFORMS Additional information, including rights and permission policies, is available atimpressions, http://journals.informs.org/. then = 1. We used = 09, which was consistent with the aggregate amount of rounding down in Legacy Algorithm. Massive’s existing scheduling our data set. algorithm—the legacy algorithm—uses a single impression p Labeling the inventory elements of Ezk such that s 1 goal qk for each campaign and assumes that impressions p ··· p = s 2 s Ezk , we deﬁne Ezk 1z as the should be spread uniformly across all weeks (i.e., it does set of the “largest” z inventory elements in zone z that not support delivery schedules that ramp up impressions the match the targeting of campaign k. Finally, we compute the week before a product launch). When a new arrival spawns supply estimates for the Weekly Plan LP: a game instance, the legacy algorithm checks whether each • t = ¯t = si si estimated number of impressions provided campaign k is ahead or behind schedule and adjusts the ser- by game i in week t; vice rates accordingly. Because the legacy algorithm serves Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 5

campaign k at the same rate in each game i that its tar- Figure 1. The Weekly Plan LP. geting matches, it is usually good at spreading impressions − − t − − t max kyk kyk kyIk kyIk ∈ ∈ ∈ ∈ ∈ ∈ ∈ ∈ across MixComponents, providing impressions in a broad k K k Kt Tk k KI k k KI kt Tk set of games, and satisfying publisher concerns. The most − t − t + t ˙ t kvIk izi ImwIm ∈ ∈ ∈ ∈ ∈ ∈ ∈ = systematic shortcoming of the legacy algorithm is that it k KI kt Tk t Ti I t Tm 1n is myopic; because it does not use supply estimates, it is + s.t. x˙ + y = q − a ∀ k ∈ K 1 unable to proactively increase the service rate of a cam- k k k k t − t t t + t ∀ ∈ ∀ ∈ paign if shortages are anticipated later. lk yk xk uk yk t Tk k K 2 − ∀ ∈ ∀ ∈ lIk yIk xIk I k k K 3 4. Weekly Plan LP t − t t ∀ ∈ ∈ ∀ ∈ lIk yIk xIk I kt Tk k K 4 We periodically solve a linear program called the Weekly t t − t t t t + t t sikIk vIk xik sikIk vIk Plan to get xik, the number of impressions allocated to ∀ ∈ ∀ ∈ ∈ ∀ ∈ campaign k from game i in week t. We update the target i I I kt Tk k K 5 t t t service rates = x /s whenever the Weekly Plan is re- t + t t ∀ ∈ ∈ ik ik ik pkxik zi ri t Ti 6 solved; the Real-Time Algorithm (see §5) uses these rates k∈K to select speciﬁc campaigns for the inventory elements of ˙t +˙t =˙t ∀ ∈ ∈ xi wi si t Ti 7 a given game instance. x˙t sˆ˙t ∀ i ∈ I t∈ T ∀ k ∈ K 8 The Weekly Plan is similar to the goal program of ik ik k k t −˙t = t ∀ ∈ ∈ ∀ ∈ Bollapragada et al. (2002), which is used to generate TV xik xik aik i Ikt Tk k K 9 ad sales plans at NBC. Both formulations have an objec- ˙ t ˙ t ∀ ∈ = − ∈ wIm Im I m 1n 1 t T 10 tive that reserves premium inventory for future sales while ˙ t − ˙ t = ∀ ∈ ∈ minimizing penalty costs for relaxing the many goal con- wIm wi 0 I t T 11 m=1n i∈I straints. The NBC sales plan has mix constraints and ˙ − ˙t = ∀ ∈ weekly weighting constraints, which take a client’s goal xk xik 0 k K 12 i∈I t∈T x = q (# of commercial slots assigned from this subset k k −˙ = ∀ ∈ of inventory = target) and relaxes it to l − y x u + y, xk xk ak k K 13 where y 0 is a slack variable penalized in the objec- t − t = ∀ ∈ ∀ ∈ xk xik 0 t Tk k K 14 ∈ tive and lu are lower and upper bounds. We follow the i Ik same construction; however, our mix constraints specify an − ˙t = ∀ ∈ ∀ ∈ xIk xik aIk I k k K 15 ∈ ∈ impression goal qIk for MixComponent I (which is a i It Tk group of games), whereas at NBC the mix targets are for a t − t = ∀ ∈ ∈ ∀ ∈ xIk xik 0 I kt Tk k K 16 single TV show. We also add a weekly mix constraint based i∈I t on the target q , a constraint that spreads impressions ˙t − ˙t = ∀ ∈ ∈ Ik xi xik 0 t Ti 17 across all games within a MixComponent, and a constraint k∈K

to this article and distributed this copy asthat a courtesy toensures the author(s). that each game is assigned a minimum dollar v w x y z 0 for all forms of these variables18 value in revenue. The NBC sales plan bounds the number of slots allocated to a given {Show, Week}, which partially The constant a denotes all impressions of x achieved up enforces saturation and competition constraints; saturation to the present, and x˙ denotes all impressions of x planned is then handled by a heuristic they call the ISCI Rotator, = +˙ copyright into the future. Hence, x a x. Similarly, dots on other which spreads commercials of the same campaign apart constants and variables indicate quantities for the remain- chronologically. We follow a similar approach by includ- der of the relevant time interval. holds t ing bounds on xik that partially enforce the saturation cap, Constraints (1) through (5) are goal-type constraints and defer the consideration of competition and saturation that model campaign contract requirements that may be to the Real-Time Algorithm. Finally, we note that NBC’s relaxed and penalized in the objective: Constraint (1) goal program is an integer program (as discrete commercial (end-of-horizon impression goal) makes sure that for each INFORMS Additional information, including rights and permission policies, is available atslots http://journals.informs.org/. are allocated) that generates a sales plan for a sin- campaign k, the number of impressions planned in the ˙ gle campaign, whereas we allocate impressions for many remainder of the horizon xk, plus the planned shortfall yk, campaigns simultaneously using a linear program. must equal the remaining end-of-horizon impression goal − + The Weekly Plan LP appears in Figure 1. For com- qk ak ; constraint (2) (weekly impression goals) t pactness, some quantities are shown with lower and upper ensures that the number of planned impressions xk is t t bounds within the same constraint; hence, slack variables between the bounds lk and uk for each campaign k and exist on both sides of some constraints. The variables are week t; constraint (3) (end-of-horizon mix target) ensures

, v, w, x, y, z; all other quantities are constants; see that at least lIk impressions from MixComponent I are Table 1 for the full list of notation. Because we re-solve the allocated to campaign k; and constraint (4) (weekly mix t Weekly Plan LP over time, we divide time into two parts: targets) ensures that at least lIk impressions from Mix- Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 6 Operations Research 59(1), pp. 1–16, © 2011 INFORMS

Table 1. Notation for Weekly Plan LP. Indices i, k, t = a game, a campaign, a week, m = a segment of the piecewise-linear expected revenue function for unused inventory, I = a set of games (can be either a MixComponent or a tier). Sets , K, T = all games, all paying campaigns, all weeks, = Ik games that campaign k can be served in, = Tk weeks that campaign k can be served in. Collections (sets of sets) = all MixComponents, = k MixComponents for campaign k, = all tiers of games. Parameters t = ak, aIk, aik # of impressions served to date for campaign k, for campaign k in MixComponent I, and for campaign k in game i during week t, t t = lk, uk bounds on the # of impressions to show for campaign k in week t, t = lIk, lIk lower bounds for the # of impressions to serve to campaign k of MixComponent I over all weeks, to campaign k of MixComponent I in week t, n = # of piecewise-linear segments in expected revenue function, = pk price per impression paid by campaign k, = qk end-of-horizon impression goal for campaign k, t = ri minimum revenue amount for game i, week t, t = si expected supply of game i in week t, t = sik expected supply of game i in week t that matches campaign k, ˆt = sik expected nonsaturated supply of game i in week t that matches campaign k, t = Im marginal revenue in the mth segment of the piecewise-linear expected revenue function for tier I, week t, t = Im width of the mth segment of the piecewise-linear expected revenue function for tier I, week t. Slack Variables and Penalty Costs t = vIkk maximum absolute deviation from the nominal number of impressions to serve campaign k in game i at week t, taken over all games i in MixComponent I, penalty cost for not spreading impressions across all games in a MixComponent, = ykk shortfall amount, penalty cost for the end-of-horizon impression goal for campaign k, t = ykk amount bounds violated, penalty cost for the impression goal for campaign k, week t, = yIkk amount bounds violated, penalty cost for the end-of-horizon impression goal for campaign k, MixComponent I, t = yIkk amount bounds violated, penalty cost for the impression goal for campaign k, MixComponent I, week t, t = zi i shortfall amount, penalty cost for the minimum revenue target of game i, week t. Decision Variables t = to this article and distributed this copy as a courtesyw toi the author(s). impressions of game i available to satisfy future demand in week t, t = wIm impressions of tier I available to satisfy future demand in week t, placed in segment m of the piecewise-linear expected revenue function, = xk impressions allocated to campaign k over the planning horizon, t = xk impressions allocated to campaign k from week t, t = copyright xi impressions allocated from game i, week t, t = xik impressions allocated to campaign k from game i, week t, = xIk impressions allocated to campaign k from MixComponent I over the planning horizon, t = holds xIk impressions allocated to campaign k from MixComponent I, week t, t = Ik nominal rate to serve impressions of MixComponent I to campaign k in week t.

Notes. Our model also makes use of some quantities not listed in the above table, but that are related to the quantities listed above. In particular, quantities which we denote with a dot superscript are analogs of their nondotted cousins. The dotted quantity measures the component of the INFORMS Additional information, including rights and permission policies, is available atnondotted http://journals.informs.org/. quantity that applies from the current time (of solving the Weekly Plan LP) until the end of the relevant time period. For example, ˙ t t t ˙ t xik is the analog of xik. Whereas xik measures the number of impressions allocated to campaign k from game i, week t, the quantity xik is the number of impressions allocated to campaign k from game i in the remainder of week t. In the case that a portion of week t has elapsed t ˙ t t = t + ˙ t at the time the Weekly Plan LP is being solved, the xik and xik do not coincide and in fact are related by the expression xik aik xik.

t Component I are allocated to campaign k in week t. {MixComponent, Campaign, Week} as the variable Ik, Constraint (5) (spread impressions to all games within serving all games in a MixComponent at the same rate t = t ∀ ∈ a MixComponent) models the following idea: “For each requires satisfying ik Ik i I. By multiplying t t = t t {MixComponent, Campaign, Week}, try to set the same ser- xik/sik Ik through by sik and introducing the slack t t t vice rate for all games.” Deﬁning the service rate for a vIk, we obtain constraint (5). Note that sikIk can be Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 7

interpreted as the nominal number of impressions to plan (shadow) cost of assigning additional inventory to extend t in game i, and so vIk penalizes the maximum absolute the campaign past its scheduled end date. Transaction costs deviation from the nominal number of impressions across include all manual processing to issue the make-good, all games in MixComponent I. including getting approval to extend the campaign. Loss of Constraint (6) (minimum revenue target for each game) goodwill models the advertiser’s displeasure with the qual- is the only goal-type constraint that models publisher ity of service they received and reﬂects the estimated loss contract requirements. This constraint ensures that enough of repeat business. Transaction costs and goodwill costs impressions are allocated to game i in week t to guarantee are likely signiﬁcant, yet are extremely hard to estimate; t at least ri dollars in revenue. fortunately, with regard to linear programming, exact esti- Constraint (7) (supply) states that for the remaining por- mation of the penalty costs is not as important as their rela- tion of week t in game i, the number of impressions tive ranking. Massive suggested the following penalty costs, = planned, plus the number of impressions left unplanned, which are proportional to campaign prices pk: k pk per must equal the forecast supply. impression short of the end-of-horizon goal for campaign k; = Constraint (8) (saturation cap bound) partially enforces k 01pk per impression over/under the allowed deviation = the saturation cap by bounding the number of impressions from the weekly goal; k 01pk over/under the allowed assigned to campaign k in game i, week t by the estimated deviation from the end-of-horizon goal for each MixCom- sˆt = nonsaturated supply ik. ponent; k 005pk over/under the allowed deviation from Constraint (9) (link total planned impressions with = the weekly goal for each MixComponent; and k 005pk remaining planned impressions) ensures that for each for not spreading impressions evenly across all games in {Game, Campaign, Week}, the total number of impressions ∈ a MixComponent. We also used i 05 025 01 as the t planned, xik, must equal the actual number of impres- per-dollar penalties of falling short of the minimum revenue t sions achieved up to the present, aik, plus the number of target of game i (penalties are differentiated by tier). impressions planned from the present until the end of the Note that in practice, make-good costs should also be ˙t t = week, xik. For weeks t 2, we always have aik 0, and so increased for preferred customers (which are often given t =˙t this constraint reduces to xik xik. quantity discounts) to compensate for the fact that the Constraints (10)–(11) model a piecewise-linear compo- above formulas assign low penalty costs to campaigns with nent of the objective function and are described with the low prices. objective in §4.1. Constraints (12)–(17) link the various Objective Term 2—General Form. Revenue potential forms of x. In an actual implementation, only the variables of unscheduled inventory is: t ˙t xik and xik are required; the other forms of x can be written t ˙t as linear combinations of xik and xik. We use the other t t fI wI forms of x to keep constraints (1)–(11) readable. I∈t∈T Constraints (3) and (4) do not have upper bounds because MixComponents are not mutually exclusive: game i may be Because supply and demand change over time, the both a sports game and a racing game; in that case, impres- scarcity of inventory, and therefore the value of unscheduled

to this article and distributed this copy assions a courtesy to the author(s). allocated to game i count toward two mix targets. inventory, will change over time. By scheduling appropri- Had upper bounds existed, they may need to be violated to ately, we maximize the total dollar value of unscheduled satisfy the end-of-horizon goal (constraint (1)). inventory, i.e., the expected future sales revenue. The general form of objective term 2 is as listed above, where is the set of all tiers of games (premium, middle, dis-

copyright 4.1. The Objective Function t count), T is the set of all weeks, and fI is a function that The objective is to maximize revenues that can be affected t by scheduling; it contains two terms. values the quantity of unscheduled inventory wI for tier I in holds Objective Term 1. Penalty costs for breaking con- week t. Suppressing subscripts and superscripts, the value tracts is: function for unscheduled inventory fw of a particular {Tier, Week} is: − y − yt − y k k k k k Ik + INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. k∈K k∈Kt∈T k∈KI∈ fw= pƐminXw = pƐX − X − w k k − t − t − t w kyIk kvIk izi = p Gx dx ∈ ∈ ∈ ∈ ∈ ∈ k KI kt Tk k KI kt Tk t∈Ti∈ 0 We model the costs of falling short of the goals mod- where p is the fixed price for the type of inventory under eled by constraints (1)–(6) as negative revenues from the consideration, X is a random variable that models market delivery of make-goods for broken contracts. Make-good demand, Gx is the cumulative distribution function of costs include direct compensation for underdelivery, trans- demand, and Gx = 1−Gx. Thus, if we leave w units of action costs, and loss of goodwill. Direct compensation is inventory available, expected sales is ƐminXw, yield- either the dollar amount refunded to the advertiser, or the ing fw dollars in expected revenue. Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 8 Operations Research 59(1), pp. 1–16, © 2011 INFORMS = w = t computed by the Weekly Plan LP, while also obeying Because f w pd/dw 0 Gx dx pGw and G ik is a nonincreasing function, fw is concave increasing for campaign targeting, saturation, and competition constraints. all demand distributions of X. Therefore, we can approx- The Real-Time Algorithm serves the same purpose imate fw by a piecewise-linear function with n seg- as the ISCI Rotator Algorithm for scheduling commer- ments of successively smaller positive slope. Denoting the cials in broadcast television (Bollapragada et al. 2004, marginal revenues (slopes) as m and segment widths as Bollapragada and Garbiras 2004). A broadcast TV sales = m for the m 1n segments, the expected revenue from plan (analogous to our Weekly Plan) specifies the set of future sales is modeled by the linear form: commercials to display during each airing of each show, Objective Term 2—Linear Form. The Weekly Plan LP leaving the ISCI Rotator to choose, for each ad, the com- uses this linear form in practice. mercial breaks and positions within those breaks where t ˙ t ImwIm each ad should be aired. The ISCI Rotator allocates ads to I∈t∈Tm=1n slots such that saturation is managed (two airings of the and is accompanied by constraint (10) (bound the same commercial are as evenly spaced as possible) and unplanned impressions in each segment) and constraint (11) competition constraints are obeyed (competing brands are (link unplanned impressions by {Tier, Week, Segment} to not displayed in the same commercial break). Although {Game, Week}). there are some high-level similarities, the Real-Time Algo- t Because the slope parameters Im are decreasing in m rithm operates differently than the ISCI Rotator. This is ˙ t = =⇒ ˙ t = and we are maximizing, wIm 0 wIm+1 0. The because targeting constraints add complexity to the Real- ˙ t ˙ t bounds on wIm, namely Im, ensure that impressions Time Algorithm and because the Real-Time Algorithm sac- ˙ t are accounted for in the correct segment, where Im rifices optimality for speed: it is run millions of times per is defined as follows ( is the percentage of week 1 day compared to the ISCI Rotator’s nightly execution. remaining):⎧ The Real-Time Algorithm performs two operations: (1) it ⎨ t = adjusts the service rates for the current game instance, and Im for t 1 ˙ t = (2) assigns campaigns to inventory elements in accordance Im ⎩ t with the adjusted service rates. Notation for the Real-time Im for t 2 We assumed stationarity in the tests we performed, and Algorithm is summarized in Table 2. computed the required cdfs from historical data. In practice, we expect the historical cdf to be used as a baseline 5.1. Stage 1: Adjust Service Rates for that Massive can manually align with their sales projections the Current Game Instance (keeping the general shape of the cdf the same, but shifting An arrival of type a = {Zone, DMA, Weekday, DayPart} is its mean). For an example that uses a discrete demand dis- initially assigned the service rate matrix a, the elements tribution to compute and , see §EC.1 of the electronic of which are companion to this paper, available as part of the online ⎧ version at http://or.journal.informs.org/. ⎪1 if campaign k matches the targeting ⎪ ia k Alternately, when only ƐX is specified, we can ⎨ of both IEC c and the {Zone, DMA, to this article and distributed this copy as a courtesy to the author(s). = Ɛ Ɛ = use Gx 0ifx< X 1ifx X to com- ck ⎪ Weekday, DayPart} specified by a Ɛ = Ɛ Ɛ Ɛ ⎪ pute fw X w if x< X, X if x X. ⎩⎪ By Jensen’s Inequality, we know fw ƐX ≡ 0 otherwise ƐminƐX w ƐminXw ≡ fw; thus, fw is a Ɛ copyright function that is bounded above by fw X, yet has sim- where ia is the game that includes the zone specified in → → 1 ilar asymptotic behavior; i.e., fw w as w 0, and a, and ia k is the service rate for campaign k, game ia, fw→ ƐX as w → . An appropriately chosen spline week 1 from the Weekly Plan. holds can be approximated with a few piecewise-linear segments Algorithm 1 modifies the rate matrix a to satisfy (1) to get the piecewise-linear form of fwused by our model. competition constraints and (2) the constraint k∈K ck 1 ∀ ∈ ac Finally, we point out that in theory, censored demand can c Cz, which states that for each IEC c in the set of be handled by the Kaplan-Meier estimator (Talluri and van IECs Cz that intersect the current zone z, the sum of the INFORMS Additional information, including rights and permission policies, is available atRyzin http://journals.informs.org/. 2005); however, this is not necessary when working service rates of campaigns Kac eligible for service to this with tiers: individual games may get sold out, but because {Arrival, IEC} must not exceed 1. In Step 1, we compute tiers are large aggregations of games, excess inventory usu- WEIGHTED a weighted average rate k for each campaign over ally exists. all IECs in the zone. Next, we use the weighted average rates to assign one random campaign from each set 5. Real-Time Algorithm of competing brands a nonzero service rate (Steps 2–5). Each arrival to the ad server invokes the Real-Time Algo- We have assumed that the groups of competing brands are rithm, which assigns campaigns to inventory elements mutually exclusive; however, a straightforward extension of in the player’s game instance. The Real-Time Algorithm this algorithm applies to overlapping groups of competing makes this allocation in accordance with the service rates brands. Finally, in Steps 6–12, the algorithm decreases the Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 9

Table 2. Notation for the Real-Time Algorithm. Indices a = an arrival, b, b, b∗ = a bucket, = bu the “unpaid” bucket, c = an IEC, k, k, k, k∗ = a campaign, pp∗ = a spot, r = a region, u = all nonpaying campaigns. Functions of Indices bck = bucket for IEC c, paying campaign k, cp = IEC corresponding to spot p, kb = campaign corresponding to bucket b, ep = inventory element corresponding to spot p, rp= region corresponding to spot p. Input: Sets = Cz IECs in zone z, = Ez inventory elements in zone z, = Ecz inventory elements in zone z and IEC c, = Ka campaigns with targeting matching arrival a, = Kac campaigns with targeting matching both arrival a and IEC c, ≡ = Pb Pcbz spots that can be placed in bucket b, = Pe spots of inventory element e, = = Pz e∈E Pe spots in zone z, = z = Pcz e∈E Pe spots in zone z and IEC c, cz = Rz regions in zone z. Input: Collections (sets of sets) = a groups of campaigns that are competing brands with targeting that matches arrival a. Input: Parameters = se expected adtime of inventory element e, sp = s /P =expected adtime of spot p, ep ep = = scz e∈E se expected adtime of IEC c in one game instance of zone z, cz = k contracted end date of campaign k, = z saturation cap of zone z. Inputs/Variables (inputs that get adjusted) = ck service rate for campaign k in IEC c, = a service rate matrix for arrival a. Variables: Scalars = nrk number of times a spot from region r was allocated to campaign k, to this article and distributed this copy as a courtesy to the author(s). = vb unallocated expected adtime of bucket b, = xp unallocated expected adtime of spot p, = ybp expected adtime of spot p allocated to bucket b, WEIGHTED = k weighted average service rate for campaign k, computed over all IECs, , = used in local computations. copyright Variables: Sets K∗ = campaigns of competing brands, U holds K = campaigns strictly under the saturation cap, B = buckets for paying campaigns, P ∗ = spots p∗ that can be placed in the chosen bucket b, and are in a region rp∗ where we have allocated campaign kb the fewest number of times, P ∗∗ = spots not completely allocated, = INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. Pk spots in which campaign k should be served.

← ∈ ∗ WEIGHTED service rates of some campaigns (if necessary) in order for 3: k k K with probability k ∀ ∈ k∈K ck 1 c Cz to hold; note that the campaigns WEIGHTED ac / ∗∈ ∗ ∗ with the latest end dates (which have the most time to catch k K k ← ∀ ∈ up later) are chosen. See §EC.2.1 for a detailed example. 4: ck k∈K∗ ck c Cz, and ← ∀ ∈ ∗\ ∈ Algorithm 1 (AdjustServiceRates) ck 0 k K k , c Cz WEIGHTED 5: end for 1: ← ∈ s / ∈ s ∀ k ∈ K k c Cz cz ck c Cz cz a ∗ ∈ ∈ 2: for all K a do 6: for all c Cz do Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 10 Operations Research 59(1), pp. 1–16, © 2011 INFORMS

∈ ← p ← ∀ ∈ ∪ \ 7: Label the campaigns k Kac such that 18: ybp s ; ybp 0 b B bu b ; ··· ← ∪ 1 2 Kac Pkb Pkb p + 8: ← ∈ − 1 ; k ← 1 k Kac ck 19: end for ∈ = 9: while >0 do 20: for all k k Pk >zk u do 10: ← min ; ← − ; ck ck ck 21: while Pk >z do ← − k ← k + 1 p U 22: p ← arg min ∈ s ; K ← k P < k= u p Pk k z 11: end while 23: if k∈KU cp k > 0 then 12: end for ← ∈ U 24: k k K with probability 13: return a cp k/ k∗∈KU cp k∗ 25: else 5.2. Stage 2: Assign Campaigns to 26: k ← u Inventory Elements 27: end if ← \ ← ∪ Let Ka be the set of paying campaigns with targeting that 28: Pk Pk p ; Pk Pk p matches arrival a, and let u represent all unpaid campaigns. 29: end while For each campaign k ∈ K ∪ u, Algorithm 2 outputs a 30: end for a set of spots Pk in which campaign k should be served. 31: return P The algorithm begins by creating a set of buckets B; the size of bucket bck ∈ B is the expected adtime for pay- In Steps 6–10, spots are fractionally assigned to buckets ing campaign k in IEC c, assuming campaign k is served b ∈ B; i.e., a spot may be assigned to more than one bucket at rate ck. An inﬁnitely large bucket bu B called the at this stage. At each iteration of the loop, we randomly “unpaid bucket” is also created; it is used when service draw a nonfull bucket b and a noncompletely assigned spot rates are too low to assign all spots to paying campaigns. p, and assign a portion (or all, if possible) of spot p to = The main variables are: vb unallocated expected adtime bucket b . We bias toward assigning large spots to large = of bucket b; xp unallocated expected adtime of spot p; buckets, which achieves a tight packing; however, we do = ybp expected adtime of spot p allocated to bucket b; and not always pick the largest bucket and spot: buckets are = nrk number of times a spot from region r has been allo- chosen randomly so the visual placement of ads within cated to campaign k. We will sometimes use index func- the game is randomized, and spots are chosen to spread tions such as rp = region corresponding to spot p; and multiple copies of the same campaign to spots in differ- kb = campaign corresponding to bucket b. Recall from ent regions. The chosen spot p is the spot with the largest §3 that the expected adtime of any spot p of inventory amount of unallocated expected adtime among those in the p ≡ p = ∗ ∗ ∗ element e is s s e se/ Pe . Steps 1–5 comprise the set P , where P is the set of all spots p that can be placed initialization phase. in chosen bucket b and are in a region rp∗ where we have allocated campaign kb the fewest number of times. Algorithm 2 (AssignCampaigns) In Steps 11–14, any spots that did not ﬁt into the buckets ← ← ∀ ∈ ←

to this article and distributed this copy as a1: courtesy tov the author(s). ; y 0 p P ; B bu bup z for paying campaigns B are assigned (in whole or in part) for all c ∈ C k ∈ K do 2: z, ac to the unpaid bucket bu. It can be shown that when the ≡ ← 3: Create bucket b bck; vb ckscz; algorithm is initialized, b∈B vb p∈P xp, so all buckets ← ∀ ∈ ← ∪ z ybp 0 p Pz; B B b b ∈ B are always filled at this point. 4: end for In Steps 15–19, the algorithm executes randomized copyright ← p ∀ ∈ ← ∀ ∈ ∈ 5: xp s p Pz; nrk 0 r Rzk Ka rounding (see Raghavan and Tompson 1987): each spot p 6: while b∈B vb > 0 do gets assigned in whole to bucket b (and removed from all holds ← ∈ ∗ ∗ p 7: b b B with probability vb/ b ∈B vb other buckets) with probability ybp/s (the proportion of ∗ ← ∗ ∈ = 8: P p Pb nrp∗ kb minp∈P nrpkb; spot p currently assigned to bucket b). At this point, buck- ← b p arg maxp∈P ∗ xp ets may be perfectly filled, overfilled, or underfilled—all ← ← + 9: minvb xp ; ybp ybp ; cases are feasible. A perfectly filled (respectively, over- INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. ← − ← − vb vb ; xp xp ; filled/underfilled) bucket bck corresponds to serving ← + nrp kb nrp kb 1 campaign k in IEC c exactly at (respectively, above/below) 10: end while the service rate ck in expectation. ∗∗ ← ∈ 11: P p Pz xp > 0 In Steps 20–30, the algorithm adjusts the final solu- ∗∗ = 12: if P then tion Pk (the set of spots in which to serve campaign k)if 13: y ← x ∀ p ∈ P ∗∗, x ← 0 ∀ p ∈ P ∗∗ bup p p the saturation cap is exceeded (the same campaign appears 14: end if in more than z spots). We reassign the smallest spots p ← ∀ ∈ ∪ 15: Pk k Ka u to minimize the resulting change to the service rates; we ∈ 16: for all p Pz do first reassign spots in whole to paying campaigns that are ← ∈ ∪ p 17: b b B bu with probability ybp/s under the saturation cap, drawn proportionally to cp k Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 11

when there are multiple such campaigns, and then to the forecasts for one week into the future have a 95% con- unpaid campaign bucket. See §EC.2.2 for an example of fidence interval of 1/ , where = e196 and = 2 Algorithm 2. (this confidence interval is symmetric on the logarithmic scale). For = 004 and = 02, this confidence interval is 5.3. Properties of the Solution 085 117 and 046 219, respectively; i.e., when = 02 and = 0, a one-week advance forecast for the actual At its conclusion, the Real-Time Algorithm has assigned a value of 1,000 is between 460 and 2,190 with probability campaign to each spot such that: 0.95. Also, for the same , confidence intervals grow wider 1. DMA, Weekday, DayPart, and IEC targeting are sat- as the forecasted period moves farther into the future. isfied because only campaigns that target the current arrival Our tests on historical data show that we significantly a receive nonzero service rates; reduce make-good costs and increase sales revenue poten- 2. Competition constraints are satisfied, due to tial relative to the legacy algorithm. Make-good costs Algorithm 1; (objective term 1 in §4.1) decrease by 80%–87%, depend- 3. The saturation cap is satisfied and local oversaturation ing on the magnitude of forecast error (see Figure 2). In the is controlled, due to Algorithm 2; most conservative case of = 02 (it is expected that fore- 4. Each campaign is served in expectation at the rate casts are at least this accurate), make-good costs decrease specified by the Weekly Plan LP (except perhaps for a by 80%. few campaigns with later due dates that had their service Revenues from the future sales of ad space (objective rates reduced by Steps 6–12 of Algorithm 1). Thus, in term 2 in §4.1) cannot be evaluated directly: because we expectation, our algorithm meets the impression goals, mix are backtesting, the sales process in our actual data cannot requirements, spread constraints, and revenue targets speci- be affected by our scheduling algorithm. Therefore, we use fied in the Weekly Plan while maximizing revenues affected the distribution of the age of served impressions, the time by scheduling decisions. This follows because, excluding between the sale of the campaign and when the impression adjustments made to satisfy the saturation cap, Algorithm 2 was served, as a proxy. fills buckets exactly to their fill lines in expectation; and A good scheduling algorithm will try to use the least 5. Variance in the service rates is low because buckets costly inventory to satisfy existing campaigns, leaving pre- are packed close to their fill lines by using small spots to mium inventory reserved for future sales. Therefore, a good finish the packing. (These are also the spots that get reas- scheduling algorithm should shift impressions from tier 1 signed by randomized rounding and to enforce the satura- (premium) games into tier 2 (middle) or tier 3 (discount) tion cap.) games when mix constraints allow. When impressions cannot be appreciably shifted between tiers, a good scheduling 6. Experimental Findings

6.1. Primary Benefit Figure 2. Performance of our algorithm compared to the legacy algorithm for different levels of Using 26 weeks of historical data, we experimentally eval- forecast error . to this article and distributed this copy asuate a courtesy to the (backtest) author(s). the performance of our algorithm using a version of Massive’s legacy algorithm that was provided as Ex-post objective comparison—Penalty costs the benchmark. We set the horizon of the Weekly Plan LP 250 at 13 weeks; therefore, at each point during the backtest we End of campaign impression goal End of campaign mix goal copyright need 13 weeks of inventory supply forecasts (for games, as Weekly campaign impression goal 200 well as inventory elements, DayPart/Weekdays, and DMAs Weekly campaign mix goal within games) as input. We generate forecasts from actual Balance impression across holds data by applying a random multiplicative noise term, the games in same mixcomponent magnitude of which is parameterized by . We backtest 150 Minimum weekly revenue target by game through weeks t = 114, using generated forecasts from weeks t through t + 12; the Weekly Plan LP is re-solved INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. hourly. Penalty cost 100 We use five forecast instances, which range from a perfect forecast ( = 0) to a highly variable forecast ( = 02). The full description of our forecast generation scheme, 50 ¯t ¯t ¯t ¯t which produces adtime estimates si , sid, sie, and siw, defined in §3, is given in §EC.3. To summarize, we generate multiplicative forecast errors for a forecast h weeks from today 0 by taking the product of h lognormal random variables; 0 0.0016 0.008 0.04 0.2 Legacy the lognormal factors have median e, where is a ran- Model dom variable dependent on with Ɛ = 0. When = 0, Note. The legacy algorithm does not use forecast data. Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 12 Operations Research 59(1), pp. 1–16, © 2011 INFORMS

Figure 3. The tier distribution of impressions served (top left) and age distributions of impressions served for each tier, for the legacy algorithm and our proposed algorithm (using = 02).

Percentage of impression by tier Tier 1 age CDF 1.0 100

0.8 80 0.6 60 Tier 3 Proposed Tier 2 Legacy 0.4 (%) 40 Tier 1 0.2 20 0 0 252321191715131197531 Proposed Legacy Weeks

Tier 2 age CDF Tier 3 age CDF 1.0 1.0

0.8 0.8

0.6 0.6 Proposed Proposed Legacy Legacy 0.4 0.4

0.2 0.2

0 0 252321191715131197531 252321191715131197531 Weeks Weeks

algorithm should try to serve tier 1 inventory as early as tier 3 impressions are served later: 63% of tier 1 impres- possible, and serve tier 2 and tier 3 inventory later on, again sions are 6 weeks old or younger using our algorithm, com- saving the most valuable inventory for future sales. pared to 56% under the legacy algorithm. In Figure 3, we compare the age distribution from our algorithm with that of the legacy algorithm; in all cases, 6.2. Stability of Our Algorithm the forecast error parameter is set to = 02 (the most Figure 4(a) shows that our algorithm makes steady, contin- conservative scenario). As expected, our algorithm shifts a uous progress toward meeting the end-of-campaign impres- substantial portion of impressions from tier 1 into tiers 2 sion goals; the plot shows the cumulative number of to this article and distributed this copy as a courtesy to the author(s). and 3. The legacy algorithm serves 92% of its impressions impressions logged by a single campaign from its start date from tier 1 games, whereas our algorithm serves just under to its end date. Figure 4(b) plots the cumulative number 80% from tier 1. Furthermore, our algorithm serves tier 1 of impressions logged by this same campaign in its ﬁrst and tier 2 impressions earlier in a campaign’s life, whereas week. Notice that the LP’s plan is revised midweek, at copyright

Figure 4. The progress of the Real-Time Algorithm (labelled “actual”) plotted over time for the largest campaign in the holds system.

(a) LP solution performance: (b) LP solution performance: Tracking campaign goal Tracking weekly campaign goal 2,500,000 180,000 INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/. 150,000 2,000,000 120,000 1,500,000 Goal Actual 90,000 Goal 1,000,000 Planned 60,000 Actual Impressions Impressions 500,000 30,000

0 0 01/01/07 01/22/07 02/12/07 03/05/07 03/26/07 01/01/07 01/03/07 01/05/07 01/07/07 Notes. The forecast error parameter is the most conservative ( = 02). In (a) the goal is the end-of-campaign impression goal; in (b) we show the end-of-week impression goal and the number of impressions planned this week by the Weekly Plan LP. Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 13

which point the LP decides it is less costly to overrun the The test cases were constructed with mean sell-through at weekly goal a bit than to use inventory from future weeks either (approximately) 27% or 52% and targeting percent- to serve this campaign. Our algorithm successfully meets ages from 1% to 10%; as a result, there is a large range for the end-of-week target, despite the fact that this campaign the standard deviation of sell-through. We use = 02in starts midweek. all test cases. As can be seen by Figure 5(a), our algorithm outper- 6.3. Performance Under Higher Sell-Through and forms the legacy algorithm in all of the six high-load test Tighter Targeting cases, confirming that our approach is robust. Relative outperformance, however, begins to decrease when the impres- We now evaluate our algorithm as the number of cam- sion goal multiplier is high (i.e., mean sell-through is high) paigns in the system increases and the supply constraints and the number of matching DMAs is low (i.e., target- become tighter (or infeasible in some cases), thereby val- ing percentage is low). There is a nearly 60% make-good idating our approach as sales volume increases. We use cost reduction for test case 5 (impression goal multiplier = sell-through and targeting percentage to measure the tight- 12, number of matching DMAs = 40), yet for test case 2 ness of the supply constraints. As defined in §3, the mean (impression goal multiplier = 24, number of matching and standard deviation of sell-through at the network level DMAs = 1) the make-good cost reduction is only 5%. The describe the tightness of supply: a high standard deviation relative performance of our algorithm diminishes because of sell-through indicates that in the space of all impres- there are regions of the network in which the sell-through sions in the network, there are pockets of inventory with is above 100%—that is, demand exceeds supply, and there- high sell-through, corresponding to intersections of games, fore make-good costs are inevitable. We therefore compute DMAs, IECs, weekdays, and dayparts that are in high an approximate lower bound on the make-good costs by demand. The targeting percentage of campaign k, com- = t t solving the Weekly Plan with perfect forecasts ( 0) and puted as ∈ ∈ s / ∈ ∈ s , represents the t Tk i Ik ik t Tk i i only the make-good cost term in the objective. This is not a percentage of impressions in the network that campaign true lower bound because, as described in §3, we compute k targets. A low targeting percentage (i.e., tight targeting) t ˆt t sik and sik using assumed independent adtime breakouts bid, alone does not make a campaign difficult to serve, but when t t bie, and biw. a tightly targeted campaign targets inventory that also has Using this bound (LB), we can measure the percentage high sell-through, serving this campaign is difficult. of the gap that we close relative to the legacy algorithm, We generate six additional test cases with different lev- defined as 1 − (Our Algorithm Cost − LB)/(Legacy Algo- els of sell-through and targeting and benchmark the per- rithm Cost − LB). As seen in Figure 5(b), the percentage formance of our algorithm relative to the legacy algorithm. gap closed is 30% or better for most cases, dropping only Each test case is built by augmenting the original problem when targeting gets very tight. This can be interpreted as instance from §6.1 (hereby called “status quo”) with a set a 30%+ cost savings on the part of the network that can of 200 randomly generated campaigns. The test cases dif- be affected by good scheduling under reasonably high sell- fer in two properties of the added campaigns: (1) the mean through and tight targeting. In particular, if we order these daily impression goal relative to the empirical distribution

to this article and distributed this copy as a courtesy to the author(s). test cases by the standard deviation of the sell-through as in (which affects sell-through), and (2) the number of DMAs Figure 5(c), we can see a clear relationship between relative that match (which affects targeting). The other campaign outperformance and increasing variability of sell-through. parameters are consistent with the empirical distributions. Even though the relative improvement is smaller, because The six test cases, which are listed in Table 3, have target- the absolute magnitudes of the make-good costs in the copyright ing percentages and coefﬁcients of variation of sell-through highly constrained test cases are higher, a 5% cost sav- in the same general range as the “status quo” test case, ings for test case 2 translates into a greater absolute dollar holds although some aspect of the instance (most often mean sell- amount saved when using our algorithm than the 60% cost through) is more highly constrained. (The precise values of savings for test case 5 (see Figure 5(d)). Thus, our algo- these parameters for the status quo case are proprietary.) rithm signiﬁcantly reduces make-good costs in all of the scenarios with high sell-through and tight targeting that we INFORMS Additional information, including rights and permission policies, is available atTable http://journals.informs.org/. 3. Test cases for tight supply constraint tests. consider.

Impression Number of Standard goal matching Mean deviation of Targeting 7. Implementation and Learnings Instance multiplier DMAs sell-through sell-through percentage 7.1. Implementation 1121027 675 0018 2241050 1188 0011 Our collaboration has led to a paradigm shift at Massive: 31210028 047 0037 Previously, their ad-serving architecture was entirely execu- 42410052 088 0034 tion based. After our project, Massive realized the beneﬁts 51240028 028 0097 of hierarchical planning, where you ﬁrst produce a plan and 62440052 060 0097 then you execute that plan. As of December 2009, a staged Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 14 Operations Research 59(1), pp. 1–16, © 2011 INFORMS

Figure 5. Performance of all six high-load test cases + status quo.

(a) Percentage cost savings relative to (b) Percentage of gap closed relative to legacy model legacy model 100 100 Status quo Status quo 80 80

60 5 = {1.2, 40} 60 5 = {1.2, 40} (%) (%) 3 = {1.2, 10} 6 = {2.4, 40} 40 3 = {1.2, 10} 40 6 = {2.4, 40} 1 = {1.2, 1} 4 = {2.4, 10} 20 4 = {2.4, 10} 20 2 = {2.4, 1} 1 = {1.2, 1} 2 = {2.4, 1} 0 0 0 0.5 1.0 1.5 2.0 2.5 0 0.5 1.0 1.5 2.0 2.5 Imp goal multiplier for added campaigns Imp goal multiplier for added campaigns

(c) Percentage of gap closed relative to (d) Absolute cost savings relative to legacy model legacy model 100 Status quo 4 = {2.4, 10} 80 5 = {1.2, 40} 6 = {2.4, 40} 60 3 = {1.2, 10} 3 = {1.2, 10} (%) 40 6 = {2.4, 40} 4 = {2.4, 10} 1 = {1.2, 1} 1 = {1.2, 1} 2 = {2.4, 1} 20 2 = {2.4, 1} 5 = {1.2, 40} Status quo 0 036912 0 0.5 1.0 1.5 2.0 2.5 Standard deviation of sell-through Imp goal multiplier for added campaigns Note. Subﬁgures are (a) top left, (b) top right, (c) bottom left, and (d) bottom right (vertical scale omitted for conﬁdentiality).

implementation of our model has been underway. When intermediate step toward LP-based planning, Massive cre- complete, an LP-based planning module will be run nightly ated a database to store impression goals for each {Cam- to produce daily targets for each campaign in each game, paign, Game, Day} and modified the legacy ad-serving logic and an improved legacy algorithm that exploits structural to track these daily goals. Presently, a greedy heuristic com- ideas from our Real-Time Algorithm will track these daily putes the daily goals and populates this database; when the targets. From a technology standpoint, Massive chose to LP is complete, this database will instead be populated by use C# and the Microsoft Foundation Solver API. the LP, and the real-time ad-serving logic will benefit from to this article and distributed this copy as a courtesyMassive to the author(s). decided to switch from weekly to daily granular- improved guidance without additional modification. This ity because they discovered advertisers place a higher value change to the legacy algorithm has been substantial: the on smooth delivery than previously thought, and Massive explicit goals for each game have increased the reach of expects the benefits of explicit daily goals to outweigh the Massive’s network; that is, by more effectively spreading

copyright increase in forecast error caused by planning at a finer gran- impressions across many games, Massive has increased the ularity. Moreover, due to our work, Massive knows that number of unique individuals that see each campaign, on a granularity finer than {Campaign, Game, Day} is unnec- average by 26% per week. holds essary. Recall from §6.3 that we tested how our model Another substantial improvement made to Massive’s performs when many campaigns request tightly targeted legacy algorithm involves the use of adtime estimates for inventory (i.e., Pittsburgh on Tuesday afternoon between each inventory element. Originally, Massive’s legacy algo- 4–7 p.m.), and found that even though our Weekly Plan LP rithm did not use forecasts of any kind. As a result, the INFORMS Additional information, including rights and permission policies, is available atused http://journals.informs.org/. the coarse granularity of {Campaign, Game, Week}, system tended to serve impressions erratically: the sys- and did not explicitly resolve inventory conflicts for overca- tem would overserve, learn that overserving had occurred pacitated geographic regions and times of day, performance (because there is a significant delay in counting impres- was quite good. We ascribe this to the fact that the schedule sions), compensate by stopping the service of the campaign was being updated periodically, so shortfalls in a specific for awhile, and then repeat the process later as the cam- geographic region could be compensated for by increasing paign began to starve. Our Real-Time Algorithm does not the overall service rate in that game. suffer from this effect because it uses adtime estimates In terms of the implementation’s rollout, Massive to track service rates that stay fixed between re-solves of decided the legacy algorithm should be adapted first, and the LP. To achieve smoother service rates in their exist- the LP implemented afterward. Specifically, as a crucial ing framework, Massive adapted their legacy algorithm to Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising Operations Research 59(1), pp. 1–16, © 2011 INFORMS 15

use adtime estimates for each inventory element. In partic- the saturation cap. This type of dependency between stages ular, Massive now assumes that when a campaign is placed of a hierarchical planning system is common in other areas into an inventory element, a number of impressions will of practice as well; for example, in broadcast TV, commer- be served in expectation. These “expected impressions” get cials are spread over time (thus dealing with saturation) in accumulated, and are part of the service rate calculation both the high-level sales plan and by the lower-level ISCI that Massive uses to throttle the service rate of a campaign Rotator Algorithm. up or down. As a result, service rates are now much more stable: the standard deviation of hourly impressions served 8. Conclusions is 33% lower than before the use of adtime estimates. We studied the scheduling problem of Massive Inc., a In addition to the measurable benefits of increased reach network provider of dynamic in-game advertising and and smoother campaign delivery, Massive has benefited developed (1) an LP to compute service rates for each from our collaboration in several intangible ways. First and {Campaign, Game} pair, as well as (2) a Real-Time Algo- foremost, the discovery process that we undertook with rithm that uses these service rates to assign campaigns to Massive to formulate the constraints of the linear program inventory elements whenever a gamer begins playing a new has been very valuable; in some cases, unwritten business level. We benchmarked our algorithm against Massive’s rules were uncovered and communicated to business ana- legacy algorithm and found that when tested on historical lysts and developers. The language of objectives, variables, data (1) our algorithm reduces make-good costs by 80%– and constraints has facilitated many engineering discus- 87%; (2) our algorithm reserves more impressions from sions and has been instrumental in building confidence in premium games for future sales; (3) performance is good the new, more transparent, system. Furthermore, the visi- for various levels of forecast accuracy; and (4) results hold bility of sold inventory has been improved; sales associates as sell-through increases and targeting becomes tighter, as can view the daily plan, allowing them to price games is anticipated in the future. based on capacity and to encourage advertisers to purchase Future work may include (1) explicitly modeling sup- impressions from undersold games. ply with random variables, (2) extending the model to plan at a finer granularity than {Campaign, Game, Week} 7.2. Learnings using Bender’s decomposition and/or column generation, Our learnings throughout the project led to several refine- (3) extending the model to incorporate reach goals by ments of our model. For example, our Weekly Plan LP requiring a minimum number of unique game players to did not initially include constraints to guarantee a mini- see a campaign, and (4) generalizing the model for other mum amount of revenue per game. This posed a problem ad networks with dynamic display content, such as digital when a very popular game was added to Massive’s net- TV and the Web. work: the new game was allocated a sizable number of As a result of our work, Massive has increased the num- impressions that previously would have gone to incumbent ber of unique individuals that see each campaign by, on games, thereby capturing their revenue. To be fair to all average, 26% per week, and achieved 33% smoother cam- game publishers, we introduced minimum revenue targets paign delivery, as measured by standard deviation of hourly to this article and distributed this copy asfor a courtesy to each the author(s). game. impressions served. Massive continues to improve their ad- We also learned that Massive sells some campaigns serving technology based on the models and algorithms we called “share of voice” that contractually require a percent- developed and the insights we generated. age of the total impressions generated by a game over a

copyright time period without having an absolute impression goal, 9. Electronic Companion e.g., an advertiser may request 25% of all impressions from An electronic companion to this paper is available as part Game A. This type of campaign is easy to serve in our holds of the online version that can be found at http://or.journal framework; we just set the service rate to 25%. .informs.org/. Finally, we learned that the yield factor (see §3) used to convert between adtime and impressions varies from one Acknowledgments game to the next, and also changes when the scheduling INFORMS Additional information, including rights and permission policies, is available atalgorithm http://journals.informs.org/. changes (i.e., from legacy to ours). We have rec- The authors are grateful to Frank O’Donnell, David ommended that yield factors be empirically measured over Sturman, Joe Ciaramitaro, and the rest of the team time in order to have accurate values of for each game. at Massive for numerous comments and support. This With regard to hierarchical planning systems in general, research was supported by Massive Incorporated and a we learned that it is usually not possible to have a complete William Larimer Mellon Fellowship. separation of tasks from one stage to the next. For example, both the Weekly Plan LP and the Real-Time Algorithm deal References with saturation: the Weekly Plan LP bounds service rates Abrams, Z., O. Mendelevitch, J. Tomlin. 2007. Optimal delivery of spon- (constraint (8) in §4) so that with high likelihood the rates sored search advertisements subject to budget constraints. Proc. 8th remain feasible when the Real-Time Algorithm enforces ACM Conf. Electronic Commerce, ACM, New York, 272–278. Turner, Scheller-Wolf, and Tayur: Scheduling of Dynamic In-Game Advertising 16 Operations Research 59(1), pp. 1–16, © 2011 INFORMS

Araman, V. F., I. Popescu. 2010. Media revenue management with audi- Mehta, A., A. Saberi, U. Vazirani, V. Vazirani. 2007. AdWords and gen- ence uncertainty: Balancing upfront and spot market sales. Manufac- eralized online matching. J. ACM 54(5) 22:1–22:19. turing Service Oper. Management 12(2) 190–212. Nakamura, A., N. Abe. 2005. Improvements to the linear programming Bollapragada, S., M. Garbiras. 2004. Scheduling commercials on broad- based scheduling of Web advertisements. Electronic Commerce Res. cast television. Oper. Res. 52(3) 337–345. 5(1) 75–98. Bollapragada, S., M. R. Bussieck, S. Mallik. 2004. Scheduling com- Raghavan, P., C. D. Tompson. 1987. Randomized rounding: A technique mercial videotapes in broadcast television. Oper. Res. 52(5) for provably good algorithms and algorithmic proofs. Combinatorica 679–689. 7(4) 365–374. Bollapragada, S., H. Cheng, M. Phillips, M. Garbiras, M. Scholes, Rossiter, J. R., P. J. Danaher. 1998. Advanced Media Planning. Kluwer Academic Publishers, Norwell, MA. T. Gibbs, M. Humphreville. 2002. NBC’s optimization systems in- Simon, H. 1982. ADPULS: An advertising model with wearout and pul- crease its revenues and productivity. Interfaces 32(1) 47–60. sation. J. Marketing Res. 19(3) 352–363. Cai, Y. 2007. Electronic gaming in the digital home: Game advertising. Svahn, M. 2005. Future-prooﬁng advergaming: A systematisation for the Parks Associates Industrial Report, Parks Associates, Dallas. media buyer. Proc. Second Australasian Conf. Interactive Enter- Chambers, J. 2005. The sponsored avatar: Examining the present reality tainment, Creativity & Cognition Studios Press, Sydney, Australia, and future possibilities of advertising in digital games. Paper pre- 187–191. sented at DIGRA 2005, Simon Fraser University, Burnaby, British Talluri, K. T., G. van Ryzin. 2005. The Theory and Practice of Revenue Columbia, Canada. Available at http://www.digra.org/dl/db/06278 Management. Springer Verlag, New York. .01541.pdf. Thompson, G. 1981. An optimal control model of advertising pulsations Chickering, D. M., D. Heckerman. 2003. Targeted advertising on the Web and wearout. J. W. Keon, ed. Marketing Measurement and Analysis. with inventory management. Interfaces 33(5) 71–77. The Institute of Management Sciences, Providence, RI. Gensch, D. H. 1973. Advertising Planning: Mathematical Models in Zhang, X. 2006. Mathematical models for the television advertising allo- Advertising Media Planning. Elsevier Scientiﬁc, Amsterdam. cation problem. Internat. J. Oper. Res. 1(3) 302–322. to this article and distributed this copy as a courtesy to the author(s). copyright holds INFORMS Additional information, including rights and permission policies, is available at http://journals.informs.org/.