Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

Yuan Deng 1 Sebastien´ Lahaie 1 Vahab Mirrokni 1 Song Zuo 1

Abstract 1981). A natural idea to circumvent this difficulty in practice Online advertisements are primarily sold via re- is to learn a reserve price from the advertisers’ historical peated auctions with reserve prices. In this paper, bids, a practice known as dynamic reserve pricing. Although we study how to set reserves to boost revenue this approach is intuitive, one must be cautious with this based on the historical bids of strategic buyers, kind of pricing policy since it complicates the long-term while controlling the impact of such a policy on incentives of the advertisers. In particular, an advertiser may the incentive compatibility of the repeated auc- have an incentive to shade her bids in order to lower future tions. Adopting an incentive compatibility metric reserve prices, even if each auction in isolation is truthful which quantifies the incentives to shade bids, we (e.g., second-price). propose a novel class of dynamic reserve pricing In the past decade, there has been a growing body of litera- policies and provide analytical tradeoffs between ture on dynamic reserve pricing with strategic advertisers their revenue performance and bid-shading incen- under different settings. Two typical assumptions made tives. The policies are inspired by the exponen- in the literature are that the advertisers are impatient, so tial mechanism from the literature on differen- that they discount their utilities in the future (Amin et al., tial privacy, but our study uncovers mechanisms 2014; Drutsa, 2018; Golrezaei et al., 2019; Deng et al., with significantly better revenue-incentive trade- 2019; 2020a), and that the market is large enough that each offs than the exponential mechanism in practice. advertiser contributes a negligible fraction of the total rev- We further empirically evaluate the tradeoffs on enue (Liu et al., 2018; Epasto et al., 2018). It has been synthetic data as well as real ad auction data from shown that a constant fraction of revenue loss against the a major ad exchange to verify and support our revenue-optimal benchmark is inevitable if these two as- theoretical findings. sumptions do not hold (Amin et al., 2013). Nonetheless, in practice, many ad slots see competition from only a few large advertisers and it is likely that these advertisers care 1. Introduction about their long-run utilities as much as their instantaneous Online advertising is the practice of placing text, banner, payoffs in each auction (Nedelec et al., 2019). or video ads on search engines, publisher websites, or so- In this paper, we consider an environment in which the ad- cial media for the purpose of raising brand awareness and vertisers are patient and the market might be small with only generating sales to site visitors. This form of advertising a few advertisers. Our objective is to understand the tradeoff represents a major source of revenue for both online pub- between bid-shading incentives and revenue performance lishers and Internet companies; the revenue of this market for different reserve pricing policies. Intuitively, in a single- exceeded $124.6 billion in 2019, witnessing a 15.9% in- buyer second-price auction environment, a reserve pricing crease from 2018 (PricewaterhouseCoopers, 2020). A large policy that does not learn from the advertisers’ historical fraction of online ads are sold via repeated auctions in which bids and always sets a zero reserve would result in zero the advertisers bid in real time. revenue, and induce no bid-shading incentive; on the other Reserve pricing is a key tool used to boost revenue in re- hand, a reserve pricing policy setting the Myersonian (i.e, peated auctions (Ostrovsky & Schwarz, 2011; Paes Leme revenue-optimal) reserve (Myerson, 1981) with respect to et al., 2016). However, according to classical auction theory, the advertisers’ historical bid distribution maximizes rev- setting the reserve in an effective manner requires knowl- enue (provided that the advertisers bid truthfully in the past), edge of the advertisers’ valuation distributions (Myerson, but it induces a significant bid-shading incentive. We consider a multi-stage model in which the auctioneer 1Google Research, New York City, NY, USA. Correspondence to: Yuan Deng . learns a reserve from the bids in the last stage and applies it in the current stage (see Section 2.1). Incentive compatibility Proceedings of the 38 th International Conference on Machine is classically defined as a binary notion: An auction is either Learning, PMLR 139, 2021. Copyright 2021 by the author(s). Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing incentive compatible or not. To achieve a more nuanced We empirically evaluate the revenue-incentive tradeoffs of comparison between different reserve pricing policies, we our class of nonlinear policies on both synthetic data and real adopt an incentive compatibility metric which can be inter- ad auction data from a major ad exchange. The experiments preted as a quantitative index of the advertiser’s incentives verify and support our theoretical findings, and uncover non- to shade its bids (see Section 2.2 and 2.3). The advantage linear policies with significantly better revenue-incentive of this metric is that it is simple to compute via black-box tradeoffs than both linear policies and the exponential mech- simulations, and more importantly, it decomposes into static anism for realistic bid distributions. and dynamic components so that the incentives specifically induced by dynamic reserve prices can be evaluated. Related Work. The recent line of study on revenue- optimal dynamic reserve pricing with strategic advertisers was initiated by Amin et al. (2013) and Medina & Mohri Our Contributions. We first provide analytical revenue- (2014); see den Boer (2015) for a survey. They design incentive tradeoff for linear reserve pricing policies. A no-regret policies in a non-contextual environment with a policy is linear if the reserve is scaled by a factor when single impatient advertiser, and the regret guarantee is later all the advertisers’ bids are scaled by . Linear policies improved by Drutsa (2017; 2018; 2020). Amin et al. (2014) include the policy of setting the Myersonian reserve, and develop a no-regret policy in a contextual setting and Gol- more generally any policy of setting the reserve as a fixed rezaei et al. (2019) enrich the model by incorporating market quantile of the bid distribution. We show that for such noise. Deng et al. (2019; 2020a) generalize these results policies, there is precisely a linear tradeoff between revenue by designing a robust dynamic mechanism which competes and bid-shading incentives: One must sacrifice one unit of favourably against the optimal dynamic mechanism. All the incentive metric in order to gain one unit of revenue. In these results concern impatient advertisers and Amin et al. other words, we obtain the intuitive result that the larger the (2013) show that no learning algorithm can achieve sublin- revenue, the stronger the bid-shading incentive on the part ear revenue loss with a patient advertiser. For a large market, of the advertiser. Our result further implies that although the Liu et al. (2018) design no-regret policies using techniques Myersonian reserve maximizes revenue, it also minimizes from differential privacy, while McSherry & Talwar (2007) the incentive metric among all linear policies in a single- and Epasto et al. (2018) provide general frameworks for de- buyer environment. signing mechanisms that are approximately incentive com- Motivated by our results for linear policies, we propose a patible. For patient buyers and small markets, Nedelec et al. novel class of non-linear policies based on the Box-Cox (2019) investigate the advertiser’s bidding strategies against transformations familiar from statistics (Box & Cox, 1964). different dynamic reserve pricing policies. These policies randomize over a set of quantiles to select the There has been a growing amount of interest in developing reserve, where the weight ascribed to a quantile is propor- metrics that quantify incentive compatibility. The common tional to some function of its revenue performance. There- approach is to evaluate the advertiser’s regret, which is fore, in these policies, bid shading affects not just the quan- her utility difference between best-responding and truthful tiles but also their relative weights. We provide a closed- reporting (Parkes et al., 2001; Duetting et al., 2015; Balcan form characterization of the revenue-incentive tradeoffs for et al., 2019). However, computing a regret-based metric is our class of non-linear policies, demonstrating that they can often a complex optimization task of solving for the best result in significantly better revenue-incentive tradeoffs than response, and moreover, it is unclear how to generalize linear policies. regret to repeated auctions. In this work, we instead adopt an Our analysis reveals the possibility of inducing bid-raising incentive compatibility metric recently introduced in Deng incentives via a non-linear policy’s mixture weights, which et al. (2020b), which generalizes the idea of the marginal can counteract the bid-shading incentives. However, there incentive to deviate (Erdil & Klemperer, 2010). The metric is no dominant non-linear policy and designing a non-linear has been adopted to develop variants of generalized second- policy to induce bid-raising incentives is an empirical ques- price auction using deep learning (Zhang et al., 2020). tion that must take into account the actual valuation distri- butions at hand. Our class of non-linear policies is inspired 2. Preliminaries by the exponential mechanism from the literature on dif- ferential privacy, which is a fundamental technique used to We consider an environment with multiple buyers partici- mitigate incentives to deviate from truthfulness when the pating in repeated auctions with a single seller (e.g., an ad market is large (McSherry & Talwar, 2007; Epasto et al., exchange). The repeated auctions sell a sequence of queries, 2018). Our results shed light on how to adapt and generalize each granting the winning buyer a chance to display ads on the exponential mechanism to achieve a significantly better the website. These queries arrive in an online manner and revenue-incentive tradeoffs in a small market. they must be sold once they arrive. For convenience, we Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing assume that there is exactly one query per round, so the t-th One can hardly hope for a good revenue performance guar- query arrives at round t. antee for dynamic reserve pricing if the environments are quite different across both stages. For the purpose of theoret- For ease of presentation, we take the perspective of one ical analysis, we further assume that F = F and G = G fixed buyer. We assume that the buyer’s private valuation t t for all t, so that they are identical across queries. We will ex- vt V = R 0 for round t is drawn independently from 2 amine how the revenue-incentive tradeoffs generalize across a continuous distribution F over V with density function t stages in our experiments for realistic bid distributions that f . Upon the arrival of the t-th query, the buyer first learns t arise practice. her private valuation vt drawn from Ft and then submits her bid b V to the seller. After receiving the bids from t 2 all buyers, the seller decides how to allocate and charge 2.2. Incentive Compatibility Metric for the ad slot according to an allocation function xt and a An auction x, p is incentive compatible (IC) if reporting h i payment function pt. As we take the perspective of one fixed truthfully is always an optimal strategy for the buyer re- buyer, xt and pt subsume the mechanism (in particular, the gardless of her private valuation and others’ bids. Formally, reserve price applied) and other buyers’ bidding behavior. for all v V , v arg max v x(b) p(b). Myerson’s 2 2 b · In line with the literature, we assume that the buyer’s utility lemma (Myerson, 1981) pins down the relationship between is quasi-linear such that her utility under bid bt at round t the allocation and payment rule for any truthful auction: (1) with private valuation vt is ut(bt; vt)=vt xt(bt) pt(bt). The allocation rule x is non-decreasing; (2) The payment · v For convenience, let uˆt(vt)=vt xt(vt) pt(vt) be the rule satisfies p(v)=v x(v) x(z)dz. In particular, the · · 0 buyer’s utility under truthful bidding. latter condition implies that for each valuation, the deriva- R tive of the buyer’s utility equals the allocation probability, We consider a widely adopted auction format: the second- dˆu(v) price auction with lazy personalized reserve (Ostrovsky & i.e., dv = x(v) for all v. Schwarz, 2011; Paes Leme et al., 2016), in which the buyer Inspired by Myerson’s lemma, Deng et al. (2020b) proposes wins the ad slot if she bids higher than the highest bid from a metric to quantify incentive compatibility. others b and the reserve rt for round t. If a buyer wins, she t Theorem 2.1 (Individual Stage-IC Metric (Deng et al., pays max(bt,rt); otherwise, she pays nothing. We further x,p The stage-IC metric SICh i for an agent with assume that at each round, the highest bid among other buy- 2020b)). valuation distribution F in an auction x, p is defined as ers is drawn independently from a continuous distribution h i G over V with density function g . In practice, many other t t E uˆ (1 + ↵)v E uˆ (1 ↵)v auction formats are used in online advertising (e.g., first- lim . ↵ 0 2↵ E [v x(v)] price auction, generalized second-price auction). In this ! ⇥ ·⇤ · ⇥ ⇤ work, we are interested in the incentives across auctions, rather than within each individual auction, so we consider The expectation here is taken over the valuation distribu- second-price auctions (which are truthful) to focus on the tion F . Intuitively, the numerator of SIC is the difference dynamic incentives. between the agent’s expected utility when her valuations are perturbed multiplicatively up versus down by ↵, while 2.1. A Multi-stage Model for Reserve Pricing the denominator is 2↵ times the expected welfare. This metric can be viewed as a test of the Myerson’s condition In a setting where the seller has no prior knowledge about dˆu(v) dv = x(v) for all v, except that only the mean value of the buyers’ valuations, a canonical way to apply reserve the test is reported. pricing is to learn a reserve to apply in the future from the buyers’ historical bids (Kanoria & Nazerzadeh, 2014; den Deng et al. (2020b) demonstrates that the metric takes the Boer, 2015). To analyze the tradeoff between incentives and form of an index between 0 and 1 for reasonable auctions: revenue under different reserve pricing policies, we propose it is non-negative if the buyer’s utility under truthful bidding a multi-stage model where each stage consists of sufficiently is non-decreasing in her true valuation, and it is at most 1 if many queries (rounds), and the reserve is set to be the same overbidding is a weakly dominated strategy. Additionally, within each stage. In particular, the metric is 1 for incentive-compatible auctions while it is 0 for first-price auctions. More importantly, the metric captures the marginal benefit of uniform bidding, in which • The reserve is set to be 0 for all queries in stage 1; the buyer always bids b = v for a constant factor ap- proaching to 1. In particular, when the metric is below 1, • For each stage t>1, the seller applies a reserve pricing the buyer has incentive to shade her bids. The smaller the policy to compute a reserve rt from the buyer’s bids metric, the stronger the bid-shading incentive. On the other from stage t 1; The seller sets the reserve to r for hand, when the metric is above 1, the buyer has the incen- t queries in stage t. tive to raise her bids. The larger the metric, the stronger Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing the bid-raising incentive. This observation is important in Since our auction format is IC for stage 1 in isolation2, x ,p practice, as uniform bidding has been shown to be a typical SICh 1 1i is always 1 (Deng et al., 2020b). Therefore, we (and sometimes optimal) strategy for buyers to participate can rewrite the metric as follows: in ad auctions in complex environments (Aggarwal et al., 2019; Balseiro & Gur, 2019; Deng et al., 2021a). E [ˆu2(1 + ↵, v)] E [ˆu2(1 ↵, v)] DIC =1+lim . ↵ 0 2↵ E [v x1(v)] ! · · 2.3. Dynamic Incentive Compatibility Metric The second term is usually negative under dynamic reserve Intuitively, a mechanism is dynamic-IC if for any stage t, pricing since dynamic reserve pricing induces the buyer to reporting truthfully at stage t is an optimal strategy for the shade her bids. buyer regardless of her private valuation and others’ bids, assuming the buyer always bids truthfully in the future (Mir- 2.4. Reserve Pricing Policy rokni et al., 2020; Deng et al., 2020b; 2021b).1 In general, a reserve pricing policy maps a distribution The IC metric can be generalized to multi-stage auctions by of the buyer’s bids to a distribution of reserves. Given taking the future stages into account to verify the definition a distribution F over V with finite and non-zero density of dynamic-IC. In our multi-stage model, when measuring function f, there is a bijection between values in V and dynamic-IC for stage t, only stage t +1is relevant since the quantiles  [0, 1] such that  = F (v). For convenience, bids at stage t only affect the reserve price at stage t+1; and 2 denote the quantile function that maps a quantile to a value the reserve prices for all future stages are the same provided by q such that q()=F 1(). As a result, it is with- that the buyer always reports truthfully from stage t +1 out loss of generality to represent a reserve pricing policy onwards, under the dynamic-IC definition. M :(V ) ([0, 1]) as a function that maps a distribu- ! We therefore focus, without loss of generality, on a multi- tion of bids to a distribution of quantiles, where (V ) is the stage model with two stages. Note that with the reserve set of distributions over V . learned from stage 1 fixed, stage 2 would be IC, and there- Denote by M the distribution of quantiles returned by M fore, the buyer has no incentive to misreport in stage 2. F under truthful bidding when the buyer’s valuation distribu- However, the buyer may still have incentive to misreport in tion is F . Notice that the dynamic-IC metric only depends stage 1 to lower the reserve in stage 2. on the bid distributions generated by the uniform bidding To define the metric for dynamic-IC in stage 1, let x1(b) strategy with factor (1 ↵) and (1 + ↵). We denote by M F and p1(b) be the expected allocation and payment when the the distribution of quantiles returned by the policy when buyer bids b in stage 1. Moreover, let x2(,b) and p2(,b) the buyer adopts the uniform bidding strategy with factor , represent the expected allocation and payment when she and the reserve price will be set to q() with a quantile · submits bids uniformly scaled by a factor from her private  selected by the policy. A policy is called linear if it is valuations in stage 1, i.e., v, and bids b in stage 2. In addi- independent of : MF = MF for all . In a linear policy, tion, let uˆ (,v)=v x (,v)+p (,v). Therefore, the the distribution over quantiles remains unchanged, while the 2 · 2 2 buyer’s expected cumulative utility under truthful bidding reserve price is scaled accordingly by . is E[ˆu1(v)] + E[ˆu2(1,v)]. We are now ready to define the dynamic-IC metric adapted from (Deng et al., 2020b). 3. Linear Policy Definition 2.2 (Dynamic-IC Metric). The dynamic-IC met- ric DIC for a buyer with valuation distribution F in a dy- We begin by analyzing linear policies. Observe that given namic auction in stage 1 is defined as a quantile distribution MF , the revenue contribution from the buyer under truthful bidding is REV = REVreserve + REVcompt, where REVreserve is the contribution from reserve x ,p E [ˆu2(1 + ↵, v)] E [ˆu2(1 ↵, v)] SICh 1 1i +lim . ↵ 0 (when the highest bid from others is below the reserve) and ! 2↵ E [v x1(v)] · · REVcompt is the contribution from competition (when the highest bid from others is above the reserve): The dynamic-IC metric (DIC) additionally takes the utility in stage 2 into account: the additional term shares the same REVreserve = E (1 )q() G q() , denominator as the stage-IC metric, while the numerator  MF · ⇠ computes the utility difference in stage 2 under truthful h v i 1 bidding when the buyer’s valuations are perturbed multi- REVcompt = E z g(z)dzf(v)dv .  MF · plicatively up versus down by ↵ in stage 1. ⇠ "Z0 Zq() # 1By a backward induction argument, this is equivalent to as- 2More precisely, the auction for stage 1 is IC if the buyer were suming the buyer bids in a way to optimize her utility in the future. to participate only in stage 1 but not stage 2. Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

Here, (1 ) G q() is the probability that the buyer’s the buyer has in stage 1. Theorem 3.1 also shows that the · valuation is above q() and the highest bid from others is metric is fully determined by REV , and as a result, reserve below q(). In other words, it is the probability that the for any two linear policies MF and MF0 that achieves the reserve q() is effective under truthful bidding. same REVreserve, their dynamic-IC metrics must be the same. Moreover, as REV = REV when there is only one Let uˆ (,v; ) be the buyer’s utility in stage 2 when (i) in reserve 2 buyer, we immediately have the following corollary for the stage 1, she adopts the uniform bidding strategy with factor single-buyer environment: and (ii) in stage 2, she truthfully bids her private value v while the reserve is the -th quantile from stage 1, i.e., Corollary 3.2. For any linear policy under the single-buyer q(). Note that uˆ2(,v; )=0if v q(); and two-stage model, the dynamic-IC metric of a buyer with ·  · valuation distribution F satisfies DIC =1 REV/W . uˆ (,v; )= v q() G q() 2 · · · v Note that the policy that applies the Myersonian re- + (v z) g(z)dz, serve (Myerson, 1981) estimated from the bid distribution q() · Z · is also linear. Hence, the Myersonian reserve maximizes if v> q(), where the first term is the utility when the revenue but also minimizes the dynamic-IC metric (i.e., cre- · highest bid from others is below q() and the second ates the strongest bid shading incentives) among all linear · term is the utility when the highest bid from others is above policies in the single-buyer environment. q(). We further define · 4. Non-linear Policy Based on the Box-Cox util(,)= E [ˆu2(,v; )] v F ⇠ Transformation as the buyer’s expected utility. We now proceed to provide In the previous section, we analyzed the revenue-incentive analytical revenue-incentive tradeoffs for linear policies. For tradeoff for linear reserve pricing policies. We now turn convenience, let W = E [v x1(v)] for the rest of the paper. to reserve pricing policies that are not linear. A canoni- · Theorem 3.1. For any linear policy, the dynamic-IC metric cal non-linear policy is inspired by the exponential mecha- of each buyer with valuation distribution F and competing nism (McSherry & Talwar, 2007; Epasto et al., 2018), which bid distribution G satisfies is widely used to establish differential privacy and leads to approximate-IC mechanisms when the market is large. For- DIC =1 REVreserve/W. mally, there is a weight function w :[0, 1] R 0 that ! assigns each quantile  a weight w(), and the probability E[ˆu2(1+↵,v)] E[ˆu2(1 ↵,v)] Proof. We focus on lim↵ 0 in of selecting each quantile  is proportional to exp(✓ w()), ! 2↵ · DIC since other terms are constant. For a linear policy, where ✓ R 0 is an adjustable parameter. For example, 2 this can be written as w() can be the revenue in stage 1 when the reserve is q(). Another canonical mechanism is the power mech- E [ˆu2(1 + ↵, v)] E [ˆu2(1 ↵, v)] lim anism (Epasto et al., 2020), in which the probability of ↵ 0 ✓ ! 2↵ selecting each quantile  is proportional to w() . 1+↵ 1 ↵ E M [util(, 1+↵)] E M [util(, 1 ↵)] =lim ⇠ F ⇠ F We consider a broad class of non-linear policies that contains ↵ 0 2↵ ! both the exponential mechanism and the power mechanism. E MF [util(, 1+↵)] E MF [util(, 1 ↵)] =lim ⇠ ⇠ It is inspired by the power transformation proposed by Box ↵ 0 ! 2↵ & Cox (1964), known as the Box–Cox transformation. The

@ E MF [util(,)] single-parameter Box-Cox transformation function is: = ⇠ , @ =1 (y 1)/, R 0 ; ⌧(y)= 2 \{ } ln y, =0. where the second equation follows that the policy is linear. ⇢ All that remains to show is that Definition 4.1 (Non-linear Policy based on the Box-Cox @util(,) Transformation). A non-linear policy M is specified by = (1 ) q() G q() @ =1 · · a tuple ,✓, w , where R and ✓ R 0. The h i 2 2 probability of selecting each quantile  is proportional to for any  (details deferred to Appendix A.1). exp(✓ ⌧ (w())). · Theorem 3.1 reveals a linear tradeoff between the dynamic- Observe that when =0, the policy becomes the power IC metric and the revenue contribution from the reserve: mechanism, and when =1, it becomes the exponential The larger REVreserve is, the stronger bid-shading incentive mechanism. Recall that the dynamic-IC metric concerns the Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing bid distribution generated by the uniform bidding strategy partial derivative for factor (1 ↵) and (1 + ↵). For convenience, denote by @ E MF [util(,)] w(,) the weight function when the buyer adopts the uni- ⇠ EV =1 = R reserve, form bidding strategy with factor in stage 1. We assume @ that w(,) is continuous and differentiable with respect to where appears in the utility of the second stage. In con- , but we do not require it to be continuous with respect to . trast, under non-linear policies, a buyer can further change This enables different weight function formats for different the distribution of the quantiles, and such an effect is cap- . The following theorem characterizes the dynamic-IC tured by the partial derivative metric for our class of non-linear policies. @ E M [util(, 1)] Theorem 4.2. For any non-linear policy M = ,✓, w , ⇠ F = ✓ , h i =1 the dynamic-IC metric of a buyer with valuation distribution @ · F and competing bid distribution G satisfies where appears in the distribution MF of the quantiles. DIC =1 (REV ✓ )/W, A properly designed non-linear policy, resulting in a positive reserve · covariance term , will create an incentive for the buyer to where raise her bids, offsetting the bid-shading incentive captured by REV . Note that util(, 1) is non-increasing as  @w(,) reserve 1 increases, and therefore, the covariance term is likely to be =cov MF util(, 1), =1 w(, 1) . ⇠ @ · @w(,) 1  positive if w(, 1) is also non-increasing @ =1 · for most quantiles. E[ˆu2(1+↵,v)] E[ˆu2(1 ↵,v)] Proof. We focus on lim↵ 0 in ! 2↵ w(,) w(,)= DIC since other terms are constant. For a non-linear policy, In particular, when is linear in , i.e., w(, 1), we immediately have: this can be written as · Corollary 4.3. For any non-linear policy M = ,✓, w E [ˆu2(1 + ↵, v)] E [ˆu2(1 ↵, v)] h i lim with w(,) linear in , the dynamic-IC metric of a buyer ↵ 0 ! 2↵ with valuation distribution F and competing bid distribution 1+↵ 1 ↵ E M [util(, 1+↵)] E M [util(, 1 ↵)] G satisfies DIC =1 (REVreserve ✓ )/W where =lim ⇠ F ⇠ F · ↵ 0 ! 2↵ =cov MF util(, 1), w(, 1) . @ E [util(,)] ⇠  MF = ⇠ ⇥ ⇤ @ =1 Particularly, for a power mechanism in which =0, the dynamic-IC metric satisfies DIC =1 REVreserve/W . @ E [util(, 1)] @ E MF [util(,)]  MF = ⇠ + ⇠ @ =1 @ =1 Corollary 4.3 implies that the revenue-incentive tradeoff in @ E [util(, 1)] a power mechanism with a weight function linear in is  MF = REVreserve + ⇠ , exactly the same as in a linear policy. In the single-buyer @ =1 environment, all the revenue comes from reserve pricing, where the last equation would follow the proof of Theo- i.e., REV = REVreserve, and is hence linear in . Similarly, @ [util(,1)] E M the welfare performance is also linear in . Therefore, if ⇠ F rem 3.1. The equation @ = ✓ then =1 · one uses a convex combination between the revenue and concludes the proof (details deferred to Appendix A.2). the welfare performance as the weight function, the tradeoff given in Corollary 4.3 still applies. Theorem 4.2 shows that compared with linear policies (The- orem 3.1), the dynamic-IC metric for non-linear policies 5. Experiments has an additional covariance term of ✓ under the quantile · In this section we report on results from simulations of the distribution MF . Note that Theorem 4.2 generalizes Theo- rem 3.1 since a linear mechanism can be implemented with non-linear policies. When simulating a policy, we use REV (i.e., the buyer’s expected payment) as the weight function. a weight function independent of so that @w(,) =0for @ By varying the parameter ✓ (in Definition 4.1), we obtain dif- any  and . ferent combinations of (REVreserve/W, DIC) which yields Intuitively, under linear policies, a buyer who adopts the a tradeoff curve that we plot. Note that the first coordinate uniform bidding strategy can only change the scale of the is (normalized) revenue under truthful bidding, but when reserve while the distribution of quantiles remains the same, DIC < 1 one would expect buyers to shade their bids. The which gives the buyer the incentive to shade her bids to plots are best interpreted as showing dominance relation- lower the reserve. Such an incentive is captured by the ships: for two mechanisms that yield the same revenue Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

1.2 0.76 0.66 1.0

0.64 0.74 0.8 0.62

0.6 DIC DIC 0.72 Revenue λ 0.60 λ −1 −1 0.4 −0.75 −0.75 −0.5 −0.5 −0.25 0.58 −0.25 0 0 0.70 0.25 0.25 0.2 Lognormal 0.5 0.56 0.5 Shifted Lognormal 0.75 0.75 1 1

0.0 0.2 0.4 0.6 0.8 1.0 0.24 0.26 0.28 0.30 0.34 0.36 0.38 0.40 0.42 Quantile Reserve Revenue / Welfare Reserve Revenue / Welfare

Figure 1: Revenue curve Figure 2: Log-normal Figure 3: Shifted log-normal

Figure: Synthetic data experiment: revenue-incentive tradeoff without competing bids under non-linear policies with different ; the dashed line corresponds to the line of DIC =1 REV /W . reserve under truthful bidding, the one with higher DIC should in the opposite. Moreover, the tradeoff curve for =0coin- practice be closer to this predicted revenue. cides with the diagonal line of DIC =1 REV /W in reserve agreement with the theory. Note that the curves do not start We first present synthetic-data experiments to verify the from 0 revenue (and 1 dynamic-IC metric) since all the non- theory, and next report on experiments over real bid data linear policies return a uniform distribution over quantiles from a major ad exchange to understand which particular when ✓ =0, resulting in a positive revenue. Moreover, the policies work well over realistic bid distributions in practice. rightmost points in Figures 2 and 3 in fact correspond to the Myerson’s auctions. 5.1. Synthetic Data We defer the experiments with competing bids and with We first provide results using synthetic data. We consider a that has a large absolute value to Appendix B. The trends for standard log-normal distribution F1 with density function both distributions are similar to the setting without compet- f (x)= 1 exp( ln x ) and a shifted log-normal dis- 1 xp2⇡ · 2 ing bids, but the curve for =0deviates from the diagonal tribution F with density function f (x)=f (x 1) for 2 2 1 line since the revenue is no longer linear in the bid-shading x>1 and f (x)=0for x (0, 1]. We use a log-normal 2 2 factor . Moreover, for a with a large absolute value, the distribution because it is commonly used to model bid and dynamic-IC metric could be above 1, resulting in an incen- value distributions in practice (Lahaie & Pennock, 2007; tive for the buyer to raise her bids beyond its true value. Ostrovsky & Schwarz, 2011; Thompson & Leyton-Brown, Therefore, one must be cautious with the choice of to 2013; Golrezaei et al., 2019). balance the bid-raising and bid-shading incentives. We consider a single-buyer environment without compet- ing bids. We compute the 100-quantiles, excluding the 5.2. Ad Auction Data 100-th quantile (i.e., the maximum), and their revenues as In this section we evaluate the revenue-incentive tradeoffs the weights. Figure 1 shows the revenue curve for both within our family of mechanisms over real data from the distributions. Observe that the revenue for the log-normal auction logs of a major display ad exchange. Our goal here distribution is increasing as the quantile increases for most is not to evaluate the exact auction mechanism and pricing of the support, while the revenue for the shifted log-normal policies used by the exchange, but rather to validate our distribution is decreasing over most of the support. Accord- theory over realistic bid distributions.3 The purpose of this ing to the covariance term in Corollary 4.3, this suggests that a non-linear policy with negative should have a better 3In particular, the exchange from which the data is drawn runs tradeoff than one with positive for the log-normal distri- a first-price auction, but we use the bids to simulate second-price bution, but have a worse tradeoff for the shifted log-normal auctions with reserves, consistent with our model so far. One could first try to back out values from bids, but we view this as distribution. Figures 2 and 3 show the tradeoffs and confirm an orthogonal problem which would not substantially alter the these theoretical findings. In addition, we see that for the experimental conclusions. To empirically evaluate the tradeoff, log-normal distribution, the smaller , the better the trade- what matters is the shape of the distribution; in ad auctions, bid and off; and the pattern for the shifted log-normal distribution is value distributions are likely to have a similar shape since one is just a rescaling of the other under the common strategy of uniform Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● 0.9 ● ●● 0.9 ● ● ●● ● ● ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ●● ●● 0.8 ● ●● ● 0.8 ● ● ●● ● ● ● ● ● ● ● ●●● DIC ●● ●● DIC ● ● ●●● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● 0.7 ● 0.7 ● ● ● ● ● ● ● λ ● λ −1 ● ● −1 ● −0.5 ● ● ● −0.5 ● ● 0 ● ● ● ● 0 ● 0.5 ● 0.5 ● 0.6 1 ● 0.6 1 ●

0.10 0.15 0.20 0.25 0.10 0.15 0.20 0.25 Reserve Revenue / Welfare Reserve Revenue / Welfare

Figure 4: Tradeoff on training data. Figure 5: Tradeoff on testing data.

Figure: Real data experiment: revenue-incentive tradeoff without competing bids under non-linear policies with different ; the dashed line corresponds to the line of DIC =1 REV /W . reserve evaluation is twofold. In Section 5.1, we saw that depending worst tradeoffs. Following the intuition from Corollary 4.3, on the bid distributions, higher could lead to better or this is because revenue increases with the reserve (and is worse tradeoffs. Therefore, we are interested in uncover- negatively correlated with utility) for a large portion of the ing the pattern that holds under realistic bid distributions. support of the empirical bid distribution. By using negative Next, we are interested in confirming that the tradeoffs can values like = 1, the resulting bid-raising incentives can be measured to a useful level of accuracy in practice (i.e., counteract the bid-shading incentives, as the DIC metric the confidence intervals are narrow enough to distinguish initially stays quite flat even as the revenue performance mechanisms using different values of ). increases, as seen in Figure 4. This figure also confirms that the power mechanism ( =0) follows a specific linear Our dataset consists of auction records of bids placed on tradeoff curve, validating the theory. In Figure 5 we evaluate queries from a single publisher over two consecutive days. the tradeoffs on the testing data; the trends are extremely We focus on a single bidder and only retain the auctions that similar, but the power mechanism no longer tracks the pre- involve this bidder, yielding over 100K auction records in dicted line because the distributions are not exactly identical total. The first day is used as the training set to compute day-to-day. the bid quantiles and their associated revenue performance as reserves under a second-price auction. Specifically, we Plots of the tradeoff curves when there are competing bids compute the (5, 10, 25, 50, 75, 90, 95)-th bid quantiles and are deferred to Appendix B. Again, we find that lower their counterfactual revenue. The non-linear policies from yield better tradeoffs. The trends are similar to the single- Section 4 are then implemented based on this information, bidder case, but because the revenue-weighting is now non- and the revenue-incentive tradeoffs are evaluated on both linear the power mechanism deviates from the diagonal line. the training (same-day) data and the testing (next-day) data. Comparing the range of the dynamic-IC metric to the single- The policies take mixtures of the bid quantiles along with the bidder case, we find that the dynamic-IC metric is higher, option of no reserve price. Note that our model has assumed because the competing bids mute the bid-shading incentives that bids are i.i.d. across days; this does not hold exactly in of the dynamic reserve prices. practice, so our empirical results will clarify the extent to which the relationships characterized in Theorem 4.2 and 6. Conclusions Corollary 4.3 extend to future time periods. This work provides analytical characterizations of the Results assuming just a single bidder (i.e., discarding all revenue-incentive tradeoffs of dynamic reserve pricing poli- competing bidders in the simulation) are presented in Fig- cies, and introduces a novel class of policies that randomize ure 4 and Figure 5. (In all plots, we present 95% confidence over bid quantiles as reserves. intervals.) We observe that lower achieve better revenue- incentive tradeoffs, in the sense that at every revenue level, For linear policies, we proved that bid-shading incentives are lower dominate higher in terms of the DIC metric; in directly proportional to the revenue extracted by a reserve. particular, the exponential mechanism ( =1) exhibits the For non-linear policies based on the Box-Cox transforma- tion, we showed that an additional term is added which bidding which motivates our analysis. Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing depends on the covariance between quantile weight and In Advances in Neural Information Processing Systems, quantile utility. As a result, by choosing an appropriate pp. 8654–8664, 2019. weighting scheme, it is possible to counteract bid-shading incentives with bid-raising incentives. The mechanism ex- Deng, Y., Lahaie, S., and Mirrokni, V. Robust pricing in hibiting the best tradeoff depends on the bidder’s valuation dynamic mechanism design. In International Conference distribution, and in particular, on the extent to which rev- on Machine Learning, pp. 2494–2503. PMLR, 2020a. enue increases with quantile over the distribution’s support. Deng, Y., Lahaie, S., Mirrokni, V., and Zuo, S. A data- For realistic distributions like the lognormal, and on an em- driven metric of incentive compatibility. In Proceedings pirical distribution of bids from a major ad exchange, the of The Web Conference 2020, pp. 1796–1806, 2020b. exponential mechanism in fact accentuates incentives to shade bids and significantly better tradeoffs can be achieved Deng, Y., Mao, J., Mirrokni, V., and Zuo, S. Towards effi- with log-concave weighting schemes. cient auctions in an auto-bidding world. In Proceedings of the Web Conference 2021, pp. 3965–3973, 2021a. In future work, it would be interesting to consider more general reserve policies, for example, a combination be- Deng, Y., Mirrokni, V., and Zuo, S. Non-clairvoyant dy- tween quantile-based reserve sand constant reserves, such namic mechanism design with budget constraints and as, the 20-th quantile of the bid distribution plus a con- beyond. In Proceedings of the 2021 ACM Conference on stant. Other reserve pricing strategies, such as anonymous Economics and Computation, EC ’21. ACM, 2021b. reserves, are also interesting to consider. Different strategies may come with different revenue-incentive tradeoffs, and Drutsa, A. Horizon-independent optimal pricing in repeated it is an intriguing question to characterize the landscape of auctions with truthful and strategic buyers. In Proceed- revenue-incentive tradeoffs. ings of the 26th International Conference on World Wide Web, pp. 33–42. International World Wide Web Confer- ences Steering Committee, 2017. References Aggarwal, G., Badanidiyuru, A., and Mehta, A. Autobid- Drutsa, A. Weakly consistent optimal pricing algorithms ding with constraints. In International Conference on in repeated posted-price auctions with strategic buyer. Web and Internet Economics, pp. 17–30. Springer, 2019. In International Conference on Machine Learning, pp. 1318–1327, 2018. Amin, K., Rostamizadeh, A., and Syed, U. Learning prices for repeated auctions with strategic buyers. In Advances Drutsa, A. Reserve pricing in repeated second-price auctions in Neural Information Processing Systems, pp. 1169– with strategic bidders. In International Conference on 1177, 2013. Machine Learning, 2020. Amin, K., Rostamizadeh, A., and Syed, U. Repeated contex- Duetting, P., Fischer, F., Jirapinyo, P., Lai, J. K., Lubin, B., tual auctions with strategic buyers. In Advances in Neural and Parkes, D. C. Payment rules through discriminant- Information Processing Systems, pp. 622–630, 2014. based classifiers. ACM Transactions on Economics and Computation (TEAC), 3(1):5, 2015. Balcan, M.-F., Sandholm, T., and Vitercik, E. Estimating approximate incentive compatibility. In Proceedings of Epasto, A., Mahdian, M., Mirrokni, V., and Zuo, S. the 2019 ACM Conference on Economics and Computa- Incentive-aware learning for large markets. In Proceed- tion, EC ’19, pp. 867–867, New York, NY, USA, 2019. ings of the 2018 World Wide Web Conference, pp. 1369– ACM. 1378, 2018. Balseiro, S. R. and Gur, Y. Learning in repeated auctions Epasto, A., Mahdian, M., Mirrokni, V., and Zampetakis, with budgets: Regret minimization and equilibrium. Man- E. Optimal approximation-smoothness tradeoffs for soft- agement Science, 65(9):3952–3968, 2019. max functions. In Advances in Neural Information Pro- cessing Systems Box, G. E. and Cox, D. R. An analysis of transforma- , 2020. tions. Journal of the Royal Statistical Society: Series B Erdil, A. and Klemperer, P. A new payment rule for core- (Methodological), 26(2):211–243, 1964. selecting package auctions. Journal of the European den Boer, A. V. Dynamic pricing and learning: historical Economic Association, 8(2-3):537–547, 2010. origins, current research, and new directions. Surveys Golrezaei, N., Javanmard, A., and Mirrokni, V. Dynamic in operations research and management science, 20(1): incentive-aware learning: Robust pricing in contextual 1–18, 2015. auctions. In Advances in Neural Information Processing Deng, Y., Lahaie, S., and Mirrokni, V. A robust non- Systems, pp. 9756–9766, 2019. clairvoyant dynamic mechanism for contextual auctions. Revenue-Incentive Tradeoffs in Dynamic Reserve Pricing

Kanoria, Y. and Nazerzadeh, H. Dynamic reserve prices for Ostrovsky, M. and Schwarz, M. Reserve prices in internet repeated auctions: Learning from bids. In Web and In- advertising auctions: A field experiment. In Proceedings ternet Economics: 10th International Conference, WINE of the 12th ACM conference on Electronic commerce, pp. 2014, Beijing, China, December 14-17, 2014, Proceed- 59–60, 2011. ings, volume 8877, pp. 232. Springer, 2014. Paes Leme, R., Pal, M., and Vassilvitskii, S. A field guide Lahaie, S. and Pennock, D. M. Revenue analysis of a family to personalized reserve prices. In Proceedings of the of ranking rules for keyword auctions. In Proceedings 25th international conference on world wide web, pp. of the 8th ACM conference on Electronic commerce, pp. 1093–1102, 2016. 50–56, 2007. Parkes, D. C., Kalagnanam, J., and Eso, M. Achieving Liu, J., Huang, Z., and Wang, X. Learning optimal reserve budget-balance with vickrey-based payment schemes in price against non-myopic bidders. In Advances in Neural exchanges. In Proceedings of the 17th International Joint Information Processing Systems, pp. 2038–2048, 2018. Conference on Artificial Intelligence, IJCAI’01, pp. 1161– 1168, San Francisco, CA, USA, 2001. Morgan Kaufmann McSherry, F. and Talwar, K. Mechanism design via dif- Publishers Inc. ferential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pp. 94– PricewaterhouseCoopers. IAB Internet adver- 103. IEEE, 2007. tising revenue report: Full year 2019 results Medina, A. M. and Mohri, M. Learning theory and algo- & Q1 2020 revenues. https://www.iab. rithms for revenue optimization in second price auctions com/wp-content/uploads/2020/05/ with reserve. In Proceedings of the 31st International FY19-IAB-Internet-Ad-Revenue-Report_ Conference on Machine Learning, pp. 262–270, 2014. Final.pdf, 2020. Accessed: 2021-01-20. Mirrokni, V., Paes Leme, R., Tang, P., and Zuo, S. Non- Thompson, D. R. and Leyton-Brown, K. Revenue opti- clairvoyant dynamic mechanism design. Econometrica, mization in the generalized second-price auction. In Pro- 88(5):1939–1963, 2020. ceedings of the fourteenth ACM conference on Electronic commerce, pp. 837–852, 2013. Myerson, R. B. Optimal auction design. Mathematics of Operations Research, 6(1):58–73, 1981. Zhang, Z., Liu, X., Zheng, Z., Zhang, C., Xu, M., Pan, J., Yu, C., Wu, F., Xu, J., and Gai, K. Optimizing multiple per- Nedelec, T., El Karoui, N., and Perchet, V. Learning to formance metrics with deep gsp auctions for e-commerce bid in revenue-maximizing auctions. In International advertising. arXiv preprint arXiv:2012.02930, 2020. Conference on Machine Learning, pp. 4781–4789, 2019.