Skewed Bidding in Pay Per Action Auctions for Online Advertising

By NIKHIL AGARWAL,SUSAN ATHEYAND DAVID YANG

Online search as well as keyword-based contex- is derived through clicks, a small publisher may be tual advertising on third-party publishers is primar- tempted to click on ads on its pages anonymously in ily priced using pay-per-click (PPC): advertisers pay order to inate its payments. only when a consumer clicks on the advertisement. One possible form of advertising lauded as a solu- Slots for advertisements are auctioned, and per-click tion to both heterogeneity in the quality of ad impres- bids are weighted by the probability of a click given sions and click fraud is the Pay Per Action advertis- that the advertisement is displayed (the “click-through ing system (Miguel Helft, 2007). In PPA, advertis- rate”) in addition to other factors. The PPC method ers pay only when consumers complete pre-dened allows the advertising platform (e.g. Google) to bun- actions on their website (Google, 2007). Marrissa dle together otherwise heterogeneous items (impres- Mayer, Google's Vice President of Search Product sions on different positions on a search page, on dif- and User Experience, deemed developing the PPA ad- ferent search phrases sharing common “keywords,” vertising system “the Holy Grail” (Stephan Spencer, and on different publishers) into more homogeneous 2007). The appeal of a PPA system arises from the units, simplifying the advertiser's bidding problem. fact that advertisers pay the platform only when valu- However, PPC pricing has some drawbacks. First, all able events occur, reducing the need for costly mon- clicks are not created equal: clicks on a Paris, France itoring of conversion rates from different sources of hotel website that is displayed on a search for Paris clicks, and providing insurance (as compared to PPC) Hilton may result in lower prot conditional on the to advertisers against variable quality of impressions click. Second, for infrequently searched phrases on or dishonest publishers.1 search engines or small content providers, it is dif- At one level, it may seem that PPA and PPC are cult for the advertiser to accurately estimate conver- just variants of the same system: it should not matter sion rates, increasing the risk and monitoring costs for whether the consumer action that triggers a payment is the advertiser and diminishing their incentives to ad- a click or an action. However, in this paper, we point vertise broadly (indeed, on contextual networks, the out a number of potential problems with PPA pric- advertising platform may not even provide the adver- ing systems that do not arise in PPC systems. First, tiser with sufcient accounting data about where the to maximize the value of the system to advertisers, a advertisements were displayed to allow the advertiser PPA system would allow advertisers to specify more to distinguish sources of clicks, and the publisher mix than one action, since most advertisers sell products may change on an ongoing basis.) Third, the problem of varying value. Second, the probability that an ac- of click fraud is fairly pervasive: when publishers re- tion is recorded can be controlled by an advertiser in ceive a share of advertising revenue, advertisers place more complex ways. These two features create in- a single bid applying to many publishers, and revenue centives for strategic behavior on the part of advertis- ers that undermines the efciency and the objective of

 Agarwal: Harvard University, Department of Eco- risk reallocation of a PPA auction. In particular, ad- nomics, 1875 Cambridge Street, Cambridge, MA 02138, vertisers have the incentive to engage in what we call [email protected]. Athey: Harvard University, “skewed bidding,” and further they have the incentive Department of Economics, 1875 Cambridge Street, Cam- bridge, MA 02138, [email protected]. Yang: Har- vard University, Department of Economics, 1875 Cambridge 1See Benjamin Edelman (2008) for some ways in which Street, Cambridge, MA 02138, [email protected]. PPA systems can be defrauded by dishonest publishers Athey acknowledges support from Microsoft Research, through manipulation of action reporting; that article does where she was a visiting researcher in 2008. not consider bidding incentives. 1 2 PAPERS AND PROCEEDINGS MAY 2009 to combine skewed bidding with strategic manipula- world PPA auction, while simplifying the analysis. tion of the probabilities of different actions through We assume that the ad platform uses a second-price destroyed links and articial stockouts. auction where rms compete in aggregate (estimated) The presence of multiple actions implies that the per-impression bids (if there were more sponsored bids of advertisers on different actions must be aggre- links, a generalized second-price auction in aggregate gated in some way. The most natural way to do this bids could be considered, similar to existing practice for a particular advertiser is to sum over the actions, for search advertising; rst-price auctions have more adding up the product of the probability of an action complex bidding strategies, but incentives to skew and the bid for that action. However, as Susan Athey are similar, as shown in Athey and Levin (2001)). and Jonathan Levin (2001), Christian Ewerhart and Throughout, let k1 denote the identity of the rm with Karsten Fieseler (2003), and Patrick Bajari, Stephanie the highest aggregate bid, and let k2 be the rm with Houghton, and Steven Tadelis (2006) have established the second-highest aggregate bid. After bids are re- in other contexts, auctions with scoring rules like this ceived, the highest bidder is informed of the second- create incentives for “skewed bidding.” If the plat- highest aggregate bid .Bk2 / and the estimated action form's estimates of the relative probability of each ac- rates, and must report per-action bids that each exceed tion differ from the beliefs of the advertiser, the ad- a per-action reserve price r and aggregate to Bk2 : The vertiser has the incentive to “over-bid” on actions that winning bidder pays only for actions that are reported, have been underestimated by the platform. This prob- where the price per action is the per-action bid for that lem is exacerbated by the fact that advertisers have action. In real-world implementations, rms are not control over the reporting of actions–they control their typically informed of their estimated action rates, and own websites and thus the ability of users to com- they bid per-action bids that are aggregated by the ad plete designated actions. An advertiser has the ability platform. However, they can modify their bids in real to prevent actions from occurring or being reported time and in principle can learn the aggregate bid re- and then bid a large amount for the now-impossible quired to obtain a position for their advertisements, action, thus increasing the advertiser's ranking in an which creates incentives similar to our stylized model. auction without a corresponding increase in advertiser Denote the rms 1; :::; I . Assume that each rm i payment. i has only two actions 1 and 2. Let p j denote the The consequence of skewing is inefciency in the true probability of rm i's action j occurring given allocation of sponsored links: bidders whose action an impression. Let V i be the rm's aggregate ex- probabilities have been mis-estimated most severely pected value per impression, which may incorporate by the ad platform will be favored, because those bid- some value from having the ad displayed in addition ders perceive the largest gap between their bid as cal- to the expected values from the actions. Let the ad i i culated by the ad platform and their payment, and thus platform's estimate of p j be denoted e j . In the PPA can afford to place bids that are perceived to be advan- pricing scheme, payments are assessed to a winning tageous by the ad platform. A second consequence is bidder only when an action is reported to the ad plat- that rms that are willing to actively game the system k1 k1 form, so the ad platform receives j p j b j in ex- can outbid those who do not. This leads to alloca- k k pectation, which is different than e 1 b 1 Bk2 ; tive inefciencies. Further, the potential gain from Pj j j D the aggregate bid that the winning bidder was required risk-reallocation is diminished, as advertisers' opti- P mal strategies do not accurately report actions. We to achieve using the ad platform's estimated proba- argue that it is difcult to resolve the inefciencies bilities. We consider Nash equilibrium in a one-shot without losing some or all of the benets of PPA pric- game unless explicitly noted otherwise. ing; in practice, we may expect to see advertising plat- forms restrict PPA systems to a single action. II. Skewed bidding with exogenous action probabilities I. Model In order to implement a PPA pricing scheme, the ad For simplicity, we assume that there is only one platform must estimate action probabilities. Although sponsored link that will be awarded to the highest an ad platform may not reveal its action rate estimates bidder. We use a stylized model of the advertising (search ad platforms currently do not reveal estimated auction that preserves the main incentives of a real- click through rates), rms can track their own past ac- VOL. VOL NO. ISSUE PAY PER ACTION AUCTIONS 3 tion rates, and with some guesses about the ad plat- that the relative valuations of the two objects do not form's algorithm, they can form beliefs about the ad matter for the equilibrium ranking of rms. The platform estimates. Firms may be better informed somewhat surprising part of this result is that the rev- than the ad platform about future action rates. For enue paid to the ad platform can be higher when it example, a rm may know that a holiday next week mis-estimates the probabilities than when it computes will lead consumers to prefer actions in different pro- them correctly. The reason is that the incorrect esti- portions than usual. Or they may have planned a sale mates can effectively bias the auction in favor of the that affects different actions differentially. weaker bidder (lower V i ), which extracts more rev- In this section, we consider a stylized assumption enue from the strong bidder in the event that the strong that rms know their own true action probabilities, bidder continues to win. Also, note that if the ad plat- which differ from the ad platform estimates. With risk form estimates the action rates correctly, the posited neutral rms (the case we consider rst), the assump- strategies are weakly optimal, and the rm with the tion of advertiser knowledge of their own probabili- highest value V i wins and pays the second-highest ties is not important, but in an extension to risk averse value, in expectation; however, a small amount of risk rms, it matters for the calculations, though not the aversion would lead a bidder to prefer more balanced qualitative insights. per-action bids. i j Now consider the case where p1 0 and p1 0 : i i i i D D PROPOSITION 1: Suppose that p1; p2; e1; and e2 both rms cannot achieve action 1, e.g. they stocked are strictly positive; V i r; and label the actions out, while all estimated actions are positive. We need i i i i to introduce maximum per-action bids, denoted b, as such that p1=p2 e1=e2: Then, there exists an N equilibrium in which each rm i places an aggregate without these, bidders have the incentive to place un- bid of ei =pi V i r pi pi ei =ei ; and bounded bids. In this case: 1 1 2 1 2 1 if rm i places the highest aggregate bid and the      PROPOSITION 2: If there is a maximum per-action k2 second-highest aggregate bid is B , it uses per- i i i i k i i i bid b; p 0 0, e 0 > 0; and e 0 > 0 for i 1; 2; then action bids b .B 2 re /=e and b r: The N 1 D 1 2 0 D 1 D 2 1 2 D there is an equilibrium where rm i's aggregate bid is expected payment per impression to the ad platform i i i i k k k k k k k k e V =p be ; and if it places the highest aggregate is .p 1 =e 1 /.V k2 =.p 2 =e 2 /+r.e 1 p 1 =p 1 -e 1 - 2 2 C N 1 1 1 1 1 1 2 1 2 bid and Bk2 is the second-highest aggregate bid; its k2 k2 k2 k2 .e1 p2 =p1 e2 ///. optimal per-action bids are bi b and bi .Bk2 1 D N 2 D ei b/=ei : The expected payment per impression to the Proof: Suppose that the maximum opponent bid is 1 N 2 k2 k2 k2 k2 k1 k1 k2 k2 i ad platform is .e V =p b.e e //p =e . B , and consider bidder i's utility from choosing b2 2 2 C N 1 1 2 2 and bi .Bk2 bi ei /=ei : The bidder's utility from 1 D 2 2 1 i winning is: Proof: Consider bidder i's utility from choosing b1 and bi .Bk2 bi ei /=ei ; which is V i pi .Bk2 2 D 1 1 2 2 V i bi pi bi pi bi ei /=ei : This is increasing in bi , so the maximal 1 1 2 2 1 1 2 1 i i i i i i k i i choice is optimal. The aggregate bid stated in the V b p p e =e B 2 p =e : i D 2 2 1 2 1 1 1 proposition yields 0 expected prots when B k D   B 2 . i i i i If p2 > p1e2=e1 (as assumed) the rm's expected Despite the potential for a large gap between ex i ante expected revenue and payments, competition still utility is maximized by decreasing b2 until the reserve price binds: Then, it remains to determine the optimal results in payments to the ad platform that can be Bi : Expected utility depends on Bi only to the extent larger or smaller than when the ad platform estimates it determines the winner. It is thus weakly dominated correctly. to do anything other than choose the Bi that yields Now consider the case of risk aversion, and for sim- 0 expected prots when Bi Bk2 ; which gives the plicity ignore reserve prices (they will not bind when D total bid in the statement of the proposition. risk aversion is high enough) and assume that a rm's The optimal per-action bids are independent of the value from the advertisement derives entirely from ac- i true per-action valuations. A bidder is advantaged by tions 1 and 2, with per-action values to that rm of v1 i over-estimates, in particular by relatively low true ac- and v2: Further, assume that actions are mutually ex- tion rates for the most over-estimated action. Note clusive. We focus on the case of constant absolute 4 PAPERS AND PROCEEDINGS MAY 2009

x k2 risk aversion, that is, u.x/ e . Fixing B , the riod to make its next-period estimates. Also for sim- expected utility from winningD is: plicity, assume that the rm can manipulate the report- ing of actions by selecting the action rate per impres- sion in each period, with a maximum action rate of i .vi bi / i .vi .Bk2 ei bi /=ei / p1e 1 1 p2e 2 1 1 2 ; one. These assumptions are stylized, but highlight the underlying incentives, incentives that would remain which is maximized at bi .ei =.ei ei // qualitatively in a more realistic model. 1 D 2 1 C 2
vi vi Bk2 =ei .1= / ln.ei pi =.ei pi // : Hold xed the maximum opponent aggregate bid 1 2 C 2 C 1 2 2 1 Bm satisfying r < Bm. The following example illus- Note that if a rm minimizes its risk by bidding   trates how rm 1 can manipulate the auction. Period the same prot margin on both actions, it would 1: Firm 1's ad is displayed and it reports action 1 at a choose bi .ei =.ei ei // vi vi Bk2 =ei : 1 D 2 1 C 2 1 2 C 2 rate of 1 per impression and action 2 at rate 0. Period However, this is not optimal: the rm instead chooses 2: Firm 1 sets b1 b > Bm and b1 r and re-   1 N 2 to take on risk by bidding more for the over-estimated ports action 2 at a rateD of 1 per impressionD and action action by an amount that decreases with risk aversion 1 at rate 0. Period 3+: Firm 1 continues to alternate and increases with the relative over-estimate of between the period 1 and period 2 strategies. In this action 1. Note that if actions are correctly esti- example, the ad platform always estimates an aggre- mated, the risk-free bid becomes optimal. With gate bid of b; but rm never pays more than r per im- N mis-estimates, instead of the break-even bid being pression. The approach can also be modied to avoid Bk2 vi ei ei vi .ei = / ln.ei pi =.ei pi //; as such extreme behavior as destroying a link and faking D 1 1 C 2 2 2 1 2 2 1 it would be with risk-free strategies, the break-even reports of actions. If the rm merely alternates peri- aggregate bid will be higher to reect the increased ods in holding “action 1 sales” and “action 2 sales,” it expected utility from optimal per-action bids. can follow the strategy for skewed bidding from Sec- Qualitatively, risk aversion will mitigate the incen- tion II, exploiting the fact that in each period, it has tives of rms to distort their bids. Risk-aversion also different beliefs than the ad platform.2 introduces some new sources of inefciencies. All It turns out that with endogenous actions there can else equal, more risk averse rms will be disadvan- be a strong incumbency advantage – a rm who has taged in the competition. And, rms will suffer a loss a chance to be displayed has the opportunity to ma- of expected utility from taking on the risk of distort- nipulate its action rates, which affect its future prof- ing their bids, relative to a scenario where they bid its. To simplify the analysis, we assume that to ini- their values for each action, which would be optimal tialize the estimated action rates, each rm at the start if action rates were correctly estimated. We focus on of the game has the opportunity to bid against (only) risk neutral rms for the remainder of this paper, but an aggregate per-impression reserve price R such that this example provides intuition about how the results i r < R < mini V . With this modication of the would generalize. game, the next two results show how market design matters. First, if rms can force action rates to be III. Endogenous action probabilities and zero, a cap on bids is necessary, and the auction can market design result in inefcient allocation and low revenue on an ongoing basis. On the other hand, if the auction de- So far, we have demonstrated that misestimates of sign puts a lower bound on each reported per-action the action rate estimation algorithm can lead rms rates as well as on the total action rate, allocation is to engage in strategic bidding. In this section, efcient in terms of value per impression, and rev- we demonstrate how rms can manipulate the PPA enue is not impacted over the case with exogenous, scheme by failing to report an action or destroying a correctly estimated action rates. However, even in the link on their website (lowering the true probability of latter case, rms must manipulate their reported ac- a given action to 0) while strategically bidding. To begin the analysis, assume for the moment that 2A variant of this method of gaming the system requires the system is updated only periodically, and that a se- the rm to create new actions. As long as the system allows ries of impressions occurs in a given period while bids the rm to introduce actions that have lower true probability and estimates are held xed; in subsequent periods, than the search engine's initial estimates, the rm can place the ad platform uses the action rate from the prior pe- bids that the search engine overvalues systematically. VOL. VOL NO. ISSUE PAY PER ACTION AUCTIONS 5 tions to win the auction, and thus the benets of PPA value; the expected payment per impression to the bidding are undermined. ad platform is V k2 ; which is the second-highest per- impression valuation. PROPOSITION 3: Suppose rms consider a time horizon of two periods in their bidding. Suppose there is a maximum per-action bid b; tie bids are broken in Conditional on winning, the bids and action rates N i favor of the most recent winner, and mini V > 2r: above minimize expected payments. The winning k A subgame perfect equilibrium exists where all rms rm's prots per period are V i e.B 2 re/=.1 e/ t initialize their action rates to be .0; 1/; all rms select .1 e/r; and so the aggregate bid in the proposition an aggregate bid of b in each period a rm that wins is optimal. N i I i when ad platform estimates are e 1; e 0 for So far we have used a naive method of estimating j D k D i k2 action rates. However, Nicole Immorlica et al (2006) . j; k/ .1; 2/; .2; 1/ uses per-action bids b j B 2 f g D designed an algorithm such that, asymptotically, click and bi r while reporting action j at rate 0 and ac- k fraud only minimally changes the advertising fees. tion k atD rate 1; and the expected payment per impres- In Theorem 1 of Mohammad Mahdian and Kerem sion to the ad platform is r. Tomak (2007), the authors extend this approach to de- i velop an “action-based” algorithm designed such that, The condition mini V > 2r guarantees that all rms are willing to pay up to r (reporting action at asymptotically, PPA fraud only minimally affects ad- rate 1 on the action with bid r) in order to give them- vertising fees; this result also applies to our model selves a chance of winning in the subsequent period when the PPA system allows only a single action. and paying r then; a total estimated action rate of less Though the framework in Mahdian and Tomak (2007) than 1 prevents a rm from achieving an aggregate bid only allows for one action per sponsored link, we can of b in the subsequent period, which is necessary to slightly modify their proposal to allow for multiple N win the auction when opponents follow the specied actions. In this section, we show that when bidding strategies. on multiple actions is permitted in the PPA system, Now consider the case where the rms are required the moral hazard stemming from the rm's ability to to report a total action rate equal to 1 in each period destroy the “complete-action- j” link allows rms to (note that it might be more reasonable to require the successfully defraud the ad platform even if it uses rm's action rate to be 1 conditional on a click, or the “fraud-resistant” algorithms described above. to allow more than two actions, but we abstract from Following Mahdian and Tomak (2007), if there are this). In addition, suppose that there is a minimum re- t j periods since the last time action j occured for porting rate for each action, e. This can be interpreted rm i, then let ei = 1 , except when t > t; where j t j j as requiring rms to bid a price for each impression O ei 0: Fix initial values of estimates 0 ei < ei (more generally, it could be tied to clicks) plus a bonus j D D 1 2 and Bk2 < ei b. Let t be the maximum value of payment if a desirable action occurs. 2 N 0 t that satises b=t > Bk2 : Firm i can strictly prot N O PROPOSITION 4: Suppose that rms consider a from manipulating action reports as follows. Period horizon of two periods when bidding. Suppose that 0: Bid Bi bei ; and upon winning, set bi r; i D N 2 1 D mini V > r; there is a minimum reporting rate for bi Bk2 =ei ; report action 1 at rate 1 and pay r per each action, 1=2 > e > 0; a requirement to re- 2 D 2 impression. Periods t 1 to t 1: Bid Bi b and, port actions at a total rate equal to 1 per impression, 0 N upon winning, set bi Dt Bk2but do not notD report and no maximum per-action bid: Then there is a sub- 1 any actions. The rmD stays
in the top position; since game perfect equilibrium with the following proper- b=t > Bk2 . Period t : The rm sets bi r and re- ties: each rm i places aggregate bid Bi V i .1 N O 0 2 ports action 2, paying r. Subsequent periods:D Firm 1 e/=e r.1 2e/=e > V i if a rm places theD highest then repeats the cycle exchanging the roles of the ac- aggregate bid and Bk2 isI the second-highest aggre- tions. The rm only generates an average of r=t per gate bid, and if ad platform estimates are ei 1 e; 0 j D impression. i ek e for . j; k/ .1; 2/; .2; 1/ , it uses per-action A full analysis of equilibrium behavior in this D k2 f g bids of bi .B 2 re/=.1 e/ and bi r, while model is beyond the scope of this paper, but the in- j D t k D reporting action j at the rate e and action k at the tuition from the above analysis can be applied to this rate 1 e; the winner has the highest per-impression variant of action rate estimation as well. 6 PAPERS AND PROCEEDINGS MAY 2009

IV. Remarks Working Paper, Harvard University, 2008. http://kuznets.harvard.edu/~athey/position.pdf. While PPA advertising appears appealing at rst Bajari, Patrick, Stephanie Houghton, and Steve glance, it possess a number of vulnerabilities. First, Tadelis, “Bidding for Incomplete Contracts,” Work- when action rates must be (for exogenous reasons) ing Paper Number W12051, National Bureau of Eco- reported honestly, but rms have better estimates of nomic Research, February, 2006. action probabilities than the ad platform, the rms Edelman, Benjamin, “Securing Online Advertis- have an incentive to skew their bids. Skewed bid- ing: Rustlers and Sheriffs in the New Wild West,” ding allows a rm with a lower valuation to outbid Harvard Business School NOM Working Paper No. one with a higher valuation. Since rm strategies de- 09-039, 2008. pend on both the estimated and true action rates, in Edelman, Benjamin, Michael Ostrovsky, and practice rms would have an incentive to expend re- Michael Schwarz. “Internet Advertising and the Gen- sources to learn them if necessary. Generally, rms eralized Second-Price Auction: Selling Billions of that have more precise estimates of them will see Dollars Worth of Keywords.” American Economic Re- higher prots. In addition, risk aversion in the pres- view. 1 August 2006. ence of mis-estimates reduces the protability of ad- Ewerhart, Christian and Karsten Fieseler. “Pro- vertising. These forces will increase the xed costs curement Auctions and Unit-Price Contracts.” The of participation, and reduce the incentive for advertis- RAND Journal of Economics, Vol. 34, No. 3 (Au- ers to enter PPA advertising auctions, relative to the tumn, 2003), pp. 569-581. ideal with common knowledge of correctly estimated Google. “What is pay-per-action advertising?” action rates. http://adwords.google.com/support/bin/answer.py?answer=61449&topic=11637. Second, because rms can manipulate the action (accessed December 15, 2007). probabilities by destroying links on their websites and Helft, Miguel. “Google Tests an Ad Idea: Pay misreporting actions, rms have the incentive to in- Only for Results.” New York Times. 21 March 2007. ate their rankings through a combination of skewed http://www.nytimes.com/2007/03/21/business/media/21google.html?ref=business&pagewanted=all. bidding and misreporting action rates. Improvements Immorlica, Nicole, Kamal Jain, Mohammad Mah- in market design can increase revenue, but they can- dian, and Kunal Talwar. “Click Fraud Resistant Meth- not restore the benets of PPA pricing, since payments ods for Learning Click-Through Rates.” Microsoft will not depend on the true action rates for the rms. Research. 2006. Although PPA auctions are not currently in use at Mahdian, Mohammad, and Kerem Tomak. “Pay- the major search engines in the U.S. (Google discon- per-action model for online advertising.” Yahoo! Re- tinued its PPA program), they are used in some dis- search. 2007. play advertising networks. This paper suggests that Spencer, Stephan. “Google deems pay-per- they will be more successful when implemented ei- action as `Holy Grail.”' August 22, 2007. CNET ther with some restrictions on how reported action News Blog. http://www.news.com/8301-10784_3- rates and/or bids can vary over time, or when only one 9764601-7.html. action is permitted. In another approach, Google's search advertiser optimization tools can be used to suggest PPC bids based on estimated action rates and rms' stated values per action. These tools help min- imize bidding costs, but since advertisers must honor their PPC bids, the platform is not exposed to the kind of manipulation described in this paper.

V. Bibliography

Athey, Susan and Jonathan Levin. “Information and Competition in U.S. Forest Service Timber Auc- tions.” Journal of Political Economy. 2001. Athey, Susan and Glenn Ellison. “Po- sition Auctions with Consumer Search.”