Equilibrium Selection in Multi-Player Games with Auction Applications∗

Paul Milgrom Joshua Mollner

May 23, 2014

Abstract We introduce two new equilibrium refinements for finite normal form games, both of which incorporate the intuitive idea that a costless deviation by one player is more likely than a costly deviation by the same or another player. These refinements lead to new restrictions in games with three or more players and select interesting equilibria in some well-known auction games.

1 Introduction

This paper introduces two new refinements of Nash equilibrium. Both incorporate the restriction that a deviation by one player to a best response is more likely than a deviation by the same or another player to a strategy that is not a best response. One incorporates the additional restriction that the same deviation is also more likely than any combination of deviations by multiple players. Neither restriction is implied by Myerson’s (1978) proper equilibrium in games with three or more players, and these restrictions can be consequential. In two well known auction games – the menu auction and the generalized second-price auction – some of the main conclusions that were originally derived using idiosyncratic equilibrium selection criteria can instead be derived using our new refinements. To motivate our equilibrium refinements, consider the three-player game in Figure 1. Each player has two strategies. The “Geo” player picks the payoff matrix – East or West. Geo’s payoff is always zero, regardless of what

∗For helpful comments, we thank Gabriel Carroll, Piotr Dworczak, Michael Ostrovsky, Bernhard von Stengel, Alex Wolitzky, and participants in seminars at Stanford.

1 Figure 1: a three player game†

[West] Left Right [East] Left Right Up 1, 1 1, 1 Up 1, 0 1, 1 Down 0, 1 0, 0 Down 0, 0 0, 0

†Row’s payoffs are listed first. Column’s payoffs are listed second. Geo’s payoffs are constant and suppressed.

anybody does, so we omit its payoff from the matrices. For Row, the strategy Up strictly dominates Down: the former always pays one and the latter always pays zero. Column’s decision is the one of interest: its best choice de- pends on what it believes the other two players will do. The strategy profiles (Up, Left, West) and (Up, Right, West) are both proper equilibria (Myerson, 1978) of this game.1 The first of these equilibria, however, is harder to rationalize if Column believes that the other players are rational, because its choice of Left is optimal only if Row is more likely to deviate to Down, which would be an obvious and costly mistake, than Geo is to deviate to East, which would be no mistake at all. Our proposed equilibrium refinements both eliminate (Up, Left, West) by insisting that Column must regard Geo’s costless deviation as being more likely than Row’s costly one. One, which we call “extended proper equilibrium,” achieves this by refining proper equilibrium (Myerson, 1978). Although extended proper equilibrium is defined to be independent of the utility scales of the payoff matrix, it is informally motivated by the idea that there must exist some scaling of the players’ payoffs such that costly mistakes by any one player are much less likely than less costly mistakes by the same player or by another player. It is the italicized condition that represents the extra restriction in this definition compared with proper equilibrium. Our other proposed solution, “test-set equilibrium,” incorporates a second idea not implied by any of the tremble-based refinements, namely, that a deviation by just one player to a best response is more likely than any collection of deviations by two or more players. Accordingly, we define each player’s test set to consist of the equilibrium strategy profiles of the other players and the related profiles in which exactly one of them deviates to a different best response. A test-set equilibrium is a Nash equilibrium pro-

1Formal proofs of all claims about the game in Figure 1 are in Appendix A.1.

2 file for which each player’s strategy is weakly undominated in the decision problem in which the possible opposing strategy profiles are just those in the test set. Both new concepts lead to interesting refinements in two celebrated auction games. Because we wish to highlight the relative ease of applying the test-set refinement, this introduction will focus on applications of that concept. We begin with the generalized second price auction. This auction and its variants have been used by Google, Yahoo!, and others to sell advertising on search pages. Higher positions on a search page tend to be clicked more frequently by searchers, and so are more valuable to advertisers. Payments by bidders depend on the next-highest bid. Edelman et al. (2007) (hereafter EOS) have modeled the generalized second-price auction mechanism and applied a particular equilibrium selection. Varian (2007) introduced a very similar model and equilibrium selection. Adopting the EOS model, let us denote the numbers of clicks associated with the I positions by κ1 > ··· > κI > 0. There are N bidders, and bidder n values clicks at vn each. A bid, bn, specifies a maximum price per-click. The highest bid wins the first position; the second highest wins the second position; and so on. The winner of position n pays a per-click (n+1) (n+1) price of b , so its total payment is κnb . With I = 1, this reduces to a second-price auction, and the search ad auction design has become known as the “generalized second price auction.” This auction game has many Nash equilibria with a variety of outcomes, including ones involving inefficient decisions, or low total payments, or both. Abstracting from ties, let us number the bidders so that the equilibrium bids satisfy b1 > ··· > bN . It is easy to see that for each n ≤ N, bidder n would not strictly prefer to have the price and allocation of any bidder m > n in a lower position on the page, for the bidder could obtain that outcome by reducing its bid to bm − for some > 0. EOS propose further limiting the equilibria to ones in which bidder n also does not envy the price and allocation of bidder n − 1, who has a higher position on the page. They label an equilibrium with this property as “locally envy-free.” Formally, an equilibrium is locally envy-free if for all n ∈ {2,...,N}, κn(vn − bn+1) ≥ κn−1(vn − bn). For generic bidder values, the generalized second price auction has the property that while a single deviation to a different best response can affect the prices paid, it nevertheless leaves the allocation of positions unchanged. It is this property that allows easy application of the test-set criterion. For these generic values, test-set equilibrium leads to the requirement

3 that any pure equilibrium is locally envy-free. For suppose that b is a pure test-set equilibrium of the generalized second price auction game. Consider bidder n’s reasoning when it thinks about raising its bid slightly to bn + for some ∈ (0, bn−1 − bn). “I should test the robustness of my planned bid against some test set, and I will use the set consisting of strategy profiles in which only one other player deviates and that player is still playing a best response. For all strategy profiles in the test set, if I continue to play my equilibrium strategy, I will still win position n. If I were to raise my bid slightly to bn + and that were to change the outcome (as can happen on the test set), that must mean that bidder n − 1 has bid less than I expected and that I will win position n−1 at a per-click price of about bn. That change is profitable for me if κn(vn − bn+1) < κn−1(vn − bn − ), and if that inequality holds, then the new bid weakly dominates the equilibrium bid on the test set.” Since test-set equilibrium allows no such dominance, it follows that for all players n and all > 0 sufficiently small, κn(vn−bn+1) ≥ κn−1(vn−bn−). Hence, every pure test-set equilibrium of the generalized second price auction is locally envy-free. A similar conclusion can be reached if extended proper equilibrium is applied to a discretized version of the game. Our second application is the menu auction game of Bernheim and Whin- ston (1986), which has been applied to study both competition among lob- byists and bidding in combinatorial auctions. In the menu auction game, an auctioneer chooses a decision x from some finite set X based on offers it receives. For the auction application, the decision is a resource allocation and the offers are bids. For the political application, the decision may concern legislation or a public good and the offers may be bribes or less direct forms of compensation, the payment of which will be conditional on the decision. We denote the utility that bidder/lobbyist n receives from decisions by vn : X → R. Similarly, we denote the utility of the auctioneer by v0 : X → R. Each bidder/lobbyist n makes a vector of bids bn : X → R+ and the ∗ P auctioneer then chooses x ∈ arg maxx∈X v0(x) + n∈N bn(x) . Bidder n’s ∗ ∗ payoff is vn(x ) − bn(x ). The menu auction game can have very many Nash equilibria, which may involve inefficient decisions, or low total payments, or both. To refine away what they regarded as the implausible Nash equilibria, Bernheim and Whin- ston consider two approaches. The first restricts attention to equilibria in which bidders adopt what they call “truthful” strategies, such that bidders become indifferent among decisions for which they have made positive bids. Their second approach introduces a new solution concept – coalition-proof equilibrium, which selects only Nash equilibria that are immune to certain coalitional deviations. In any Nash equilibrium of the menu auction satisfy-

4 ing either of these conditions, the resulting payoff vector is in the core and, among core payoff vectors, it is Pareto optimal for the bidders. To apply test-set equilibrium to this game with a continuum of bids, we introduce a convenient tie-breaking assumption, namely, that between any two outcomes with the same payoff for a bidder or for the auctioneer, the bidder or auctioneer prefers the outcome with the higher total payoff. This assumption ensures that the auctioneer’s choice is unique and that no bidder is ever indifferent among bids that lead to different outcomes. This implies that, in parallel to generic discrete versions of the auction, the continuous auction is “non-bossy,” meaning that if a bidder changes its bid in a way that affects the outcome, then it is never indifferent about that change. The “non-bossiness” property allows easy application of the test-set criterion to this mechanism. Test-set equilibrium leads to the requirement that any pure equilibrium leads to payoff vectors that are in the core, although not necessarily bidder- optimal among such outcomes as in Bernheim and Whinston (1986). Sup- pose that b is a pure test-set equilibrium of the menu auction game, that the equilibrium decision is x∗, and that each player n’s payoff is denoted by ∗ ∗ ∗ πn = vn(x ) − bn(x ). Let x 6= x and consider bidder n’s reasoning as it thinks about whether to raise its bid for outcome x from bn(x) to bn(x) + , for some > 0, leaving all its other bids unchanged. “I should test the robustness of my planned bid against some test set, and I will use the set consisting of strategy profiles in which only one other player deviates and that player is still playing a best response. For all strategy profiles in the test set, if I continue to play my equilibrium strategy, the decision will still be x and my payoff will still be πn. If I were to raise my bid for x and that were to change the outcome (as can happen on the test set), that must mean that the new outcome would be x and my new payoff would be vn(x) − bn(x) − . That change is profitable for me if and only if vn(x) − bn(x) − > πn, and if that inequality holds, then the new bid weakly dominates the equilibrium bid on the test set.” Since test-set equilibrium allows no such dominance, it follows that for all bidders n and decisions x, there is an implied lower bound for all losing bids: bn(x) ≥ vn(x) − πn. Using these bounds, we show that the equilibrium payoffs are in the core.2 If we instead apply extended proper equilibrium to a discretized version of the game, the analysis is subtler and the conclusion is weaker: starting

2A nearly identical argument, in which extended proper equilibrium implies the same lower bounds for bids and leads to a bidder-optimal core allocation, works as well for the “core-selecting auction” models of Day and Milgrom (2013).

5 from an extended proper equilibrium bid profile, no pair of bidders can change their bids in a way that would lead to significantly higher total value for the coalition consisting of them and the auctioneer. In the body of the paper, we introduce the two concepts. We begin by defining extended proper equilibrium and showing that, for finite games, such equilibria always exist and are always proper equilibria. For games with two players, the set of extended proper equilibria coincides with the set of proper equilibria. We also characterize extended proper equilibrium in terms of hierarchies of beliefs. Next, we define test-set equilibrium. Compared to perfect and proper equilibrium, the definition of test-set equilibrium excludes the requirements that all strategies could potentially be played by mistake and that some (more costly) mistakes are much less likely than others. Given this construction, it is perhaps unsurprising that test-set equilibrium does not imply perfect or proper equilibrium. Yet, unlike perfect and proper equilibrium, test-set equilibrium does require that simultaneous mistakes by multiple players must be less likely than any single, costless mistake. As the preceding analyses demonstrate, this restriction can be particularly useful for equilibrium selection in games with three or more players. However, as we show by example, there are games for which no test-set equilibrium exists. Like test-set equilibrium, extended proper equilibrium also satisfies a kind of “no-domination” condition, which we will describe in detail in the relevant section below. In test-set equilibrium, there are just two impor- tant categories of deviations: those that are relatively likely (the test set) and those that are not. In contrast, extended proper equilibrium involves additional categories, particularly categories involving multiple deviations. The likelihood of such deviations in extended proper equilibrium is ambigu- ous: they may be as likely as some single deviations to best responses or as unlikely as some costly mistakes. This ambiguity implies that, in applications, extended proper equilibrium can be harder to use and sometimes less powerful than test-set equilibrium. Finally, we return to the applications, using discrete bid spaces to apply the extended proper equilibrium concept and obtaining “approximation” results for that case. For the generalized second-price auction game, we find that every extended proper equilibrium and every test-set equilibrium is (approximately) locally envy-free. For the menu auction game, extended proper equilibrium allocation has a certain “pairwise (approximate) efficiency” property. As observed above, a test-set equilibrium allocation of the same game has a stronger property: it is an (approximate) core allocation.

6 2 Extended Proper Equilibrium

Extended proper equilibrium is an equilibrium refinement for finite normal form games. Its formal definition and proof of existence are both similar to those for proper equilibrium in (Myerson, 1978). Informally, both concepts restrict players’ “big” (more costly) mistakes to be less likely than their own “small” (less costly) mistakes, but unlike the original proper equilibrium concept, extended proper equilibrium also restricts the relative likelihood of mistakes by different players. It requires that there exists some scaling of the payoff matrix for which each player’s bigger mistakes are less likely than other players’ smaller mistakes. Adapting Myerson’s method, we show that for any finite game and any scaling, such an equilibrium exists. We show that the set of extended proper equilibria is a subset of the set of proper equilibria, and prove that the two sets coincide in games with precisely two players. We then characterize the set of extended proper equilibria in terms of hierarchies of beliefs.

2.1 Deﬁnition

A finite N-person game in normal form is a 2N-tuple G = (S1,...,SN ; u1, . . . , uN ), where for each agent n ∈ {1,...,N}, Sn is a nonempty, finite N set of pure strategies, and un : ×n=1 Sn → R is a utility function. A mixed N strategy profile is denoted σ = (σ1, . . . , σN ) ∈ ×n=1 ∆(Sn). We embed Sn N in ∆(Sn) and extend the utility functions un to the domain ×n=1 ∆(Sn) in the usual way. Given a mixed strategy profile σ, we define Ln(sn|σ) as the expected loss for player n from playing sn instead of a best response when others play σ−n: Ln(sn|σ) = max un(ˆsn, σ−n) − un(sn, σ−n). sˆn∈Sn

This quantity is zero for strategies that are best responses to σ−n and pos- N itive otherwise. Given a vector α ∈ R++, we deﬁne an (α, )-extended proper equilibrium to be a combination of totally mixed strategies in which the probability of strategies with higher scaled losses is at most times the probability of strategies with lower scaled losses.

N Deﬁnition 1. Let α ∈ R++ and > 0. An (α, )-extended proper equi- N 0 librium is a proﬁle of totally mixed strategies (σ1, . . . , σN ) ∈ ×n=1 ∆ (Sn) such that

αnLn(sn|σ) > αmLm(sm|σ) ⇒ σn(sn) ≤ · σm(sm)

7 for all n, m, all sn ∈ Sn, and all sm ∈ Sm. We also deﬁne an extended proper equilibrium to be a strategy proﬁle that, for some scaling vector α, is a limit of (α, )-extended proper equilibria, as → 0.

N Deﬁnition 2. A strategy proﬁle σ = (σ1, . . . , σN ) ∈ ×n=1 ∆(Sn) is an N extended proper equilibrium if and only if there exist α ∈ R++ and sequences ∞ t ∞ {t}t=0 and {σ }t=0 such that:

(i) each t > 0 and limt→∞ t = 0,

t (ii) each σ is an (α, t)-extended proper equilibrium, and

t (iii) limt→∞ σ = σ.

The set of extended proper equilibria is unaffected by affine transforma- tions of the utility functions of the players. The following theorem states that extended proper equilibria exist in every finite normal form game. Its proof and all other proofs are deferred to Appendix A.

Theorem 1. Every ﬁnite normal form game has at least one extended proper equilibrium.

2.2 Lexicographic Characterization In this section, we employ the framework of Blume et al. (1991) to characterize extended proper equilibrium in terms of hierarchies of the players’ beliefs. Toward that end, let ρ = (p1, . . . , pK ) be a sequence of probability N measures on ×n=1 Sn. Blume et al. (1991) refer to such a sequence as a lexicographic probability system (LPS). 1 K N An LPS ρ = (p , . . . , p ) on ×n=1 Sn gives rise to a partial order on [ the elements of × Sj, which is defined as follows. Given some j∈J J⊆{1,...,N} 1 K 3 J ⊆ {1,...,N}, let ρJ = (pJ , . . . , pJ ) be the marginal on ×j∈J Sj of ρ. If sJ ∈ ×j∈J Sj and sI ∈ ×i∈I Si, then we say that sJ >ρ sI , read as “sJ is k infinitely more likely than sI according to the LPS ρ,” if min{k : pJ (sJ ) > 3This notation differs slightly from that used by Blume et al. (1991). First, while they designate players with superscripts and levels of an LPS with subscripts, we do the n reverse. In addition, what we denote by ρ−n, they would instead denote by ρ . Our purposes differ slightly from theirs, and our notation is a bit more natural for what is presented here.

8 k 0} < min{k : pI (sI ) > 0}. We write sJ ≥ρ sI to mean it is not the case that sI >ρ sJ . We deﬁne the best response set of player n to ρ−n as follows, where we 4 use the symbol ≥L to represent the lexicographic ordering.

  K   X k BRn(ρ−n) = sn ∈ Sn :  p−n(s−n)un(sn, s−n) ≥L  s ∈S  −n −n k=1  K   X k 0 0   p−n(s−n)un(sn, s−n) ∀sn ∈ Sn s ∈S  −n −n k=1 

Deﬁnition 3. A pair (ρ, σ) is a lexicographic Nash equilibrium if

1 (i) for all n ∈ {1,...,N}, pn(sn) > 0 implies sn ∈ BRn(ρ−n), and (ii) p1 = σ.

We next introduce two properties that an LPS may possess. The first, “respects within-person preferences,” is equivalent to what Blume et al. (1991) define as “respects preferences.” We use different terminology in order to accentuate the distinction between this property and “respects within- and-across-person preferences,” also defined below.

1 K N Deﬁnition 4. An LPS ρ = (p , . . . , p ) on ×n=1 Sn respects within-person 0 0 preferences if for all n ∈ {1,...N} and all sn, sn ∈ Sn with sn ≥ρ sn, it is the case that  K  K X k X k 0  p−n(s−n)un(sn, s−n) ≥L  p−n(s−n)un(sn, s−n) . s ∈S s ∈S −n −n k=1 −n −n k=1

1 K N Deﬁnition 5. An LPS ρ = (p , . . . , p ) on ×n=1 Sn respects within-and- N across-person preferences if there exists some α ∈ R++ such that for all ∗ ∗ n, m ∈ {1,...N}, sn ∈ BRn(ρ−n), sm ∈ BRm(ρ−m), and all sn ∈ Sn,

4 K k k Formally, for a, b ∈ R , a ≥L b if and only if whenever b > a , there exists an l < k such that al > bl.

9 sm ∈ Sm with sn ≥ρ sm, it is the case that

 K X k ∗ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s ∈S −n −n k=1  K X k ∗ ≤L αm p−m[um(sm, s−m) − um(sm, s−m)] . s ∈S −m −m k=1 It is easy to see that Definition 5 is a strengthening of Definition 4, which is a strengthening of Condition (i) of Definition 3. Finally, we state two additional definitions, which are generalizations for LPSs of what it means for a probability measure to have full support or to be a product measure.

1 K N Definition 6. An LPS ρ = (p , . . . , p ) on ×n=1 Sn has full support if for N k each s ∈ ×n=1 Sn, p (s) > 0 for some k ∈ {1,...,K}. N Definition 7. An LPS ρ on ×n=1 Sn satisfies strong independence if there is an equivalent F-valued probability measure that is a product measure, where F is some non-Archimedean ordered field that is a proper extension 5,6 of R. For comparison, we incorporate as Propositions 2-4 below the characterizations of Blume et al. (1991). To these, we add the characterization of extended proper equilibrium. These characterizations are useful both because they make some proofs easier (LPSs are easier to work with than sequences of trembles) and because, compared to sequences of trembles, LPSs correspond more closely to intuitive statements about some actions being “infinitely more likely” than others.

5 1 An ordered field is non-Archimedean if it contains an element > 0 such that < n ∀n ∈ N. Such an element is called an infinitesimal. 6 1 K N An LPS ρ = (p , . . . , p ) on ×n=1 Sn and an F-valued probability measurep ˆ on N ×n=1 Sn, are said to be equivalent if there exists a vector of positive infinitesimals = K−1 N (1, . . . , K−1) ∈ F such thatp ˆ(s) = ρ(s) ∀s ∈ ×n=1 Sn, where ρ is used to denote the probability measure

1 2 3 ρ = (1 − 1)p + 1 (1 − 2)p + 2 (1 − 3)p + h h K−1 K i iii 3 ··· + K−2 (1 − K−1)p + K−1p ··· .

10 N Proposition 2. The strategy proﬁle σ ∈ ×n=1 ∆(Sn) is a Nash equilibrium 1 K N if and only if there exists some LPS ρ = (p , . . . , p ) on ×n=1 Sn that satis- ﬁes strong independence for which (ρ, σ) is a lexicographic Nash equilibrium.

N Proposition 3. The strategy proﬁle σ ∈ ×n=1 ∆(Sn) is a perfect equilib- 1 K N rium if and only if there exists some LPS ρ = (p , . . . , p ) on ×n=1 Sn that satisﬁes strong independence and has full support for which (ρ, σ) is a lexicographic Nash equilibrium.

N Proposition 4. The strategy proﬁle σ ∈ ×n=1 ∆(Sn) is a proper equilib- 1 K N rium if and only if there exists some LPS ρ = (p , . . . , p ) on ×n=1 Sn that satisﬁes strong independence, has full support, and respects within-person preferences for which (ρ, σ) is a lexicographic Nash equilibrium.

N Proposition 5. The strategy proﬁle σ ∈ ×n=1 ∆(Sn) is an extended proper 1 K N equilibrium if and only if there exists some LPS ρ = (p , . . . , p ) on ×n=1 Sn that satisﬁes strong independence, has full support, and respects within-and- across-person preferences for which (ρ, σ) is a lexicographic Nash equilibrium.

2.3 Relationship to Proper Equilibrium It is apparent from Propositions 4 and 5 that every extended proper equilibrium is a proper equilibrium. The two sets can be diﬀerent in games with three or more players, but not for two-player games.

Theorem 6. In two player games, the sets of proper equilibria and extended proper equilibria coincide.

2.4 Undominatedness Property Next, we establish that extended proper equilibrium strategies must be undominated in a special sense against several speciﬁc sets of strategy proﬁles, which represent plausible individual and joint deviations by others. This attention to possible joint deviations is one factor that distinguishes this solution concept from one we will introduce later: test-set equilibrium.

Proposition 7. If σ is an extended proper equilibrium, then there is no pair of players n and m and strategies σˆn ∈ ∆(Sn) and sˆm ∈ BRm(σ−m) such that (i) un(ˆσn, s−n) > un(σn, s−n), for some s−n ∈ A, and (ii) un(ˆσn, s−n) ≥

11 7 un(σn, s−n), for all s−n ∈ A ∪ B, where A and B are deﬁned as follows. sm =s ˆm and A = (sm, s−nm) σi(si) > 0 ∀i∈ / {n, m} sm ∈ BRm(σ−m) \ sˆm and B = (sm, s−nm) si ∈ BRi(σ−i) ∀i∈ / {n, m}

The theorem states that strategies played with positive probability in any extended proper equilibrium must be undominated in a certain sense. To intuitively convey the idea behind the statement and the proof, observe that given any strategy profile σ, any player n, any player m, and any strategys ˆm ∈ BRm(σ−m), the elements of ×i6=n Si can be partitioned into four sets: A and B as defined in the statement of Proposition 7, as well as C and D, defined below. sm ∈/ BRm(σ−m) or C = (sm, s−nm) ∃i∈ / {n, m} s.t. si ∈/ BRi(σ−i)    sm =s ˆm and  D = (sm, s−nm) si ∈ BRi(σ−i) ∀i∈ / {n, m} and  ∃i∈ / {n, m} s.t. σi(si) = 0 

The set A contains the strategy profiles in which m plays some best response sˆm to σ and all other opponents of n play strategies that are played with positive probability under σ. The set B contains the strategy profiles in which m plays some other best response and all other opponents of n also play strategies that are best responses to σ. The set C contains the strategy profiles in which one opponent of n plays a strategy that is not a best response to σ. The set D contains the strategy profiles in which m plays sˆm and some other opponent of n plays a strategy that is not played with positive probability under σ. The theorem says that if there is some strategyσ ˆn that weakly dominates σn against A and performs weakly better than σn against all elements of B, then σ cannot be extended proper. To show this, we argue that if σ is an extended proper equilibrium, then the elements of A must be “infinitely more likely” than the elements of C and D (although the elements of B cannot be ranked against the elements of the other sets.). Consequently, if σn were dominated in this way, then player n would wish to deviate toσ ˆn.

7 Both below and throughout the paper we use s−nm to indicate a strategy proﬁle for the players {1,...,N}\{n, m}.

12 3 Test-Set Equilibrium

The second equilibrium refinement we propose is test-set equilibrium. In- formally, a Nash equilibrium is a test-set equilibrium if no player uses a strategy that is weakly dominated against his test set, which consists of the equilibrium strategy profiles of the other players and the related profiles in which exactly one player deviates to a different best response. For games with three or more players, test-set equilibrium neither implies nor is implied by any trembles-based refinement. We also show by example that there are games for which no test-set equilibrium exists.

3.1 Definition Unlike extended proper equilibrium, test-set equilibrium is well-defined for both finite and infinite games.

Deﬁnition 8. A strategy proﬁle σ = (σ1, . . . , σN ) is a test-set equilibrium if and only if it is a Nash equilibrium and for all n, σn is not weakly dominated by anyσ ˆn ∈ ∆(Sn) against [ Tn(σ) = {(ˆsm, σ−nm) :s ˆm ∈ BRm(σ−m)} . m6=n

We refer to the set Tn(σ) in the above deﬁnition as the test set of player n. A test-set equilibrium is a Nash equilibrium in which all players use strategies that are robust to a certain type of “testing.” Intuitively, each player “tests” his strategy against the set of opponents’ strategy proﬁles in which at most one opponent deviates, and that deviation is to a best response. The following result states that the undominatedness property possessed by extended proper equilibrium, as stated in Proposition 7, is in fact implied by the test-set condition.

Proposition 8. If σ is a test-set equilibrium, then there is no pair of players n and m and strategies σˆn ∈ ∆(Sn) and sˆm ∈ BRm(σ−m) such that (i) un(ˆσn, s−n) > un(σn, s−n), for some s−n ∈ A, and (ii) un(ˆσn, s−n) ≥

13 8 un(σn, s−n), for all s−n ∈ A ∪ B, where A and B are deﬁned as follows. sm =s ˆm and A = (sm, s−nm) σi(si) > 0 ∀i∈ / {n, m} sm ∈ BRm(σ−m) \ sˆm and B = (sm, s−nm) si ∈ BRi(σ−i) ∀i∈ / {n, m}

3.2 Existence Test-set equilibria are not guaranteed to exist in general finite normal-form games. Figure 2 presents an example of a game with no test-set equilibrium.9 (Up, Left, West) is the unique Nash equilibrium of this game. How- ever, it is not a test-set equilibrium because East weakly dominates West against Geo’s test set: {(Up, Left), (Up, Center), (Up, Right), (Up, Down)}. This game therefore contains no test-set equilibrium, although the unique Nash equilibrium is an extended proper equilibrium. To check that, let α = (1, 1, 1) and note that for all > 0 sufficiently small, σ as defined √ below is an (α, )-extended proper equilibrium.

1 3 σ = , row 1 + 3 1 + 3 1 6 2 σ = , , col 1 + 2 + 6 1 + 2 + 6 1 + 2 + 6 1 σ = , geo 1 + 1 +

With these trembles, the joint deviation from equilibrium to (Down, Right) is more likely than the single deviation to Center. Test-set equilibrium, on the other hand, would require the reverse. Moreover, this example illustrates that it may not always be possible to ﬁnd trembles and an associated equilibrium in which deviations are ordered so that all single deviations to best responses are more likely than any joint deviation to best responses. While test-set equilibria are not guaranteed to exist in all ﬁnite normal form games, there are certain classes of games in which their existence can

8 Both below and throughout the paper we use s−nm to indicate a strategy proﬁle for the players {1,...,N}\{n, m}. 9We thank Michael Ostrovsky and Markus Baldauf for helpful suggestions that led to the construction of this example.

14 Figure 2: a three player game†

[West] Left Center Right [East] Left Center Right Up 0, 0, 0 0, 0, 0 0, 0, 0 Up 0, 0, 0 0, −1, 1 0, 1, 0 Down 0, 0, 0 −1, 1, 0 1, −1, 0 Down −1, 0, 0 −1, −1, 0 1, −1, −1

†Row’s payoffs are listed first. Column’s payoffs are listed second. Geo’s payoffs are listed third. be guaranteed. The following result states three conditions, each of which is sufficient to guarantee the existence of a test-set equilibrium.

Proposition 9. A ﬁnite normal form game has at least one test-set equilibrium if it also satisﬁes at least one of the following conditions:

(i) it is a two-player game,

(ii) it is a potential game,10 or

(iii) it is a three-player game in which each player has two pure strategies.

3.3 Relationship to Other Equilibrium Concepts The previous example shows that not every extended proper equilibrium is a test-set equilibrium. In the game in Figure 3, it is easily checked that (Up, Left) is a test-set equilibrium of this game but is not a trembling- hand perfect equilibrium.11 Thus, in games with three or more players, the strongest tremble-based reﬁnement does not imply test-set equilibrium, and test-set equilibrium does not imply the weakest tremble-based reﬁnement: the concepts are logically independent for such games. For two-player games, the situation is more complex: proper equilibrium implies test-set equilibrium, although trembling hand perfect equilibrium does not.12 10Monderer and Shapley (1996). 11It is not a trembling-hand perfect equilibrium because Row uses Up, a weakly dominated strategy. However, since Row’s test-set contains only Left, Up is not weakly dominated against his test-set. 12The game depicted in Figure 2 of Myerson (1978) can be used as a counterexample for illustrating the latter fact. As he shows, (α2, β2) is a perfect equilibrium of this game. However, it is not a test-set equilibrium.

15 Figure 3: a two player game†

Left Right Up 1, 1 0, 0 Down 1, 1 1, 0

†Row’s payoffs are listed first. Column’s payoffs are listed second.

Theorem 10. In any ﬁnite, two-player game, every proper equilibrium is a test-set equilibrium.

4 Generalized Second Price Auction

Edelman et al. (2007) study the Generalized Second Price (GSP) auction, and focus on equilibria of the auction that possess a property they call local envy-freeness. Varian (2007) studies the same auction, and makes the same equilibrium selection (he refers to these equilibria as symmetric Nash equilibria). This section introduces the GSP auction, and studies the properties of both extended proper equilibria and test-set equilibria of this auction game. We find that every pure test-set and extended proper equilibrium is locally envy-free, but our refinements also imply additional restrictions, so the existence of such equilibria in pure strategies can depend on the full specification of the game.

4.1 Environment There are I ad positions and N bidders. The click-rate of the ith position is κi > 0, and positions are labeled so that their click rates are in descending order. The value per click of bidder n is vn > 0. Advertiser n’s payoﬀ from being in position i is κivn minus his payments to the search engine. In the event that I < N, we deﬁne κI+1 = ··· = κN = 0 for notational convenience. The N bidders simultaneously submit bids. A bid is a scalar bn ∈ R+. We denote the set of feasible bids for bidder n by Bn. Two special cases are of interest. First is the case when the bid spaces 0 are continuous, each taking the form B = R+. This is the case studied by Edelman et al. (2007). Second is the case when the bid spaces are discrete, γ each taking the form B = γZ ∩ [0, Γ]. Here, γ is a parameter controlling

16 the fineness of the discretization, and Γ controls the upper bound of the bid space. To ensure that Γ is not restrictively small, we assume Γ ≥ γ+maxn vn. This bound ensures that no bidder would wish to bid above Γ even if it were possible and additionally, all bids constructed in the proofs of the following results are contained in the relevant bid spaces. While we will apply test- set equilibrium with both continuous and discretized bid spaces, extended proper equilibrium, which is defined only for finite games, will be applied only with discretized bid spaces. (i) Suppose that the bidders submit bids b = (b1, . . . , bN ). Let b and G(i) denote the ith highest bid and the identities of all advertisers bidding at that level, respectively. For notational convenience, we define b(N+1) = 0. The mechanism functions as follows. First, bidders are sorted in order of their bids, where ties are broken uniformly at random. After ties are broken in a particular way, let g(i) denote the ith highest bidder. For every i ∈ {1, 2,..., min{I,N}}, the mechanism allocates position i to bidder g(i) (i+1) (i+1) at a per-click price of b , for a total payment of κib . If N > I, then bidders g(I + 1), . . . , g(N) win nothing and pay nothing. We denote the expected payoff to bidder n under the bid profile b by (in(b)+1) πn(b) = E κin(b) vn − b , where the expectation is taken over the random variable in(b), the position won by bidder n. A GSP auction then N N becomes a game G = [{Bn}n=1, {πn(·)}n=1]. Edelman et al. (2007) introduce locally envy-free equilibrium as a refinement of Nash equilibrium for this mechanism. In words, an equilibrium of the generalized second price auction is locally envy-free if no bidder would be able to improve his payoff by exchanging bids with the bidder one position above him. The following definition extends their notion to allow for “approximate” local envy-freeness. In the definition below, 0-local envy-freeness is equivalent to local envy-freeness.

Deﬁnition 9. A pure equilibrium b of a GSP auction is δ-locally envy-free (i) if for any i ≤ min{I + 1,N}, and for any g(i) ∈ G(i), κi−1 vg(i) − b − (i+1) κi vg(i) − b ≤ δ. The subsequent results rely on the following assumptions about the environment.

Assumption 1. We assume the following:

(i) {κ1, . . . , κI } is linearly independent over Q, and (ii) no two bidders have the same value per click.

17 Assumptions 1(i) and 1(ii) both hold generically. The ﬁrst of these in- cludes the condition that no two click-rates are identical.

4.2 Extended Proper Equilibrium and Test-Set Equilibrium We now present two results. First, we show that, in this game, pure extended proper equilibria are test-set equilibria. Second, we show that pure test-set equilibria (and therefore also pure extended proper equilibria) are approximately locally envy-free. Furthermore, this approximation can be made arbitrarily good by using a suﬃciently ﬁne bid space.

Theorem 11. There exists γ¯ > 0 such that for almost every γ ∈ (0, γ¯), every pure extended proper equilibrium of the GSP auction with bid spaces Bγ is a test-set equilibrium.13

Theorem 12. For all δ > 0 there exists γ¯ > 0 such that for almost every γ ∈ [0, γ¯) every pure test-set equilibrium of the GSP auction with bid spaces Bγ is δ-locally envy-free.14

Corollary 13. For all δ > 0 there exists γ¯ > 0 such that for almost every γ ∈ (0, γ¯) every pure extended proper equilibrium of the GSP auction with bid spaces Bγ is δ-locally envy-free.15

Corollary 13 follows from the two preceding theorems. To prove Theorem 12, we show that if some bidder, say bidder g(i∗), envies the bidder one slot above him, bidder g(i∗ − 1), then bidder g(i∗)’s equilibrium bid is weakly dominated against his test set by a slightly higher alternative bid. Indeed, this alternative performs no worse than the original bid against any element of the test set. Moreover, the alternative performs strictly better against certain downward deviations by bidder g(i∗ − 1), which are best responses for bidder g(i∗ − 1) and therefore in the test set. As a consequence of Theorem 12, with sufficiently finely discretized bid spaces, pure extended proper equilibria and pure test-set equilibria inherit the attractive properties of locally envy-free equilibria. In particular, for sufficiently small γ, they are efficient (that is, for all i ∈ {1,..., min{I,N}}, the ith highest ad slot is won by the bidder with the ith highest value for clicks). n o 13 ∆v ∆κ∆v Specifically, the result can be proven withγ ¯ = min , , where ∆v = 2 8κ1 minn∈{1,...,N−1}{vn − vn+1} and ∆κ = mini∈{1,...,I}{κi − κi+1}. n o 14 ∆v ∆κ∆v δ Specifically, the result can be proven withγ ¯ = min 2 , 8κ , κ . n 1 1 o 15Specifically, the result can be proven withγ ¯ = min ∆v , ∆κ∆v , δ . 2 8κ1 κ1

18 Proposition 14. There exists γ¯ > 0 such that for almost every γ ∈ [0, γ¯) every pure test-set equilibrium of the GSP auction with bid spaces Bγ is eﬃcient.16

Corollary 15. There exists γ¯ > 0 such that for almost every γ ∈ (0, γ¯) every pure extended proper equilibrium of the GSP auction with bid spaces Bγ is eﬃcient.17

4.3 Proper Equilibria Theorem 12 would not remain true if “extended proper equilibrium” were replaced with “proper equilibrium.” Rather, for parameters satisfying As- sumption 1, there exist proper equilibria whose deviations from locally envy- freeness remain bounded away from zero as the ﬁneness of the bid space increases.

Proposition 16. Let (v1, v2, v3) = (15, 10, 5), and (κ1, κ2, κ3) ∈ [100, 101]× [3, 4] × [1, 2]. For almost every γ ∈ (0, 1], the bid profile 5 κ2 − κ3 b3 = γ · γ κ2 10 κ2 − κ3 b2 = γ · γ κ2 1 κ2 b1 = γ 15 − (15 − b2) γ κ1 is a proper equilibrium of the GSP auction with bid spaces Bγ that is not 338 δ-locally envy-free for all δ ≤ 3 . The intuition behind this example is that if bidder 2 is deciding whether or not to increase his bid from b2 to some ˆb2 > b2, then he must weigh the relatively probabilities of bidders 1 and 3 deviating to bids in [b2, ˆb2]. If bidder 1 makes such a deviation, then bidder 2 would profit by raising his bid. However, if bidder 3 makes such a deviation, then bidder 2 would lose by raising his bid. Such a deviation is a best response for bidder 1 but not for bidder 3. Consequently in an extended proper equilibrium, bidder 2 must believe that bidder 1 is much more likely to deviate than bidder 3, in which case bidder 2 should wish to deviate from equilibrium by raising his n o 16 ∆v ∆κ∆v Specifically, the result can be proven withγ ¯ = min 2 , 8κ . n 1 o 17Specifically, the result can be proven withγ ¯ = min ∆v , ∆κ∆v . 2 8κ1

19 bid. However, proper equilibrium allows bidder 2 to believe that bidder 3 is much more likely to deviate than bidder 1, in which case bidder 2 would not wish to deviate, leaving the equilibrium intact.

4.4 Converses The converses of Theorems 11 and 12 are not true. As the following example demonstrates, for parameters satisfying Assumption 1, there exist δ-locally envy-free equilibria that are not test-set equilibria. Similarly, there exist test-set equilibria that are not extended proper.

Proposition 17. Let (v1, v2) = (2, 1), and κ1 ∈ [1, 2]. For almost every 1 γ ∈ 0, 8 , we have the following. For every δ ∈ [0, 1) the set of δ-locally envy-free equilibria of the GSP auction with bid spaces Bγ is γ γ δ ELEF = (b1, b2) ∈ B × B | b2 ≥ 1 − , b2 ≤ 2, b1 > b2 , κ1 the set of pure test-set equilibria is 1 2 E = (b , b ) ∈ Bγ × Bγ | b ≥ γ , b ≤ 2, b > b , b ≤ γ , TS 1 2 2 γ 2 1 2 1 γ and the unique pure extended proper equilibrium is 2 1 (b , b ) = γ , γ . 1 2 γ γ In this example, the set of test-set equilibria is a strict subset of the set of locally envy-free equilibria. A rough intuition for why some locally envy-free equilibria fail to be test-set equilibria is the following. Local envy-freeness, in essence, requires players to use bids that are undominated against the set of profiles in which at most one bidder changes his bid, and where that change is an allocation-preserving decrease. In contrast, test-set equilibrium requires players to use bids that are undominated against the set of profiles in which at most one bidder changes his bid, and where that change is either an allocation-preserving decrease or an allocation-preserving increase. Another feature of this example is that the set of extended proper equilibria is a strict subset of the set of test-set equilibria. This is the case because the test-set condition places no weight on deviations that are not allocation-preserving. In contrast, extended proper equilibrium places some weight on such deviations, albeit ‘infinitely less’ weight than on allocation- preserving deviations.

20 4.5 Nonexistence The following example illustrates that for parameters satisfying Assumption 1, pure test-set equilibria (and therefore also pure extended proper equilibria) do not exist.

Proposition 18. Let (v1, v2, v3) = (15, 10, 5), and (κ1, κ2, κ3) ∈ [100, 101]× 5 [3, 4] × [1, 2]. For almost every γ ∈ 0, 808 , there exists no pure test-set equilibrium of the GSP auction with bid spaces Bγ.

To see the intuition for this result, suppose that b = (b1, b2, b3) were a test-set equilibrium. We show that this requires b1 > b2 > b3. Next, we show that if b2 is too low, then it is weakly dominated by b2 + γ; by increasing his bid, bidder 2 can take advantage of downward deviations by bidder 1. On the other hand, if b2 is too high, then it is weakly dominated by b2 − γ; by decreasing his bid, bidder 2 can protect himself against upward deviations by bidder 3. For appropriately chosen parameters, such as those given in the proposition, all possible values of b2 are either “too low” or “too high.”

5 First Price Menu Auctions

Bernheim and Whinston (1986) define the first price menu auction and propose two refinements – truthful equilibrium and coalition-proof equilibrium. The payoffs of any Nash equilibrium of the menu auction satisfying either condition lie on the bidder-optimal frontier of the core. This section introduces the menu auction, and studies the properties of both extended proper equilibria and test-set equilibria of this auction game. We find that the payoffs of every pure test-set equilibrium lie in the core (although not necessarily on the bidder-optimal frontier of the core). Pure extended proper equilibria of this game possess a weaker “pairwise efficiency” property.

5.1 Environment There is one auctioneer, who selects a decision that affects N ≥ 2 bidders, each of whom offer a menu of payments contingent on the decision chosen. Possible choices for the auctioneer are given by a finite set of decisions X. The gross monetary payoffs that bidder n receives from each decision are described by the function vn : X → R, and the auctioneer receives gross monetary payoffs described by v0 : X → R. The N bidders simultaneously offer contingent payments to the auctioneer, who subsequently chooses a decision that maximizes his total payoff. A

21 bid is a function bn : X → R. We denote the set of feasible bids for bidder n by Bn. Two special cases are of interest. First is the case when the bid spaces 0 are continuous, each taking the form B = {bn : X → R+}. This is the case studied by Bernheim and Whinston (1986). Second is the case γ when the bid spaces are discrete, each taking the form B = {bn : X → R+ | bn(x) ∈ γZ ∩ [0, Γ] ∀x ∈ X}. Here, γ is a parameter controlling the fineness of the discretization, and Γ controls the upper bound of the bid space. To ensure that Γ is not restrictively small, we assume Γ ≥ PN n=0 [maxx∈X vn(x) − minx∈X vn(x)]. This bound ensures that no bidder would wish to bid above Γ even if it were possible and additionally, all bids constructed in the proofs of the following results are contained in the relevant bid spaces. While we will apply test-set equilibrium with both continuous and discretized bid spaces, extended proper equilibrium, which is defined only for finite games, will be applied only with discretized bid spaces. The menu auction game is in an extension of the Bertrand pricing game. For such games, adopting tie-breaking rules that favor the efficient outcome can simplify the description and analysis of equilibrium. So, for simplicity, we assume below that between any two outcomes with the same payoff for a bidder or for the auctioneer, the bidder or auctioneer prefers the outcome with the higher total payoff. The auctioneer chooses a decision that maximizes his total payoff–i.e., N given some b ∈ ×n=1 Bn, the auctioneer selects an element of the set arg maxx∈X h PN i v0(x) + n=1 bn(x) . We assume that, for the reasons described above, the auctioneer resolves ties in favor of the decision with the higher total payoff when this argmax is not a singleton. Payoff maximization and this tie- N breaking rule together induce a decision function x : ×n=1 Bn → X. Let πn(b) = vn(x(b)) − bn(x(b)), and π(b) = (π1(b), . . . , πN (b)). A menu auction N N is then a game G = [{Bn}n=1, {πn(·)}n=1]. We now introduce the following notation. If J ⊆ {1,...,N}, let J¯ = P PN {1,...,N}\J. Let BJ (x) = n∈J bn(x) and B(x) = n=1 bn(x). Similarly, P PN N let VJ (x) = n∈J vn(x) and V (x) = n=1 vn(x). In addition, if π ∈ R , P PN then let ΠJ = n∈J πn and Π = n=1 πn. We also make the following assumption, which holds generically.

Assumption 2. For any J ⊆ {1,...,N}, both VJ (x) and v0(x) + VJ (x) are injective.

22 5.2 Test-Set Equilibrium J Introducing additional notation, let x = arg maxx∈X {v0(x) + VJ (x)} be the decision that maximizes the total payoff of the coalition consisting of J together with the auctioneer, and let xopt = x{1,...,N} to be the efficient decision.18 We also define a family of payoff sets. C0 is the set of the core payoffs, and for γ > 0, Cγ is an outer approximation of the set of core payoffs:

( ∀J ⊆ {1,...,N}, ) γ N h i C = π ∈ R opt opt J¯ J¯ ¯ . ΠJ ≤ [V (x ) + v0(x )] − VJ¯ x + v0 x + |J|γ

We also deﬁne the bidder-optimal frontier of C0:

0 N 0 0 0 0 E = π ∈ R π ∈ C and @π ∈ C with π ≥ π .

The following lemma is extremely useful in proving the following results about test-set equilibria of menu auctions, and is also of independent interest. In words, it says that test-set equilibria of menu auctions are those equilibria in which bidders bid ‘suﬃciently aggressively’ on all losing decisions.

Lemma 19. For all γ ≥ 0, a pure Nash equilibrium b = (b1, . . . , bN ) of the menu auction with bid spaces Bγ is a test-set equilibrium if and only if for all n ∈ {1,...,N} and all x ∈ X, letting x∗ = x(b),

∗ ∗ vn(x) − bn(x) ≤ vn(x ) − bn(x ) + γ.

To prove this lemma, we show that if some bidder’s Nash equilibrium bid fails this condition for some decision x, then it is weakly dominated against his test set by an alternative bid that is slightly higher for that decision. Indeed, this alternative performs no worse than the original bid against any element of the test set, and it performs strictly better in the event that another bidder deviates by raising its bid on x to the highest level consistent with a best response, creating a strategy profile in the test set. Using the lemma, we can prove the following result, which states a sense in which test-set equilibria of menu auctions yield payoffs that are approximately in the core. Furthermore, the payoffs can be made to lie arbitrarily close to the core by making the bid space sufficiently fine.

18Note that Assumption 2 implies that these decisions are uniquely deﬁned.

23 Theorem 20. There exists γ¯ > 0 such that for all γ ∈ [0, γ¯), it is the case that in all pure test-set equilibria of the menu auction with bid spaces Bγ, the auctioneer implements the eﬃcient decision xopt and the bidders receive payoﬀs in Cγ.19

The following result may be interpreted as a partial converse of Theorem 20. It states that any bidder-optimal core payoﬀ vector can be supported by a test-set equilibrium of certain menu auctions (including the auction with bid spaces B0).

0 Theorem 21. Let π ∈ E . Let bn(x) = max{vn(x) − πn, 0} be the π-proﬁt- target strategy of bidder n. Then b = (b1, . . . , bN ) supports π as payoﬀs of a test-set equilibrium of any menu auction in which bn ∈ Bn ∀n.

5.3 Extended Proper Equilibrium Theorem 20 demonstrated that pure test-set equilibria yield (approximate) core payoffs. Whether this is also the case for extended proper equilibria (or even proper equilibria) remains an open question. We are, however, able to demonstrate that extended proper equilibria are “approximately bilaterally efficient,” a weaker property, which states that there is no pair of bidders who could change their bids in a way that would lead to significantly higher total value for the coalition consisting of them and the auctioneer.

Definition 10. A bid profile b is δ-bilaterally efficient if there is no pair of bidders i and j and decisionx ˆ ∈ X such that, letting x∗ = x(b),

X ∗ ∗ ∗ X ∗ v0(ˆx)+vi(ˆx)+vj(ˆx)+ bn(ˆx) ≥ v0(x )+vi(x )+vj(x )+ bn(x )+δ. n6=i,j n6=i,j

The following theorem demonstrates a sense in which pure extended proper equilibria are approximately bilaterally efficient (i.e. approximately 0-bilaterally efficient). Furthermore, it shows that pure extended proper equilibria can be made arbitrarily close to bilaterally efficient by making the bid space sufficiently fine.

Theorem 22. For all δ > 0, there exists a γ¯ > 0 such that for almost every γ ∈ (0, γ¯), every pure extended proper equilibrium of the menu auction with bid spaces Bγ is δ-bilaterally eﬃcient.20

19 1 opt opt Speciﬁcally, the result can be proven withγ ¯ = N minx6=xopt {[V (x ) + v0(x )] − [V (x) + v0(x)]}. 20 δ Speciﬁcally, the result can be proven withγ ¯ = 2 .

24 To prove this result, we show that if δ-bilateral efficiency fails (with, say, bidders 1 and 2 and decisionx ˆ), then bidder 1 should prefer to deviate from equilibrium by raising his bid onx ˆ. While the bidder expects that this deviation will probably not change anything, it would be beneficial if bidder 2 also deviates by raising his bid onx ˆ. Bidder 1’s deviation might also be costly to him, but only if the other bidders deviate in a way that would change the outcome compared to equilibrium. We show that in the most likely such scenario, some deviator, say bidder 3, must be worse off than if he had played according to equilibrium. We then show that the likelihood of bidder 2’s deviation being costly to him is much smaller than the likelihood of bidder 3’s deviation being costly to him. Thus, the across-person restrictions that extended properness places on trembles require bidder 2’s deviation to be much more likely. Consequently, bidder 1 is much more likely to gain than lose if he deviates from equilibrium by altering his bid in this way.

6 Conclusion

We introduce two new refinements of Nash equilibrium for normal form games – extended proper equilibrium and test-set equilibrium – that disci- pline the players’ beliefs about the most likely deviations in games, especially games with three or more players. Both incorporate the novel condition that players believe that another player is more likely to deviate to an alternative best response to the equilibrium than the same or another is to deviate to a strategy that is not a best response. Extended proper equilibrium is defined to resemble and refine proper equilibrium. Test-set equilibrium is defined differently: it neither implies nor is implied by any tremble-based concept. Instead, it directly restricts players to act as if the only possible deviations from equilibrium are ones by a single player to a different best response. We have illustrated the power of these concepts using two examples: the generalized second price auction studied by Edelman et al. (2007) and the first price menu auction studied by Bernheim and Whinston (1986). The former introduced a refinement – “locally envy-free equilibrium” – that is defined only for the GSP mechanism. We show any pure strategy equilibrium satisfying either of our solutions is locally envy-free. In the latter example, the original analysis selects an equilibrium either by restricting attention to equilibria in which players adopt “truthful” strategies or by requiring that the solution be coalition-proof. We show that any test-set equilibrium payoff vector is in the core, even though test-set equilibrium is defined without reference to coalitional games or coalitional deviations and

25 without “truthfulness” restrictions on bidders’ strategies. We also show that extended proper equilibrium possesses a weaker pairwise eﬃciency property. These ﬁndings illuminate the structure and nature of equilibrium in these two celebrated auction games.

26 A Omitted Proofs

A.1 Example Games Proposition 23. For the game in Figure 1, (Up, Left, West) is a proper equilibrium, but is neither an extended proper equilibrium nor a test-set equilibrium. Proof of Proposition 23. Proper Equilibrium. Let ρ = (p1, p2, p3, p4, p5), where p1 puts full weight on (Up, Left, West), p2 puts equal weight on (Up, Right, West) and (Down, Left, West), p3 puts equal weight on (Down, Right, West) and (Up, Left, East), p4 puts equal weight on (Up, Right, East) and (Down, Left, East), and p5 puts full weight on (Down, Right, East). Notice that ρ has full support. We next show that it satisfies strong independence. Let ∈ F be a positive infinitesimal, and let δ be the following vector of positive infinitesimals (2 + 2 + 22 + 3) (2 + 2 + 2) (2 + ) δ = , , , . (1 + )2(1 + 2) 2 + 2 + 22 + 3 2 + 2 + 2 2 +

Then δρ is the F-valued probability measure deﬁned in Figure 4, which can 1 be expressed as the product of the probability measuresp ˆrow = 1+ , 1+ , 1 1 2 pˆcol = 1+ , 1+ , andp ˆgeo = 1+2 , 1+2 .

Figure 4: the probability measure δρ

[West] Left Right [East] Left Right Up 1 2 3 (1+)2(1+2) (1+)2(1+2) Up (1+)2(1+2) (1+)2(1+2) 2 Down 3 4 (1+)2(1+2) (1+)2(1+2) Down (1+)2(1+2) (1+)2(1+2)

Therefore, ρ satisﬁes strong independence. We next show that ρ respects within-person preferences. Row lexicographically prefers Up to Down,21 and indeed we have Up >ρ Down. Column lexicographically prefers Left 22 to Right, and indeed we have Left >ρ Right. Geo is lexicographically indiﬀerent between East and West, so there is nothing to check for him. Therefore, (Up, Left, West) is a proper equilibrium by Proposition 4.

21 1 Up dominates Down against the support of pcol,geo. 22 1 They perform equally well against the support of prow,geo, but Left weakly dominates 2 Right against the support of pcol,geo.

27 Extended Proper Equilibrium. We use the notation of Proposition 7 with σ = (Up, Left, West), Column taking the role of n, Right taking the role ofσ ˆn, Geo taking the role of m, and East taking the role ofs ˆm. Then A = (Up, East) and B = (Up, West). Because ucol(Up, Right, East) > ucol(Up, Left, East) and ucol(Up, Right, West) ≥ ucol(Up, Left, West), Propo- sition 7 implies that (Up, Left, West) is not an extended proper equilibrium.

Test-Set Equilibrium. Letting σ = (Up, Left, West), Column’s test set is

Tcol(σ) = {(Up, West), (Up, East)} .

Right weakly dominates σcol = Left against Tcol(σ), and so this cannot be a test-set equilibrium.

A.2 Extended Proper Equilibrium A.2.1 Technical Lemmas The following two lemmas will be useful in proving Proposition 5. The ﬁrst is a version of Proposition 1 from Blume et al. (1991).

1 K N Lemma 24. Given any LPS ρ = (p , . . . , p ) on ×n=1 Sn, ∃δ > 0 such that if r ∈ (0, δ)K−1, then

X 0 (rρ−n)(s−n) un(sn, s−n) − un(sn, s−n) > 0 s−n∈S−n  K  K X k X k 0 ⇔  p−n(s−n)un(sn, s−n) >L  p−n(s−n)un(sn, s−n) . s ∈S s ∈S −n −n k=1 −n −n k=1

0 ∀sn, sn ∈ Sn, and ∀n. Proof of Lemma 24. To economize on notation, deﬁne for each n ∈ {1,...,N}, the following function

0 X k 0 Hn(k, sn, sn) = p−n(s−n)[un(sn, s−n) − un(sn, s−n)]. s−n∈S−n

0 0 If Hn(k, sn, sn) = 0 for all choices of n, sn, sn, and k, then for each n, each k strategy sn ∈ Sn must produce identical expected payoﬀs against every p−n.

28 As a result, the statement holds vacuously with any choice of δ. We therefore assume henceforth that this is not the case. Deﬁne the constants

0 W = max un(sn, s−n) − un(sn, s−n) n∈{1,...,N} 0 sn,sn∈Sn s−n∈S−n 0 0 B = min Hn(k, sn, sn) : Hn(k, sn, sn) 6= 0 n∈{1,...,N} k∈{1,...,K} 0 sn,sn∈Sn As a result of the previous assumption, B is a well-deﬁned positive number. B We also obviously have W ≥ 0. We then choose δ = B+W > 0, and now show that the statement of the lemma holds with this choice of δ. Suppose 0 that for some n and sn, sn ∈ Sn,  K  K X k X k 0  p−n(s−n)un(sn, s−n) >L  p−n(s−n)un(sn, s−n) . s ∈S s ∈S −n −n k=1 −n −n k=1 0 0 Then there is some k such that Hn(k, sn, sn) ≥ B > 0 and Hn(l, sn, sn) = 0 for all l < k. Suppose r ∈ (0, δ)K−1. Lettingρ ˆ = (pk+1, . . . , pK ) and rˆ = (rk+1, . . . , rK−1), we have that

X 0 (rρ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] s−n∈S−n  0 ≥ r1 ··· rk−1 (1 − rk)Hn(k, sn, sn) +

 X 0 rk (ˆrρˆ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] s−n∈S−n

≥ r1 ··· rk−1 [(1 − rk)B − rkW ]

> r1 ··· rk−1 [B − δ(B + W )] = 0, where the last step follows from the deﬁnition of δ. Now suppose instead 0 that for some n and sn, sn ∈ Sn,  K  K X k X k 0  p−n(s−n)un(sn, s−n) =L  p−n(s−n)un(sn, s−n) . s ∈S s ∈S −n −n k=1 −n −n k=1

29 0 Then Hn(k, sn, sn) = 0 for all k. If this is the case, then we have

X 0 (rρ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] = 0. s−n∈S−n

This completes the proof.

1 K N N Lemma 25. Given any LPS ρ = (p , . . . , p ) on ×n=1 Sn, any α ∈ R++, and any sequence r(t) ∈ (0, 1)K−1 with r(t) → 0, there exists some T such that for all t ≥ T , X αn · max (r(t)ρ−n)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X > αm · max (r(t)ρ−m)(s−m)[um(ˆsm, s−m) − um(sm, s−m)] sˆm∈Sm s−m∈S−m  K X k ∗ ⇔ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s ∈S −n −n k=1  K X k ∗ >L αm p−m[um(sm, s−m) − um(sm, s−m)] s ∈S −m −m k=1

∗ ∗ ∀sn ∈ Sn, sm ∈ Sm, sn ∈ BRn(ρ−n), sm ∈ BRm(ρ−m), and ∀n, m. N ∗ Proof of Lemma 25. Let α ∈ R++. Given ρ, select for every n some sn ∈ BRn(ρ−n). Since all elements of BRn(ρ−n) must produce identical expected k payoﬀs against every p−n, it will not matter which element is chosen. To economize on notation, deﬁne for each n, m ∈ {1,...,N} (possibly equal), the following function

X k ∗ Hnm(k, sn, sm) = αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s−n∈S−n X k ∗ − αm p−m[um(sm, s−m) − um(sm, s−m)]. s−m∈S−m

If Hnm(k, sn, sm) = 0 for all choices of n, m, sn, sm, and k, then for each n, each strategy sn ∈ Sn must produce identical expected payoﬀs against every k p−n. As a result, the statement holds vacuously with any choice of T . We

30 therefore assume henceforth that this is not the case. Deﬁne the constants

0 W = max αn un(sn, s−n) − un(sn, s−n) n∈{1,...,N} 0 sn,sn∈Sn s−n∈S−n

B = min {|Hnm(k, sn, sm)| : Hnm(k, sn, sm) 6= 0} n,m∈{1,...,N} k∈{1,...,K} sn∈Sn,sm∈Sm

As a result of the previous assumption, B is a well-deﬁned positive number. B We also obviously have W ≥ 0. Therefore B+2W > 0. In addition, some δ > 0 must exist that satisﬁes the conditions of Lemma 24. Let T be such K−1 n B o that r(t) ∈ 0, min δ, B+2W for all t ≥ T . Because r(t) → 0, such a T must exist. We now prove that the statement of this lemma holds with this choice of T . Suppose that for some n, m, sn ∈ Sn, and sm ∈ Sm,  K X k ∗ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s ∈S −n −n k=1  K X k ∗ >L αm p−m[um(sm, s−m) − um(sm, s−m)] . s ∈S −m −m k=1

Then there is some k such that Hnm(k, sn, sm) ≥ B > 0 and Hnm(l, sn, sm) = k+1 K 0 for all l < k. Lettingρ ˆ = (p , . . . , p ) andr ˆ(t) = (rk+1(t), . . . , rK−1(t)),

31 we have that for all t ≥ T , X αn · max (r(t)ρ−n)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X − αm · max (r(t)ρ−m)(s−m)[um(ˆsm, s−m) − um(sm, s−m)] sˆm∈Sm s−m∈S−m X ∗ = αn (r(t)ρ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] s−n∈S−n X ∗ − αm (r(t)ρ−m)(s−m)[um(sm, s−m) − um(sm, s−m)] s−m∈S−m 

≥ r1(t) ··· rk−1(t) (1 − rk(t))Hnm(k, sn, sm) +

X ∗ rk(t) (ˆr(t)ρˆ−n)(s−n)αn[un(sn, s−n) − un(sn, s−n)] s−n∈S−n  X ∗ − rk(t) (ˆr(t)ρˆ−m)(s−m)αm[um(sm, s−m) − um(sm, s−m)] s−m∈S−m

≥ r1(t) ··· rk−1(t) [(1 − rk(t))B − rk(t)2W ] B > r (t) ··· r (t) B − (B + 2W ) 1 k−1 B + 2W = 0.

The ﬁrst step in the above follows from Lemma 24, which implies that any maximizerss ˆn ∈ Sn ands ˆm ∈ Sm must be elements of BRn(ρ−n) and BRm(ρ−m), respectively. Now suppose instead that that for some n, m, sn ∈ Sn, and sm ∈ Sm  K X k ∗ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s ∈S −n −n k=1  K X k ∗ =L αm p−m[um(sm, s−m) − um(sm, s−m)] . s ∈S −m −m k=1

32 Then Hnm(k, sn, sm) = 0 for all k. If this is the case, then for all n, X αn · max (r(t)ρ−n)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X − αm · max (r(t)ρ−m)(s−m)[um(ˆsm, s−m) − um(sm, s−m)] sˆm∈Sm s−m∈S−m X ∗ = αn (r(t)ρ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] s−n∈S−n X ∗ − αm (r(t)ρ−m)(s−m)[um(sm, s−m) − um(sm, s−m)] s−m∈S−m = 0, where as before, the ﬁrst step in the above follows from Lemma 24. This completes the proof.

We now state and prove some properties possessed by LPSs, which will be used in later arguments.

1 K N Lemma 26. Suppose that an LPS ρ = (p , . . . , p ) on ×n=1 Sn is equivalent N to an F-valued probability measure pˆ on ×n=1 Sn. For all sJ ∈ ×j∈J Sj and pˆJ (sJ ) 1 23 all sI ∈ ×i∈I Si, sJ ≥ρ sI iff ≥ for some n ∈ . pÎ (sI ) n N k k Proof of Lemma 26. Let m0 = mink,J min{pJ (sJ ): sJ ∈ supp pJ }. Let 1 n ∈ N be such that m0 > n . Because ρ andp ˆ are equivalent, there exists K−1 a vector of infinitesimals = (1, . . . , K−1) ∈ F such thatp ˆ = ρ. l l Suppose sJ ≥ρ sI , so that k = min{l : pJ (sJ ) > 0} ≤ min{l : pI (sI ) > 0}. Thenp ˆJ (sJ ) ≥ 1 ··· k−1(1 − k)m0, andp Î (sI ) ≤ 1 ··· k−1(1 − k). pˆJ (sJ ) 1 Therefore ≥ m0 > . pÎ (sI ) n l Suppose instead that sJ <ρ sI , so that k = min{l : pJ (sJ ) > 0} > l min{l : pI (sI ) > 0}. Thenp ˆJ (sJ ) ≤ 1 ··· k−1(1 − k) andp Î (sI ) ≥ pˆJ (sJ ) k−1(1−k) 1 ··· k−2(1 − k−1)m0. Consequently, ≤ , which is an pÎ (sI ) (1−k−1)m0 1 infinitesimal, and therefore is less than n ∀n ∈ N. Lemma 27. The following are true of an LPS ρ = (p1, . . . , pK ) that satisfies strong independence:

0 (i) Let I and J be disjoint subsets of {1,...,N}. Then sJ ≥ρ sJ iﬀ 0 (sI , sJ ) ≥ρ (sI , sJ ).

23 If J ⊆ {1,...,N}, then we usep ˆJ to denote the marginal ofp ˆ on ×j∈J Sj .

33 (ii) Let I and J be disjoint subsets of {1,...,N}. Let the same be true of 0 0 0 0 0 0 I and J . Then sI >ρ sI0 and sJ ≥ρ sJ0 ⇒ (sI , sJ ) >ρ (sI0 , sJ0 ). Proof of Lemma 27. Claim (i). Since ρ satisﬁes strong independence, there exists an F-valued product measurep ˆ and a vector of positive in- ﬁnitesimals = (1, . . . , K−1) for whichp ˆ = ρ. Then

0 pˆJ (sJ ) 1 sJ ≥ρ sJ ⇔ 0 ≥ for some n ∈ N pˆJ (sJ ) n pÎ,J (sI , sJ ) 1 ⇔ 0 ≥ for some n ∈ N pÎ,J (sI , sJ ) n 0 ⇔ (sI , sJ ) ≥ρ (sI , sJ ), where the first and third steps follow from Lemma 26, and the second step follows from the fact thatp ˆ is a product measure.

0 0 Claim (ii). Suppose that sI >ρ sI0 and sJ ≥ρ sJ0 . Then by Lemma 0 pˆ 0 (s ) I I0 1 pˆJ (sJ ) 1 26, < ∀n ∈ . Similarly, 0 > for some n ∈ , or, pˆ (s ) n N pˆ 0 (s ) n 0 N I I J J0 0 0 pˆ 0 (s ) J J0 equivalently, < n0. Then sincep ˆ is a product measure, pˆJ (sJ ) 0 0 0 0 pÎ0,J0 (sI0 , sJ0 ) pÎ0 (sI0 ) pˆJ0 (sJ0 ) n0 = · < for all n ∈ N, pÎ,J (sI , sJ ) pÎ (sI ) pˆJ (sJ ) n

0 0 pˆ 0 0 (s ,s ) which implies that I ,J I0 J0 < 1 for all n ∈ . Another application of pˆI,J (sI ,sJ ) n N 0 0 Lemma 26 gives the desired result, (sI , sJ ) >ρ (sI0 , sJ0 ).

A.2.2 Main Results

24 N Proof of Theorem 1. Let α ∈ R++. We ﬁrst demonstrate the existence of an (α, )-extended proper equilibrium for any ∈ (0, 1). Let M = PN K ∗ maxn |Sn|, K = n=1 |Sn|, and δ = 2M . For any player n, let ∆ (Sn) = {σn ∈ ∆(Sn): σn(sn) ≥ δ, ∀sn ∈ Sn}, which is a nonempty and compact sub- 0 N ∗ set of ∆ (Sn). Next, we deﬁne the correspondence F : ×n=1 ∆ (Sn) ⇒ N ∗ ×n=1 ∆ (Sn) by  if α L (s |σ) > α L (s |σ)  N n n n m m m  ∗ ∗ ∗ ∗  F (σ) = σ ∈ × ∆ (Sn) then σn(sn) ≤ · σm(sm), . n=1  ∀n, ∀m, ∀sn ∈ Sn, ∀sm ∈ Sm 

24With a few small adjustments, the argument follows the proof of existence of proper equilibria given by Myerson (1978).

34 For any σ, the points in F (σ) are those that satisfy a ﬁnite collection of linear inequalities, so F (σ) is a closed, convex set. We next demonstrate that F (σ) is nonempty. Let φm(sn) be the number of pure strategies sm ∈ Sm PN such that αnLn(sn|σ) < αmLm(sm|σ). Let φ(sn) = m=1 φm(sn), and let 0 Ln = |{sn ∈ Sn : Ln(sn|σ) = 0}|. Finally, let

φ(s ) ( n if L (s |σ) > 0 ∗ 2M n n σn(sn) = 1 P φ(sn) 0 1 − if Ln(sn|σ) = 0 Ln sn:Ln(sn|σ)>0 2M

∗ ∗ ∗ To demonstrate that σ ∈ F (σ), we show (i) σn ∈ ∆ (Sn), and (ii) if ∗ ∗ αnLn(sn|σ) > αmLm(sm|σ), then σn(sn) ≤ · σm(sm). To show (i), we ∗ φ(sn) K consider two cases. First, if Ln(sn|σ) > 0, then σn(sn) = 2M ≥ 2M = δ. ∗ 1 P φ(sn) Second, if Ln(sn|σ) = 0, then σ (sn) = 0 1 − ≥ n Ln sn:Ln(sn|σ)>0 2M 1 1 1 1 0 1 − M = 0 ≥ ≥ δ. To show (ii), we also consider two Ln 2M 2Ln 2M cases. First, suppose Ln(sn|σ) > 0 and Lm(sm|σ) > 0. Then αnLn(sn|σ) > ∗ φ(sn) αmLm(sm|σ) implies that φ(sn) ≥ φ(sm) + 1, and so σn(sn) = 2M ≤ φ(sm)+1 ∗ 2M = · σm(sm). Second, suppose Ln(sn|σ) > 0 and Lm(sm|σ) = 0. ∗ φ(sn) ∗ 25 Then φ(sn) ≥ 1, and so σn(sn) = 2M ≤ 2M ≤ · σm(sm). Finally, continuity of each Ln(sn|·) function implies that F (·) must be upper-semicontinuous. Therefore F satisfies all the conditions of the Kaku- tani Fixed Point Theorem (Kakutani, 1941), so there exists some σ ∈ N ∗ ×n=1 ∆ (Sn) such that σ ∈ F (σ ). By the definition of F , this point is an (α, )-extended proper equilibrium. So for any 0 < < 1, there exists an (α, )-extended proper equilib- N rium. Since ×n=1 ∆(Sn) is a compact set, there must exist a convergence subsequence, and an extended proper equilibrium σ = lim→0 σ . Proof of Proposition 2. Sufficiency. Suppose (ρ, σ) is a lexicographic Nash equilibrium, where ρ satisfies strong independence. As Blume et al. (1991) observe, strong independence of ρ implies that p1 must be a product 1 measure. Furthermore, if pn(sn) > 0 for a player n, then by condition (i) of Definition 3,

X 1 X 1 0 p−n(s−n)un(sn, s−n) ≥ p−n(s−n)un(sn, s−n) s−n∈S−n s−n∈S−n

25 The last step uses the fact, proved in the process of showing (i), that Lm(sm|σ) = 0 ∗ 1 implies that σm(sm) ≥ 2M .

35 0 1 for all sn ∈ Sn and all n. Therefore p satisfies the definition of a Nash equilibrium. Thus, by condition (ii) of Definition 3, σ = p1 is a Nash equilibrium. Necessity. Suppose that σ is a Nash equilibrium. Taking K = 1 and p1 = σ produces an LPS ρ such that (ρ, σ) is a lexicographic Nash equilibrium. Furthermore, σ, and therefore p1, must be a product measure. Since K = 1, ρ therefore satisfies strong independence.

Proof of Proposition 3. This is a restatement of Proposition 7 from Blume et al. (1991).

Proof of Proposition 4. This is a restatement of Proposition 8 from Blume et al. (1991).

Proof of Proposition 5. Sufficiency. Suppose (ρ, σ) is a lexicographic Nash equilibrium, where ρ = (p1, . . . , pK ) is an LPS that satisfies the stated conditions. As Blume et al. (1991) point out, ρ satisfies strong independence if and only if there is a sequence r(t) ∈ (0, 1)K−1 with r(t) → 0 such that r(t)ρ is a product measure for all t. For each n, define σn(t) as the marginal on Sn of r(t)ρ, and let σ(t) = (σ1(t), . . . , σN (t)). Note that 0 limt→∞ σ(t) = p1 = σ. Additionally, since ρ has full support, σn(t) ∈ ∆ (Sn) ∀n and ∀t. Define

k k m0 = min min{pn(sn): sn ∈ supp pn} n∈{1,...,N} k∈{1,...,K} r (t) (t) = max k k∈{1,...,K−1} [1 − rk(t)]m0

Since r(t) → 0, we also have (t) → 0. Let T be as in the statement of N Lemma 25, and let α ∈ R++ be as in the deﬁnition of “respects within- and-across-person preferences.” We claim that for all t ≥ T , σ(t) is an (α, (t))-extended proper equilibrium. To see this, suppose n, m, sn ∈ Sn, and sm ∈ Sm are such that αnLn(sn|σ(t)) > αmLm(sm|σ(t)). Thus, X αn · max σ−n(t)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X > αm · max σ−m(t)(s−m)[um(ˆsm, s−m) − um(sm, s−m)]. sˆm∈Sm s−m∈S−m

36 Since r(t)ρ is a product measure, (r(t)ρ−n)(s−n) = ×m6=n σm(t)(sm). Therefore the above equation is equivalent to X αn · max (r(t)ρ−n)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X > αm · max (r(t)ρ−m)(s−m)[um(ˆsm, s−m) − um(sm, s−m)]. sˆm∈Sm s−m∈S−m

∗ ∗ By Lemma 25, we have that for all sn ∈ BRn(ρ−n) and sm ∈ BRm(ρ−m),  K X k ∗ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] s ∈S −n −n k=1  K X k ∗ >L αm p−m[um(sm, s−m) − um(sm, s−m)] , s ∈S −m −m k=1 which, because ρ respects within-and-across-person preferences, implies that l sn <ρ sm. Let k = min{l : pm(sm) > 0}. Because ρ has full support, such a k must exist. Furthermore, we must then have σm(t)(sm) ≥ r1(t) ··· rk−1(t)[1−rk(t)]m0. Because sn <ρ sm, we must also have σn(t)(sn) ≤ r1(t) ··· rk(t), and therefore

(t) · σm(t)(sm) ≥ (t) · r1(t) ··· rk−1(t)[1 − rk(t)]m0

rk(t) ≥ · r1(t) ··· rk−1(t)[1 − rk(t)]m0 [1 − rk(t)]m0

= r1(t) ··· rk(t)

≥ σn(t)(sn).

In review, we have shown that if t ≥ T , then αnLn(sn|σ(t)) > αmLm(sm|σ(t)) implies that σn(t)(sn) ≤ (t) · σm(t)(sm). Consequently, for all t ≥ T , σ(t) is an (α, (t))-extended proper equilibrium. Since (t) → 0 and σ(t) → σ, σ is an extended proper equilibrium. Necessity. Suppose σ is an extended proper equilibrium. Then there ex- N N 0 ists some scaling vector α ∈ R++ and some sequence σ(t) ∈ ×n=1 ∆ (Sn) of (α, (t))-extended proper equilibria that converge to σ. By Proposition 2 of 1 K N Blume et al. (1991), there is an LPS ρ = (p , . . . , p ) on ×n=1 Sn such that a subsequence σ(τ) of σ(t) can be written as σ(τ) = r(τ)ρ for a sequence r(τ) ∈ (0, 1)K−1 with r(τ) → 0. Note that p1 = σ, since σ(τ) → σ. Also,

37 N 0 since σ(τ) ∈ ×n=1 ∆ (Sn), ρ has full support and satisﬁes strong independence. Finally, we will show that ρ respects within-and-across-person preferences (with the same choice of α). Suppose that for some n, m, sn ∈ Sn, ∗ ∗ sm ∈ Sm, sn ∈ BRn(ρ−n), and sm ∈ BRm(ρ−m),  K X k ∗ αn p−n(s−n)[un(sn, s−n) − un(sn, s−n)] (1) s ∈S −n −n k=1  K X k ∗ >L αm p−m[um(sm, s−m) − um(sm, s−m)] . s ∈S −m −m k=1 By Lemma 25, there exists some T such that for all τ ≥ T , X αn · max (r(τ)ρ−n)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] sˆn∈Sn s−n∈S−n X > αm · max (r(τ)ρ−m)(s−m)[um(ˆsm, s−m) − um(sm, s−m)]. sˆm∈Sm s−m∈S−m That is, X αn · max σ−n(τ)(s−n)[un(ˆsn, s−n) − un(sn, s−n)] (2) sˆn∈Sn s−n∈S−n X > αm · max σ−m(τ)(s−m)[um(ˆsm, s−m) − um(sm, s−m)], sˆm∈Sm s−m∈S−m

l or αnLn(sn|σ(τ)) > αmLm(sm|σ(τ)). Let k = min{l : pn(sn) > 0}. Since ρ has full support, k must exist. To accommodate the case of k = K, the following argument remains valid if we deﬁne rK (τ) = 0 for all τ. Since r(τ) → 0 and (τ) → 0, there must exist some τ ∗ ≥ T for which (τ ∗) < ∗ k [1 − rk(τ )]pn(sn). Consequently, ∗ ∗ ∗ ∗ k σn(τ )(sn) ≥ r1(τ ) ··· rk−1(τ )[1 − rk(τ )]pn(sn) ∗ ∗ ∗ > (τ ) · r1(τ ) ··· rk−1(τ ).

l Suppose by way of contradiction that sm ≤ρ sn, so that min{l : pm(sm) > ∗ ∗ ∗ 0} ≥ k, and thus σm(τ )(sm) ≤ r1(τ ) ··· rk−1(τ ). Then we would have ∗ ∗ σn(τ )(sn) > (τ )·σm(sm), which, taken together with equation (2), would contradict the fact that σ(τ ∗) is an (α, (τ ∗))-extended proper equilibrium. It must therefore be the case that sn <ρ sm. In review, we have shown that equation (1) implies that sn <ρ sm, which establishes that ρ respects within-and-across-person preferences.

38 Proof of Theorem 6. Letσ ˆ be a proper equilibrium. Using the characterization of proper equilibrium in Proposition 4, there must exist some 1 K LPS ρ = (p , . . . , p ) on S1 × S2 that satisﬁes strong independence, has full support, respects within-person preferences, and for which (ρ, σˆ) is a lexicographic Nash equilibrium. For n ∈ {1, 2} and k ∈ {1,...,K}, let k k l πn = {sn ∈ Sn : pn(sn) > 0 and pn(sn) = 0 for l < k}. 1 K Because ρ has full support, {πn, . . . , πn } is a partition of Sn. Because ρ k respects within-person preferences, each sn ∈ πn is optimal for player n K l (against ρ−n) among strategies in ∪l=kπn. We can then use Lemma 24 to construct a number δ > 0 such that for all n ∈ {1, 2}, k ∈ {1,...,K}, K−1 k 0 K l r−n ∈ (0, δ) , sn ∈ πn, and sn ∈ ∪l=kπn, X 0 (r−nρ−n)(s−n)[un(sn, s−n) − un(sn, s−n)] ≥ 0. s−n∈S−n Next, for n ∈ {1, 2}, deﬁne the following family of sets:

K−1 ∗ h 2K2+K i ∆n, = σn ∈ ∆(Sn) σn = rnρn for some rn ∈ , .

∗ In words, ∆n, is a set of totally mixed probability distributions that are 1 close to pn =σ ˆn (where the level of closeness is controlled by ). We also define k k m0 = min min{pn(sn): sn ∈ supp pn}. n∈{1,2} k∈{1,...,K} n o We begin by arguing that for any α ∈ 2 and any ∈ 0, min δ, m0 , R++ m0+1 ∗ ∗ there exists an (α, )-extended proper equilibrium in the set ∆1, × ∆2,. To ∗ ∗ ∗ ∗ do so, we define the correspondence F : ∆1, × ∆2, ⇒ ∆1, × ∆2, by   if αnLn(sn|σ) > αmLm(sm|σ)  ∗ ∗ ∗ ∗ ∗  F (σ) = σ ∈ ∆1, × ∆2, then σn(sn) ≤ · σm(sm), .  ∀n, ∀m, ∀sn ∈ Sn, ∀sm ∈ Sm  For any σ, the points in F (σ) are those that satisfy a finite collection of linear inequalities, so F (σ) is a closed, convex set. ∗ ∗ We next demonstrate that F (σ) is nonempty for any σ ∈ ∆1, × ∆2,. 0 k 0 Because < δ, we know that Ln(sn|σ) ≤ Ln(sn|σ) for any sn ∈ πn, sn ∈ K l ∪l=kπn. For k ∈ {1,...,K}, let 2 k X k l ηn = max{l : αnLn(sn|σ) > αmLm(sm|σ) ∀sn ∈ πn, ∀sm ∈ πm}. m=1

39 Then for k ∈ {1,...,K − 1}, let

k+1 k k 1+(K+1)(η−n −η−n) rn = .

k+1 k k For all n ∈ {1, 2} and all k ∈ {1,...,K − 1}, ηn ≥ ηn, and so rn = k+1 k 1+(K+1)(η−n −η−n) k+1 k ≤ < δ. Additionally, ηn ≤ 2K − 1, and so rn = 1+(K+1)(ηk+1−ηk ) 2 −n −n ≥ 2K +K . From the previous two inequalities we con- ∗ clude rnρn ∈ ∆n,. Next, suppose that αnLn(sn|σ) > αmLm(sm|σ). Let k k l and l be the indices for which sn ∈ πn and sm ∈ πm. We must then have k l ηn ≥ ηm + 1. Consequently,

1 k−1 rnρn(sn) ≤ rn ··· rn k = k−1+(K+1)ηn l ≤ k+K+(K+1)ηm k+K−l l+(K+1)ηl = m (1 − )m0 · (1 − )m0 l+(K+1)ηl ≤ m (1 − )m0 · (1 − )m0 l+(K+1)ηl ≤ m (1 − )m0 l l+(K+1)ηm l ≤ (1 − rm)m0 1 l−1 l = · rm ··· rm (1 − rm)m0 ≤ · rmρm(sm). In the above, the fifth step follows from K ≥ l and k ≥ 1; the sixth step follows from < m0 ; and the seventh step follows from rl ≤ . To deal 1+m0 m K with the case where l = K, the above remains correct if we set rm = 0. ∗ To review, we have shown (i) rnρn ∈ ∆n, and (ii) if αnLn(sn|σ) > αmLm(sm|σ), then rnρn(sn) ≤ · rmρm(sm). Thus, (r1ρ1, r2ρ2) ∈ F (σ), and so F (σ) is nonempty, as desired. Finally, continuity of each Lm(sm|·) implies that F (·) must be upper- semicontinuous. Therefore F satisfies all the conditions of the Kakutani ∗ ∗ Fixed Point Theorem (Kakutani, 1941), so there exists some σ ∈ ∆1, ×∆2, such that σ ∈ F (σ). By the definition of F , this point is an -extended proper equilibrium. n o So for all ∈ 0, min δ, m0 , there exists an (α, )-extended proper m0+1 ∗ ∗ ∗ equilibrium σ ∈ ∆1, × ∆2,. Because σn ∈ ∆n, for all n ∈ {1, 2}, we 1 1 must have |σn(sn) − pn(sn)| ≤ ∀sn ∈ Sn, and so lim→0 σn = pn =σ ˆn.

40 Consequently, this sequence of (α, )-extended proper equilibria converges toσ ˆ as → 0, which establishes thatσ ˆ is extended proper.

Proof of Theorem 7. Suppose that σ is an extended proper equilibrium. By Proposition 5, there must exist some LPS ρ that satisﬁes strong independence, has full support, respects within-and-across-person preferences, and for which (ρ, σ) is a lexicographic Nash equilibrium. Let n, m ∈ {1,...,N} with n 6= m, letσ ˆn ∈ ∆(Sn), and lets ˆm ∈ BRm(σ−m). The elements of ×i6=n Si can be partitioned into the following four sets: sm =s ˆm and A = (sm, s−nm) σi(si) > 0 ∀i∈ / {n, m} sm ∈ BRm(σ−m) \ sˆm and B = (sm, s−nm) si ∈ BRi(σ−i) ∀i∈ / {n, m} sm ∈/ BRm(σ−m) or C = (sm, s−nm) ∃i∈ / {n, m} s.t. si ∈/ BRi(σ−i)    sm =s ˆm and  D = (sm, s−nm) si ∈ BRi(σ−i) ∀i∈ / {n, m} and  ∃i∈ / {n, m} s.t. σi(si) = 0 

0 00 00 Let s−n ∈ A and let s−n ∈ C. For some j 6= n, we must have sj ∈/ 0 BRj(σ−j). On the other hand, sm =s ˆm ∈ BRm(σ−m). Therefore, since ρ respects within-and-across-person preferences, it must be the case that 0 00 0 sm >ρ sj . In addition, because σi(si) > 0 ∀i∈ / {n, m}, we must have 1 0 0 00 p−nm(s−nm) > 0, and therefore s−nm ≥ρ s−nj. Then by Lemma 27(ii), we 0 00 must have s−n >ρ s−n. 0 00 0 Let s−n ∈ A and let s−n ∈ D. Because σi(si) > 0 ∀i∈ / {n, m}, we 1 0 00 must have p−nm(s−nm) > 0. On the other hand, because σi(si ) = 0 for 1 00 0 00 some i∈ / {n, m}, we must have p−nm(s−nm) = 0. Therefore s−nm >ρ s−nm. 0 00 Finally, because sm = sm =s ˆm, we can apply Lemma 27(i) to obtain 0 00 s−n >ρ s−n. As a consequence of the fact that all elements of A are inﬁnitely more likely under the LPS ρ than any element of either C or D, along with the fact that ρ has full support, the criterion given in the theorem is a necessary condition for σn to lexicographically outperformσ ˆn against ρ−n, which is a necessary condition for (ρ, σ) to be a lexicographic Nash equilibrium.

41 A.3 Test-Set Equilibrium

Proof of Proposition 8. Let σ be a strategy proﬁle for which σn is weakly dominated byσ ˆn in this way. Therefore,

un(ˆσn, sˆm, σ−nm) > un(σn, sˆm, σ−nm).

Similarly, we also have that for any sm ∈ BRm(σ−m)\sˆm, un(ˆσn, sm, σ−nm) ≥ un(σn, sm, σ−nm). Consequently, we must have that σm(ˆsm) = 0, since otherwise these inequalities imply un(ˆσn, σ−n) > un(σn, σ−n), contradicting Nash equilibrium. Therefore (ˆsm, σ−nm) ∈ Tn(σ) ⊆ (ˆsm, σ−nm) ∪ ∆(B). Consequently, σn is weakly dominated byσ ˆn against Tn(σ), and thus σ is not a test-set equilibrium.

Proof of Proposition 9. Claim (i). This follows immediately from The- orem 10 and the existence of proper equilibria in ﬁnite normal form games.

Claim (ii). We use the fact that any potential game is strategically N equivalent to a game in which un(s) = P (s) ∀n ∀s ∈ ×n=1 Sn, where P is referred to as the potential function of the game. We demonstrate that ∗ any s ∈ arg max N P (s) is a pure strategy test-set equilibrium. It s∈×n=1 Sn is well-known that such a strategy proﬁle is a Nash equilibrium, so it only remains to check the test-set condition. ∗ Let n ∈ {1,...,N} and σ−n ∈ Tn(s ). Then ∃m 6= n ands ˆm ∈ ∗ ∗ BRm(s ) for which σ−n = (ˆsm, s ). Let P¯ = max N P (s). Since −m −nm s∈×n=1 Sn ∗ ∗ ∗ ∗ ∗ ∗ ¯ sˆm ∈ BRm(s−m), we have P (sn, sˆm, s−nm) = P (sn, sm, s−nm) = P . More- ∗ ¯ ∗ over, for anyσ ˆn ∈ ∆(Sn), P (ˆσn, sˆm, s−nm) ≤ P . Therefore, P (sn, σ−n) ≥ P (ˆσn, σ−n), as desired.

Claim (iii). We ﬁrst show that for three-player games in which each player has two pure strategies, every extended proper equilibrium is a test- set equilibrium. The result will then follow from Theorem 1. Consider a three player game with strategy sets Sn = {an, bn}. Suppose σ is an extended proper equilibrium of this game that fails the test-set condition. Then without loss of generality, there existsσ ˆ1 ∈ ∆(S1) that weakly dominates σ1 against T1(σ). Also without loss of generality, let a2 ∈ BR2 (σ−2) be such that

u1(ˆσ1, a2, σ3) > u1(σ1, a2, σ3). (3)

Now if σ2(a2) = 1, then (3) is a contradiction to Nash equilibrium. Therefore σ2(a2) < 1, so b2 ∈ BR2 (σ−2), and so by the failure of player 1’s test-set

42 condition,

u1(ˆσ1, b2, σ3) ≥ u1(σ1, b2, σ3) (4)

We must then have σ2 = b2, or else equations (3) and (4) would together contradict Nash equilibrium. We consider two cases. First, BR3 (σ−3) = {a3, b3}. Then T1(σ) = {(a2, σ3), (b2, σ3), (b2, a3), (b2, b3)}. So in addition to (3) and (4) we have

u1(ˆσ1, b2, a3) ≥ u1(σ1, b2, a3) (5)

u1(ˆσ1, b2, b3) ≥ u1(σ1, b2, b3) (6)

The above inequalities imply that for all totally mixed strategy proﬁles σ suﬃciently close to σ,

u1(ˆσ1, σ−1) − u1(σ1, σ−1) = [u1(ˆσ1, a2, σ3) − u1(σ1, a2, σ3)] σ2(a2) + [u1(ˆσ1, b2, σ3) − u1(σ1, b2, σ3)] σ2(b2) > 0.

To see the last step, (3) implies that the ﬁrst term is positive for totally mixed σ suﬃciently close to σ, and (5) and (6) imply that the second term is nonnegative. This contradicts σ being a perfect equilibrium. Without loss of generality, the second case is BR3 (σ−3) = {a3}, so that σ3 = a3. Then T1(σ) = {(a2, a3), (b2, a3)}. Using the notation of Proposition 7 (with n = 1, m = 2, ands ˆm = a2), A = (a2, a3) and B = (b2, a3). Applying the theorem, (3) and (4) contradict σ being an extended proper equilibrium.

Proof of Theorem 10. Let σ = (σ1, σ2) be a proper equilibrium. Using the characterization of proper equilibrium in Proposition 4, there must exist 1 K some LPS ρ = (p , . . . , p ) on S1 × S2 that satisﬁes strong independence, has full support, respects within-person preferences, and for which (ρ, σ) is a lexicographic Nash equilibrium. Suppose that σ is not a test-set equilibrium. Then without loss of generality, there existsσ ˆ1 ∈ ∆(S1) ands ˆ2 ∈ BR2(σ1) such that (i) u1(ˆσ1, sˆ2) > u1(σ1, sˆ2), and (ii) u1(ˆσ1, s2) ≥ u1(σ1, s2) ∀s2 ∈ BR2(σ1). Because σ1 ∈ BR1(ρ2) and because ρ has full support, there must exist somes ˜2 ≥ρ sˆ2 for which u1(ˆσ1, s˜2) < u1(σ1, s˜2). By the contrapositive of (ii),s ˜2 6∈ BR2(σ1). Becauses ˆ2 ∈ BR2(σ1) and because ρ respects within-person preferences, we obtains ˆ2 >ρ s˜2, a contradiction.

43 A.4 Generalized Second Price Auction A.4.1 Technical Lemmas The following lemmas will be useful in proving results about GSP auctions. Lemma 28 shows that for suﬃciently small values of γ > 0, there will be no ties among the highest min{I + 1,N} bids in any pure equilibrium. Lemma 29 uses Assumption 1(i) to establish that best responses are allocation- preserving for almost every γ ∈ R. h n o Lemma 28. For all γ ∈ 0, min ∆v , ∆κ∆v , if b is a pure equilibrium of 2 8κ1 the GSP auction with bid spaces Bγ, then for all i ∈ {1,..., min{I,N − 1}}, b(i) > b(i+1).26 n o Proof of Lemma 28. Let γ ∈ 0, min ∆v , ∆κ∆v and i ∈ {1,..., min{I, 2 8κ1 N − 1}}. We show this result for this value of γ. That it also applies for γ = 0 can be proven using similar methods. By Assumption 1(i), κi > κi+1. Suppose b(i) = ··· = b(i+K) = b∗ is a maximal tie (i.e. there are exactly K + 1 bidders who bid b∗, where K ≥ 1). Any bidder with per-click value v who is part of the tie earns the payoﬀ

" K−1 # 1 X κ v − b(i+K+1) + [κ (v − b∗)] . K + 1 i+K i+k k=0

If b∗ > b(i+K+1) + γ, then by lowering his bid by γ the bidder would earn the payoﬀ (i+K+1) κi+K v − b . And by raising his bid by γ the bidder would earn a payoﬀ of at least

∗ κi (v − b − γ) .

Let v0 < v00 be the per-click values of two of the tied bidders. We prove the result by contradiction, considering two cases. Case 1 : b∗ ≤ b(i+K+1) + γ. We must have v0 ≥ b∗ − γ, or else this bidder 00 0 ∗ would be better oﬀ bidding zero. Therefore v ≥ v + ∆v ≥ b − γ + ∆v ≥ 26Given the underlying parameters, we deﬁne the quantities

∆v = min {vn − vn+1} n∈{1,...,N−1}

∆κ = min {κi − κi+1} i∈{1,...,I}

By Assumption 1(i), ∆κ > 0. By Assumption 1(ii), ∆v > 0.

44 ∗ ∆v 00 b + 2 . We then show that the v bidder would beneﬁt by raising his bid to b∗ + γ.27 The incremental proﬁt is at least

"K−1 # 1 X κ v00 − b∗ − γ − [κ (v00 − b∗)] + κ v00 − b(i+K+1) i K + 1 i+k i+K k=0 1 ≥ κ v00 − b∗ − γ − κ (v00 − b∗) i K + 1 i K − 1 1 − κ (v00 − b∗) − κ v00 − b∗ + γ K + 1 i+1 K + 1 i+1 K 1 = (κ − κ )(v00 − b∗) − γ κ + κ K + 1 i i+1 i K + 1 i+1 1 ∆ ≥ ∆ v − 2γκ 2 κ 2 1 > 0, where the last step follows because γ < ∆κ∆v . 8κ1 Case 2 : b∗ > b(i+K+1) + γ. Because v0 does not want to reduce his bid to b∗ − γ,

"K−1 # 1 X [κ (v0 − b∗)] + κ v0 − b(i+K+1) ≥ κ v0 − b(i+K+1) K + 1 i+k i+K i+K k=0 0 ∗ 0 (i+K+1) 0 (i+K+1) =⇒ Kκi(v − b ) + κi+K v − b ≥ (K + 1)κi+K v − b

0 ∗ 0 (i+K+1) =⇒ κi(v − b ) ≥ κi+K v − b (7)

As a result we conclude

"K−1 # 1 X κ (v0 − b∗) ≥ [κ (v0 − b∗)] + κ v0 − b(i+K+1) . i K + 1 i+k i+K k=0

We next show that the v00 bidder would proﬁt by increasing his bid to

27 00 ∆v 00 ∗ ∆v By assumption, Γ ≥ v and γ < 2 . We have also seen that v ≥ b + 2 . Therefore ∗ 00 ∆v 00 ∗ b + γ ≤ v − 2 + γ ≤ v ≤ Γ, and so b + γ is an allowable bid.

45 b∗ + γ.28 The incremental proﬁt is at least

"K−1 # 1 X κ v00 − b∗ − γ − [κ (v00 − b∗)] + κ v00 − b(i+K+1) i K + 1 i+k i+K k=0 " K # 1 X = κ (v00 − v0) − κ (v00 − v0) − κ γ i K + 1 i+k i k=0 "K−1 # 1 X + κ (v0 − b∗) − [κ (v0 − b∗)] + κ v0 − b(i+K+1) i K + 1 i+k i+K k=0 1 K ≥ κ (v00 − v0) − κ (v00 − v0) − κ (v00 − v0) − κ γ i K + 1 i K + 1 i+1 i K ≥ (κ − κ )(v00 − v0) − κ γ K + 1 i i+1 i 1 ≥ ∆ ∆ − κ γ 2 κ v 1 > 0, where the last step follows from γ < ∆κ∆v . 8κ1 Lemma 29. For almost every γ > 0, in the GSP auction with bid spaces Bγ it is the case that for every bidder n and any bid proﬁle b−n, it is the case 0 00 0 00 that if bn, bn ∈ BRn(b−n), then (bn, b−n) and (bn, b−n) generate the same lottery over allocations.

Proof of Lemma 29. There are only countably many numbers of the form PI v i=1 qiκi I I PI , where {qi}i=1 ∈ Q, {zi}i=1 ∈ Z, and v ∈ {v1, . . . , vN }. Therefore, i=1 qiziκi almost every γ > 0 cannot be expressed in this form. We prove the statement for those values of γ. I Given any n and any fixed b−n, there are integers {zi}i=1 ∈ Z such that the set of expected payoffs obtainable by bidder n, given b−n, lie in the linear space over Q spanned by the set {κ1(vn − γz1), . . . , κI (vn − γzI )}. If bidder n is to be indifferent between two different lotteries over allocations that he may receive, then this set must be linearly dependent over Q. Equivalently, 28We must show that b∗ + γ is an allowable bid. To see this, notice that we must have v0 ≥ b(i+K+1), or else the v0 bidder would be better off by bidding zero. Equation (7) 0 ∗ 00 0 ∆v then implies that v ≥ b . By assumption, Γ ≥ v ≥ v + ∆v and γ < 2 . Consequently, ∗ 0 ∗ b + γ ≤ v + γ ≤ Γ − ∆v + γ ≤ Γ, and so b + γ is an allowable bid.

46 I there must exist {qi}i=1 ∈ Q, not all zero, such that

I X qiκi(vn − γzi) = 0 i=1 I I X X =⇒ γ qiziκi = vn qiκi (8) i=1 i=1

I Because vn > 0, {qi}i=1 ∈ Q are not all zero, and by Assumption 1(i), {κ1, . . . , κN } is linearly independent over Q, the right hand side of (8) is PI nonzero. We must then have i=1 qiziκi 6= 0, and so we can write

v PI q κ γ = n i=1 i i , PI i=1 qiziκi which is a contradiction to the earlier assumption on γ. Therefore, if 0 00 0 00 bn, bn ∈ BRn(b−n), then (bn, b−n) and (bn, b−n) generate the same lottery over allocations for bidder n. They therefore also generate the same lottery over allocations for all bidders. h n o Lemma 30. For every γ ∈ 0, min ∆v , ∆κ∆v and every δ ∈ [0, ∆ ∆ ), 2 8κ1 κ v every δ-locally envy-free equilibrium of the GSP auction with bid spaces Bγ is efficient. h n o Proof of Lemma 30. Let γ ∈ 0, min ∆v , ∆κ∆v and δ ∈ [0, ∆ ∆ ). 2 8κ1 κ v Let b be a δ-locally envy-free equilibrium. By Lemma 28, for all i ∈ {1,..., min{I,N − 1}}, b(i) > b(i+1), and so G(i) is a singleton with element g(i). To demonstrate efficiency, it will suffice to show that vg(i) ≥ vg(i+1). By δ-locally envy-freeness,

(i+1) (i+2) κi vg(i+1) − b − κi+1 vg(i+1) − b ≤ δ.

Additionally, because g(i) does not wish to deviate to b(i+1),

(i+1) (i+2) κi vg(i) − b − κi+1 vg(i) − b ≥ 0.

Subtracting, (κi − κi+1) vg(i+1) − vg(i) ≤ δ.

If vg(i) < vg(i+1), then the left-hand side is greater than or equal to ∆v∆κ, while the right-hand side is less than the same quantity, a contradiction.

47 A.4.2 Main Results n o Proof of Theorem 11. Letγ ¯ = min ∆v , ∆κ∆v . Let γ ∈ (0, γ¯) be such 2 8κ1 that the conclusion of Lemma 29 holds. Suppose b is a pure equilibrium of the GSP auction with bid spaces Bγ that fails the test-set condition. We argue that b is not extended proper. By Lemmas 28 and 29, we can characterize the best response sets of the bidders as follows. Bidder g(1)’s best response set consists of all feasible bids exceeding b(2). For i ∈ {2,..., min{I,N −1}}, bidder g(i)’s best response set consists of all feasible bids in the interval b(i+1), b(i−1). For i ∈ {min{I + 1,N},...,N}, the best response set of any bidder g(i) ∈ G(i) consists of all feasible bids below b(min{I,N−1}). Because b fails the test-set condition, there exists some bidder, whom we denote g(i∗), ˆ (i∗) for whom some alternate bid b weakly dominates b against Tg(i∗)(b). Ob- ˆ ∗ ˆ viously, b is a best response for g(i ) and also b 6= bg(i∗). We then consider three cases.

Case 1: ˆb > b(i∗). It must then be the case that g(i∗) prefers winning position min{i∗ − 1,I} at price b(min{i∗,I+1}) + γ to winning position i∗ at price b(i∗+1). We compare whether bidder g(i∗) is better off by playing b(i∗) or b(min{i∗,I+1}) + γ against two classes of bid profiles of his opponents: (i) First, suppose that bidder g(min{i∗ −1,I}) plays b(min{i∗,I+1}) +γ, and no other opponents deviate. If g(i∗) bids b(i∗), then he wins position i∗ at price b(i∗+1). If g(i∗) bids b(min{i∗,I+1})+γ, then he either obtains the former outcome or wins position min{i∗−1,I} at price b(min{i∗,I+1})+γ, each with probability one half. Therefore b(min{i∗,I+1}) + γ achieves strictly higher expected utility than b(i∗) in this case. (ii) Second, suppose that bidder g(min{i∗ − 1,I}) plays a best response other than b(min{i∗,I+1}) + γ, and all other bidders play best responses. We show that b(min{i∗,I+1}) + γ and b(i∗) achieve the same expected utility in this case. It is not a best response for any bidder g(i), i < min{i∗ − 1,I} to bid b(min{i∗,I+1}) + γ or lower. Similarly, it is not a best response for g(min{i∗−1,I}) to bid b(min{i∗,I+1}) or lower. Thus, in this case, min{i∗ − 1,I} bidders will bid above b(min{i∗,I+1}) + γ. In addition, if I > i∗ − 1, then we also know that it is not a best response for any bidder g(i), i > i∗ to bid b(i∗) or higher, so that N − i∗ bidders will bid below b(i∗). Consequently, g(i∗) will receive the same profit whether he bids b(min{i∗,I+1}) + γ or b(i∗) in this case. Moreover, b(min{i∗,I+1}) + γ is a best response for bidder g(min{i∗ − 1,I}). Therefore, by Proposition 7, b is not an extended proper equilibrium.

48 Case 2: ˆb < b(i∗) and i∗ < I. It must then be the case that g(i∗) prefers winning position i∗ +1 at price b(i∗+2) to winning position i∗ at price b(i∗) −γ. We compare whether bidder g(i∗) is better oﬀ by playing b(i∗) or b(i∗) − γ against two classes of bid proﬁles of his opponents: (i) First, suppose that bidder g(i∗ + 1) plays b(i∗) − γ, and no other opponents deviate. If g(i∗) bids b(i∗), then he wins position i∗ at price b(i∗) − γ. If g(i∗) bids b(i∗) − γ, then he either obtains the former outcome or wins position i∗ + 1 at price b(i∗+2), each with probability one half. Therefore b(i∗) − γ achieves strictly higher expected utility than b(i∗) in this case.

(ii) Second, suppose that bidder g(i∗ +1) plays a best response other than b(i∗) − γ, and all other bidders play best responses. We show that b(i∗) − γ and b(i∗) achieve the same expected utility in this case. It is not a best response for any bidder g(i), i < i∗ to bid b(i∗) or lower. It is not a best response for any bidder g(i), i > i∗ + 1 to bid b(i∗) − γ or higher. Similarly, it is not a best response for g(i∗ + 1) to bid b(i∗) or higher. Thus, in this case, i∗ − 1 bidders will bid above b(i∗), and N − i∗ bidders will bid below b(i∗) − γ. Consequently, g(i∗) will receive the same proﬁt whether he bids b(i∗) − γ or b(i∗) in this case. Moreover, b(i∗) − γ is a best response for bidder g(i∗ + 1). Therefore, by Proposition 7, b cannot be an extended proper equilibrium.

Case 3: ˆb < b(i∗) and i∗ ≥ I. It must then be the case that g(i∗) prefers winning position i∗ +1 at price b(i∗+2) to winning position i∗ at price b(i∗) −γ. This implies that i∗ = I and i∗ < N. We compare whether bidder g(i∗) is better oﬀ by playing b(i∗) or b(i∗) − γ against two classes of bid proﬁles of his opponents: (i) First, suppose that some bidder g(i), i > i∗ plays b(i∗) −γ, and all other bidders play best responses. If g(i∗) bids b(i∗), then he wins position i∗ at price b(i∗) − γ. If g(i∗) bids b(i∗) − γ, then he either obtains the 1 former outcome with probability K+1 or fails to win a position (since ∗ K i = I), with probability K+1 , where K is the number of bidders g(i), i > i∗ who play b(i∗) − γ. Therefore b(i∗) − γ achieves strictly higher expected utility than b(i∗) in this case.

(ii) Second, suppose that all bidders play best responses, where no bidders g(i), i > i∗ play b(i∗) − γ. We show that b(i∗) − γ and b(i∗) achieve the same expected utility in this case. It is not a best response for any

49 bidder g(i), i < i∗ to bid b(i∗) or lower. It is not a best response for any bidder g(i), i > i∗ + 1 to bid b(i∗) or higher. Thus, in this case, i∗ − 1 bidders will bid above b(i∗), and N − i∗ bidders will bid below b(i∗) − γ. Consequently, g(i∗) will receive the same proﬁt whether he bids b(i∗) − γ or b(i∗).

Moreover, b(i∗) − γ is a best response for bidder g(i∗ + 1). Therefore, by Proposition 7, b cannot be an extended proper equilibrium. n o Proof of Theorem 12. Fix δ > 0. Letγ ¯ = min ∆v , ∆κ∆v , δ . Let 2 8κ1 κ1 γ ∈ (0, γ¯) be such that the conclusions of Lemma 29 holds. We prove the theorem for this value of γ. That it also applies for γ = 0 can be proven using similar methods. Suppose that b is a pure extended proper equilibrium of the GSP auction with bid spaces Bγ that is not δ-locally envy-free. By Lemmas 28 and 29, we can characterize the best response sets of the bidders as follows. Bidder g(1)’s best response set consists of all feasible bids exceeding b(2). For i ∈ {2,..., min{I,N −1}}, bidder g(i)’s best response set consists of all feasible bids in the interval b(i+1), b(i−1). For i ∈ {min{I + 1,N},...,N}, the best response set of any bidder g(i) ∈ G(i) consists of all feasible bids below b(min{I,N−1}). Let i∗ be an index for which the locally envy-free inequality is violated by more than δ, so that for some member of G(i∗), whom we denote g(i∗),

h (i∗)i h (i∗+1)i κi∗−1 vg(i∗) − b − κi∗ vg(i∗) − b > δ.

Henceforth, for notational simplicity, we use b∗ to denote b(i∗). Note that the above inequality requires i∗ ≤ I + 1, which means that b∗ + γ is a best response for bidder g(i∗ −1). We now compare whether bidder g(i∗) is better oﬀ by playing b∗ or b∗ +γ against two classes of bid proﬁles by his opponents:

(i) First, suppose that bidder g(i∗ − 1) plays b∗ + γ, and no other opponents deviate. In this case, bidder g(i∗) would earn the payoff (i∗+1) ∗ κi∗ vg(i∗) − b from playing b , and he would earn the payoff 1 ∗ 1 (i∗+1) ∗ 2 κi∗−1 vg(i∗) − b − γ + 2 κi∗ vg(i∗) − b from playing b + γ. Therefore the incremental payoff to g(i∗) of playing b∗ + γ over b∗

50 in this case is

1 ∗ 1 h (i∗+1)i 1 κ ∗ v ∗ − b − κ ∗ v ∗ − b − κ ∗ γ 2 i −1 g(i ) 2 i g(i ) 2 i −1 1 1 > δ − γκ 2 2 1 > 0

(ii) Second, suppose that bidder g(i∗ −1) plays a best response other than b∗ + γ, and no other bidders deviate. In this case, g(i∗) will earn the same proﬁt whether he bids b∗ or b∗ + γ.

(iii) Third, suppose that one opponent other than g(i∗ − 1) plays a best response other than his equilibrium bid, and that no other bidders deviate. In this case, g(i∗) will also earn the same proﬁt whether he bids b∗ or b∗ + γ.

∗ ∗ Therefore, b + γ weakly dominates b against Tg(i∗)(b), and so b is not a test-set equilibrium. n o Proof of Proposition 14. Letγ ¯ = min ∆v , ∆κ∆v . Let γ ∈ [0, γ¯) be 2 8κ1 such that the conclusion of Theorem 12 applies. Let b be a test-set equilibrium of the GSP auction with bid spaces Bγ. By Lemma 28, for all i ∈ {1,..., min{I,N − 1}}, b(i) > b(i+1) and so G(i) is a singleton with element g(i). To demonstrate eﬃciency, it will suﬃce to show that vg(i) ≥ vg(i+1). ∆κ∆v By Theorem 12, b is a 8 -locally envy free equilibrium. Therefore, ∆ ∆ κ v − b(i+1) − κ v − b(i+2) ≤ κ v . i g(i+1) i+1 g(i+1) 8

Additionally, because g(i) does not wish to deviate to b(i+1),

(i+1) (i+2) κi vg(i) − b − κi+1 vg(i) − b ≥ 0.

Subtracting, ∆ ∆ (κ − κ ) v − v ≤ κ v . i i+1 g(i+1) g(i) 8

∆κ∆v If vg(i) < vg(i+1), then this would imply 8 ≥ ∆v∆κ, a contradiction.

51 Proof of Proposition 16. Not locally envy-free. Given the parameter val- 29 ues, it is straightforward to verify that b1 > b2 > b3. We can also bound the following quantities: κ3 1 κ1(v2 − b2) ≥ 100(10 − b2) > 100 10 · − γ ≥ 100 10 · − 1 = 150 κ2 4 κ3 2 112 κ2(v2 − b3) ≤ 4(10 − b3) < 4 5 + 5 · + γ ≤ 4 5 + 5 · + 1 = κ2 3 3 338 Therefore, κ1(v2 −b2)−κ2(v2 −b3) > 3 , and so bidder 2’s locally envy-free 338 inequality is violated by at least 3 . Properness. For ease of notation, we refer to the equilibrium bid proﬁle ∗ ∗ ∗ as (b1, b2, b3) in this section of the proof. Let γ ∈ (0, 1] be such that the γ l Γ m conclusions of Lemma 29 hold. Let M = |B | = γ + 1. For all ∈ (0, 1), deﬁne the following sets  ∗ ∗  σ3(b3) ≤ · σ3(b3) if b3 6= b3 ∗  γ 0 ∗ 0 ∗  ∆3, = σ3 ∈ ∆(B ) σ3(b3) ≤ · σ3(b3) if b3 ∈/ (0, b2) and b3 ∈ (0, b2)  M  σ3(b3) ≥ 2M  ∗ ∗   σ2(b2) ≤ · σ2(b2) if b2 6= b2   0 ∗ ∗ 0 ∗ ∗  ∗  γ σ2(b2) ≤ · σ2(b2) if b2 ∈/ (b3, b1) and b2 ∈ (b3, b1)  ∆2, = σ2 ∈ ∆(B ) M ∗  σ2(b2) ≤ 2M if b2 6= b2   2M   σ2(b2) ≥ 2M   ∗ ∗   σ1(b1) ≤ · σ1(b1) if b1 6= b1  ∗  γ 2M ∗  ∆1, = σ1 ∈ ∆(B ) σ1(b1) ≤ 2M if b1 6= b1  3M   σ1(b1) ≥ 2M 

29 It is immediate that b2 > b3. To see that b1 > b2: κ2 b1 − b2 ≥ 15 − (15 − b2) − b2 κ1 κ2 κ2 = 15 · 1 − + b2 · − 1 κ1 κ1 κ κ κ ≥ 15 · 1 − 2 + 10 1 − 3 + γ · 2 − 1 κ1 κ2 κ1 κ2 1 1 = 5 − γ − (5 − γ) + 10κ3 − κ1 κ2 κ1 4 ≥ 5 − 1 − · 5 100 > 0.

52 ∗ 0 γ For all n and all , ∆n, is a nonempty and compact subset of ∆ (B ). Let Vn(bn|σ) = πn(bn, σ−n) and deﬁne for each player the correspondence 3 ∗ ∗ Fn : ×m=1 ∆m, ⇒ ∆n, by

 0  if Vn(bn|σ) < Vn(bn|σ)  ∗ ∗ ∗ ∗ 0  Fn(σ) = σn ∈ ∆n, then σn(bn) ≤ · σn(bn),  0 γ  ∀bn, bn ∈ B

3 ∗ For all n and for any σ ∈ ×m=1 ∆m,, the points in Fn(σ) are those that satisfy a ﬁnite collection of linear inequalities, so Fn(σ) is a closed, convex set. We next demonstrate that for suﬃciently small values of , Fn(σ) is 0 γ nonempty. Let φn(bn) be the number of bids bn ∈ B such that Vn(bn|σ) < 0 0 γ Vn(bn|σ), and let Pn = |{bn ∈ B : φn(bn) = 0}|. Then let

φ (b )+(3−n)M ( n n if φ (b ) > 0 ∗ 2M n n σn(bn) = 1 h P φn(bn)+(3−n)M i 0 1 − 0 0 if φn(bn) = 0 Pn bn:φn(bn)>0 2M It is straightforward (but tedious) to show that if > 0 is sufficiently 3 ∗ ∗ small, then for all σ ∈ ×m=1 ∆m,, we have the following: (i) V3(b3|σ) > ∗ 0 0 ∗ ∗ V3(b3|σ) ∀b3 6= b3,(ii) V3(b3|σ) > V3(b3|σ) ∀b3 ∈ (0, b2), b3 ∈/ (0, b2), ∗ ∗ 0 0 ∗ ∗ (iii) V2(b2|σ) > V2(b2|σ) ∀b2 6= b2,(iv) V2(b2|σ) > V2(b2|σ) ∀b2 ∈ (b3, b1), b2 ∈/ ∗ ∗ ∗ ∗ (b3, b1), and (v) V1(b1|σ) > V1(b1|σ) ∀b1 6= b1. As a consequence of these ob- ∗ ∗ servations, for all > 0 sufficiently small, the σn defined above lies in ∆n,. 0 ∗ ∗ 0 Moreover, by construction, if Vn(bn|σ) < Vn(bn|σ), then σn(bn) ≤ · σn(bn). ∗ Consequently, σn ∈ Fn(σ), which demonstrates that Fn(σ) is nonempty, as desired. 3 Let F (·) = ×n=1 Fn(·). Then F satisfies all the conditions of the Kaku- tani Fixed Point Theorem (Kakutani, 1941), so there exists some σ ∈ 3 ∗ ×n=1 ∆n, such that σ ∈ F (σ ). By the definition of F , this point is an -proper equilibrium. So for all > 0 sufficiently small, there exists an -proper equilibrium in 3 ∗ ∗ ∗ σ ∈ ×n=1 ∆n,. By virtue of the fact that σn ∈ ∆n,, we must have σn(bn) ∈ h 1 i ∗ 1+(M−1) , 1 , and so lim→0 σn(bn) = 1. Consequently, this sequence of - ∗ ∗ ∗ proper equilibria converges to (b1, b2, b3) as → 0, which establishes that ∗ ∗ ∗ (b1, b2, b3) is proper. 1 Proof of Proposition 17. Let γ ∈ 0, 8 be such that the conclusions of Theorem 11, Proposition 14, and Lemma 30 hold. δ-locally envy-free equilibrium. Suppose b is a δ-locally envy-free equilibrium of the GSP auction with bid spaces Bγ for some δ ∈ [0, 1). We must

53 γ γ obviously have (b1, b2) ∈ B × B . That b1 > b2 follows from Lemma 30. Next, suppose that b2 > 2. Then bidder 1’s payoff is κ1(2 − b2) < 0, so deviating to ˆb = 0 would be profitable. Finally, suppose that b < 1 − δ . 1 2 κ1 Then bidder 2 envies bidder 1 by κ1(v1 − b2) = κ1(1 − b2) > δ, contradicting δ-local envy-freeness. Therefore b ∈ ELEF. Now suppose that b ∈ ELEF. It is easily checked that b is a Nash equilibrium. The amount by which bidder 2 (who wins the second slot) envies bidder 1 (who wins the first slot) is κ1(v1 − b2) = κ1(1 − b2) ≤ δ. Hence, b is δ-locally envy-free.

Test-set equilibrium. Suppose that b is a pure test-set equilibrium of the γ γ γ GSP auction with bid spaces B . We must obviously have (b1, b2) ∈ B ×B . That b1 > b2 follows from Proposition 14. We must have b2 ≤ 2 for the same l 2 m reasons as above. We cannot have b1 > γ γ , because such a bid is weakly ˆ j 2 k j 1 k dominated by b1 = γ γ against T1(b). We also cannot have b2 < γ γ , ˆ l 1 m because such a bid is weakly dominated by b2 = γ γ against T1(b). For the reverse direction, it is easily checked that any b ∈ ETS is a test-set equilibrium.

Extended proper equilibrium. Suppose that b is a pure extended proper equilibrium of the GSP auction with bid spaces Bγ. By Theorem 11, b ∈ l 2 m ETS. We begin by deriving a contradiction with b1 < γ γ . First, we can- j 2 k ˆ j 2 k not have b1 < γ γ because it is weakly dominated by b1 = γ γ (extended proper equilibria are trembling-hand perfect, which in two player games cannot involve weakly dominated strategies.) Second, suppose by way of con- j 2 k ˆ l 2 m tradiction that b1 = γ γ . Consider b1 = γ γ . Both perform equally well 0 j 2 k 00 l 2 m against all actions of bidder 2 except b2 = γ γ and b2 = γ γ . Against the former, ˆb1 outperforms b1, but against the latter, b1 outperforms ˆb1. Notice 0 1 j 2 k 00 j 2 k that u2(b1, b2) = 2 κ1 1 − γ γ and u2(b1, b2) = κ1 1 − γ γ . Since 00 0 γ < 1, u2(b1, b2) < u2(b1, b2). Because b is extended proper, there exists, by Proposition 5, an LPS ρ on Bγ × Bγ that satisﬁes strong independence, has full support, respects within-and-across-person preferences, and for which 00 0 (ρ, b) is a lexicographic Nash equilibrium. Since u2(b1, b2) < u2(b1, b2), we 0 00 ˆ must have b2 >ρ b2. Therefore b1 lexicographically outperforms b1, and so b1 ∈/ BR1(ρ−1), which constitutes a contradiction to (ρ, b) being a lexicographic Nash equilibrium. The only remaining possibility is therefore

54 l 2 m j 1 k b1 = γ γ . A similar argument shows that we must have b2 = γ γ . l 2 m j 1 k For the reverse direction, we must show that (b1, b2) = γ γ , γ γ is an extended proper equilibrium. This can be shown using methods similar to those in the proof of Proposition 16.

5 Proof of Proposition 18. Let γ ∈ 0, 808 be such that the conclusions of Lemma 29 and Proposition 14 hold. Suppose that (b1, b2, b3) is a pure test- set equilibrium of the GSP auction with bid spaces Bγ. By Proposition 14, b1 > b2 > b3. By Lemma 29, the best responses of bidder 1 are all feasible bids greater than b2, and the best responses of bidder 3 are all feasible bids less than b2. We compare the performance of b2 + γ to that of b2 against the elements of T2(b). They perform equally well against all elements of T2(b), with the exception of when bidder 1 deviates to b2 + γ. In this case, b2 generates the payoff κ2(v2 − b3). On the other hand, b2 + γ generates the payoff 1 1 2 κ2(v2 − b3) + 2 κ1[v2 − (b2 + γ)]. Since b is a test-set equilibrium, b2 + γ must not generate a higher expected utility than b2. Thus, 1 0 ≥ [κ (v − b − γ) − κ (v − b )] 2 1 2 2 2 2 3 1 5 ≥ 100 10 − b − − 40 , 2 2 808 which implies b2 ≥ 9.59. We next compare the performance of b2 − γ to that of b2 against the elements of T2(b). They perform equally well against all elements of T2(b), with the exception of when bidder 3 deviates to b2 − γ. In this case, b2 generates the payoff κ2[v2 − (b2 − γ)]. On the other hand, b2 − γ generates 1 1 the payoff 2 κ2[v2 − (b2 − γ)] + 2 κ3v2. Since b is a test-set equilibrium, b2 − γ must not generate a higher expected utility than b2. Thus, 1 0 ≥ [κ v − κ (v − b + γ)] 2 3 2 2 2 2 1 5 ≥ 10 − 4 10 − b + 2 2 808 which implies b2 ≤ 7.51, a contradiction.

A.5 First Price Menu Auction A.5.1 Technical Lemmas The following three lemmas will be used to prove Theorem 22.

55 Lemma 31. For all γ ≥ 0, the menu auction with bid spaces Bγ possess the 0 0 00 00 following property. Suppose that ˆbn(x ) = bn(x ) and ˆbn(x ) ≤ bn(x ), where 0 00 0 00 x 6= x . If x(bn, b−n) = x , then x(ˆbn, b−n) 6= x .

0 00 Proof of Lemma 31. Suppose that x(bn, b−n) = x and x(ˆbn, b−n) = x . These imply

N N 0 X 0 00 X 00 v0(x ) + bm(x ) ≥ v0(x ) + bm(x ) m=1 m=1 00 00 X 00 0 0 X 0 v0(x ) + ˆbn(x ) + bm(x ) ≥ v0(x ) + ˆbn(x ) + bm(x ), m6=n m6=n and because x0 6= x00, consistent tie-breaking by the auctioneer requires that at least one of these be strict. Subtracting the second inequality from the 0 0 00 00 0 ﬁrst yields bn(x ) − ˆbn(x ) > bn(x ) − ˆbn(x ). However, because bn(x ) = 0 00 00 ˆbn(x ) and bn(x ) ≥ ˆbn(x ), this is a contradiction. Lemma 32. For almost all γ > 0, the menu auction with bid spaces Bγ possesses the following properties.

∗ ∗ ∗ (i) If x(bn, b−n) = x and ˆbn(x ) 6= bn(x ), then π(bn, b−n) 6= π(ˆbn, b−n).

(ii) (Non-bossiness). If π(bn, b−n) 6= π(ˆbn, b−n), then πn(bn, b−n) 6= πn(ˆbn, b−n). Proof of Lemma 32. By Assumption 2, ∀n, ∀x, x0 ∈ X with x 6= x0 we 0 have vn(x) − vn(x ) 6= 0. Thus, for almost all γ > 0, it is the case that ∀n, 0 0 0 ∀x, x ∈ X with x 6= x we have vn(x) − vn(x ) 6∈ γZ. Fix some such value of γ. ∗ ∗ ∗ Claim (i). Suppose x(bn, b−n) = x and ˆbn(x ) 6= bn(x ). We then ∗ consider two cases. First, suppose x(ˆbn, b−n) = x . Then

∗ ∗ ∗ ∗ πn(ˆbn, b−n) = vn(x ) − ˆbn(x ) 6= vn(x ) − bn(x ) = πn(bn, b−n).

∗ Second, suppose x(ˆbn, b−n) 6= x . By the assumption on γ, this then means πn(bn, b−n) 6= πn(ˆbn, b−n). Claim (ii). Suppose that πn(bn, b−n) = πn(ˆbn, b−n). By the assumption on γ, this then means x(bn, b−n) 6= x(ˆbn, b−n). Since the bids of all bidders m 6= n remain constant, we obtain πm(bn, b−n) 6= πm(ˆbn, b−n) ∀m 6= n. Thus, π(bn, b−n) = π(ˆbn, b−n). Lemma 33. For almost all γ > 0, the menu auction with bid spaces Bγ possesses the following properties. Suppose b is a Nash equilibrium in which

56 ∗ ˆ ˆ x(b) = x , Suppose bn is such that for some xˆ ∈ X, bn(ˆx) ∈ γZ∩[bn(ˆx), vn(ˆx)− ∗ ∗ vn(x ) + bn(x )] and for all x 6=x ˆ, ˆbn(x) = bn(x). Suppose also ˆb−n is such ∗ that x(bn, ˆb−n) = x . Then

(i) πn(ˆbn, ˆb−n) ≥ πn(bn, ˆb−n). ∗ (ii) If in addition x(ˆbn, ˆb−n) 6= x , then πn(ˆbn, ˆb−n) > πn(bn, ˆb−n). Proof of Lemma 33. By Assumption 2, ∀n, ∀x, x0 ∈ X with x 6= x0 we 0 have vn(x) − vn(x ) 6= 0. Thus, for almost all γ > 0, it is the case that ∀n, 0 0 0 ∀x, x ∈ X with x 6= x we have vn(x) − vn(x ) 6∈ γZ. Fix some such value of γ. ∗ ∗ Claim (i). By Lemma 31, x(ˆbn, ˆb−n) ∈ {x , xˆ}. If x(ˆbn, ˆb−n) = x , ∗ ∗ 30 then since ˆbn(x ) = bn(x ) it follows that πn(ˆbn, ˆb−n) = πn(bn, ˆb−n). On the other hand, if x(ˆbn, ˆb−n) =x ˆ, then πn(ˆbn, ˆb−n) = vn(ˆx) − ˆbn(ˆx) > ∗ ∗ vn(x ) − bn(x ) = πn(bn, ˆb−n). ∗ Claim (ii). Since x(ˆbn, ˆb−n) 6= x = x(bn, ˆb−n), the previous assumption on γ implies πn(ˆbn, ˆb−n) 6= πn(bn, ˆb−n). Together with part (i), this establishes the claim.

A.5.2 Main Results Proof of Lemma 19. We prove the result for γ > 0. That it is also true for γ = 0 can be proven using similar methods. Let b be a Nash equilibrium of the menu auction with bid spaces Bγ. Let x∗ = x(b). Necessity. Recall that between any two outcomes that give a bidder the same payoﬀ, we assume the bidder prefers the outcome with the higher total payoﬀ. Therefore, ∀n it must be the case that ˆbn ∈ BRn(b−n) if and only if ∗ ∗ ∗ both x(ˆbn, b−n) = x and ˆbn(x ) = bn(x ). Suppose there exists some n and somex ˆ ∈ X for which vn(ˆx) − bn(ˆx) > ∗ ∗ vn(x ) − bn(x ) + γ. Choose some m 6= n and let ( bn(x) if x 6=x ˆ ˆbn(x) = bn(x) + γ if x =x ˆ (b (x) if x 6=x ˆ ˆ m bm(x) = j 1 ∗ ∗ k bn(x) + γ · γ [v0(x ) + B(x ) − v0(x) − B(x)] if x =x ˆ ˆ It is easily checked that the following are true: (i) if bk ∈ BRk(b−k) ˆ ˆ ˆ ˆ ˆ for some k, then πn(bn, bk, b−nk) ≥ πn(bn, bk, b−nk), (ii) πn(bn, bm, b−nm) >

30 ∗ ˆ ∗ ∗ ˆ 0 ∗ Ifx ˆ = x , then we obtain bn(x ) = bn(x ) from bn(ˆx) ∈ [bn(ˆx), vn(x ) − vn(xn) + ∗ ∗ ˆ ∗ ∗ ˆ bn(xn)]. Ifx ˆ 6= x , then we obtain bn(x ) = bn(x ) from the deﬁnition of bn.

57 ˆ ˆ πn(bn, bm, b−nm), and (iii) bm ∈ BRm(b−m). This contradicts b = (b1, . . . , bN ) having been a test-set equilibrium.

Suﬃciency. Suppose that b is not a test-set equilibrium. Then there exists a bidder n, a bid ˆbn ∈ BRn(b−n), a bidder m 6= n, and a bid ˆbm ∈ BRm(b−m) for which πn(ˆbn, ˆbm, b−nm) > πn(bn, ˆbm, b−nm). Since ˆbn ∈ ∗ ∗ BRn(b−n), we must have ˆbn(x ) = bn(x ). Since ˆbm ∈ BRm(b−m), we must ∗ have x(bn, ˆbm, b−nm) = x . From the previous two conclusions, combined ∗ with πn(ˆbn, ˆbm, b−nm) 6= πn(bn, ˆbm, b−nm), we have x(ˆbn, ˆbm, b−nm) 6= x . Furthermore, lettingx ˆ = x(ˆbn, ˆbm, b−nm), we must have ˆbn(ˆx) > bn(ˆx), which implies ˆbn(ˆx) ≥ bn(ˆx) + γ. We then have

0 < πn(ˆbn, ˆbm, b−nm) − πn(bn, ˆbm, b−nm) ∗ ∗ = vn(ˆx) − ˆbn(ˆx) − [vn(x ) − bn(x )] ∗ ∗ ≤ vn(ˆx) − bn(ˆx) − γ − [vn(x ) − bn(x )]

The ﬁrst step is because πn(ˆbn, ˆbm, b−nm) > πn(bn, ˆbm, b−nm). The second step is by deﬁnition. The third step is because ˆbn(ˆx) ≥ bn(ˆx) + γ. We ∗ ∗ conclude that vn(ˆx) − bn(ˆx) > vn(x ) − bn(x ) + γ.

1 opt opt Proof of Theorem 20. Letγ ¯ = N minx6=xopt {[V (x )+v0(x )]−[V (x)+ v0(x)]}. By Assumption 2,γ ¯ > 0. Let γ ∈ [0, γ¯). Assume the auctioneer selects some x∗ 6= xopt. Then by Lemma 19, we have for all n ∈ {1,...,N},

opt opt ∗ ∗ bn(x ) ≥ vn(x ) − [vn(x ) − bn(x )] − γ

Summing, B(xopt) ≥ V (xopt) − V (x∗) + B(x∗) − Nγ so

opt opt opt opt ∗ ∗ ∗ ∗ v0(x )+B(x ) ≥ [V (x )+v0(x )]−[V (x )+v0(x )]+B(x )+v0(x )−Nγ

opt opt ∗ ∗ Since γ ∈ [0, γ¯), [V (x ) + v0(x )] − [V (x ) + v0(x )] > Nγ, so

opt opt ∗ ∗ v0(x ) + B(x ) > v0(x ) + B(x ), a contradiction. Next, we argue that any such equilibrium generates payoﬀs π ∈ Cγ. Let J ⊆ {1,...,N}. We know that

opt opt J J B(x ) + v0(x ) ≥ B(x ) + v0(x )

58 so

opt opt opt J J¯ J¯ BJ (x ) + BJ¯(x ) + v0(x ) ≥ BJ (x ) + BJ¯ x + v0 x

J¯ J¯ ≥ BJ¯ x + v0 x J¯ opt opt J¯ ¯ ≥ VJ¯ x − VJ¯(x ) + BJ¯(x ) + v0 x − |J|γ where this last inequality comes from the fact that by Lemma 19, ∀n ∈ J¯, J¯ J¯ opt opt bn x ≥ vn x − [vn(x ) − bn(x )] − γ. Rearranging, we see that the equilibrium yields payoﬀs

opt opt ΠJ = VJ (x ) − FJ (x ) opt opt h J¯ J¯i ¯ ≤ [V (x ) + v0(x )] − VJ¯ x + v0 x + |J|γ as desired.

Proof of Theorem 21. By an argument almost identical to that given in Bernheim and Whinston (1986, Theorem 2), b is a Nash equilibrium resulting in the decision xopt.31 Furthermore, ∀n ∈ {1,...,N} and ∀x ∈ X,

opt opt bn(x) ≥ vn(x) − πn = vn(x) − vn(x ) + bn(x ).

By logic similar to the proof of suﬃciency in Lemma 19, this condition implies that b is a test-set equilibrium.

δ Proof of Theorem 22. Fix δ > 0. Letγ ¯ = 2 . Let γ ∈ (0, γ¯) be such that Lemmas 32 and 33 apply. Suppose that b is an extended proper equilibrium of the menu auction with bid spaces Bγ that is not δ-bilaterally eﬃcient. Let x∗ = x(b) and π∗ = π(b). Because b is not δ-bilaterally eﬃcient, there exists a decisionx ˆ ∈ X such that, after possibly relabeling the bidders,

N N X ∗ ∗ ∗ X ∗ v0(ˆx) + v1(ˆx) + v2(ˆx) + bn(ˆx) ≥ v0(x ) + v1(x ) + v2(x ) + bn(x ) + δ. n=3 n=3

31Bernheim and Whinston (1986) include the decision as part of the solution concept, defining a Nash equilibrium as a bid profile and a decision. Because there may be multiple decisions that maximize the auctioneer’s total utility, this technique allows them to resolve the auctioneer’s indifference in whichever way is convenient. Our approach is in the same spirit. A Nash equilibrium, for our purposes, consists only of a bid profile. However, as previously stated, we resolve the auctioneer’s indifference in favor of the efficient outcome, xopt.

59 Because b is extended proper, there exists, by Proposition 5, an LPS ρ = 1 K N (p , . . . , p ) on ×n=1 Bn that satisﬁes strong independence, has full support, and respects within-and-across-person preferences, for which (ρ, b) is a lexicographic Nash equilibrium. For n ∈ {1, 2}, let ( bn(x) + γzn if x =x ˆ ˆbn(x) = bn(x) if x 6=x ˆ

j 1 ∗ ∗ k ˜ where zn = γ [vn(ˆx) − bn(ˆx) − vn(x ) + bn(x )] . Furthermore, let b be any “most probable outcome-changing deviation,” that is any strategy proﬁle for ∗ 0 ∗ 0 which (i) π(˜b) 6= π and (ii) if π(˜b ) 6= π then ˜b ≥ρ ˜b . Because ˜b 6= b, there must exist some bidder j for whom ˜bj 6= bj. (It is possible that j ∈ {1, 2}.) We establish the result by proving a series of claims. At several junctures, we use the properties of ρ that follow from Lemma 27 without directly invoking the lemma.

˜0 ˜ ˜0 ˜ ˜0 ˜ Claim: For all b−j ≥ρ b−j, πj(bj, b−j) ≥ πj(bj, b−j). Furthermore, πj(bj, b−j) > πj(˜bj, ˜b−j).

Proof of Claim. We begin by proving the first part of the claim. If it is the case that ˜b−j = b−j, then we need only show that πj(bj, b−j) ≥ πj(˜bj, b−j), which follows directly from the fact that b is an equilibrium. We therefore assume henceforth that ˜b−j 6= b−j. ∗ ∗ We claim that ˜bj(x ) = bj(x ). To see this, suppose otherwise. Then by Lemma 32(i), π(˜bj, b−j) 6= π(bj, b−j). This would imply (˜bj, b−j) ≤ρ ˜b, which contradicts ˜b−j 6= b−j. ˜0 ˜ ˜0 ˜ ˜0 ˜0 Suppose ∃b−j ≥ρ b−j for which πj(bj, b−j) < πj(bj, b−j). Because b−j ≥ρ ˜ ˜0 ˜ ˜ ˜ ˜0 ∗ b−j,(bj, b−j) ≥ρ (bj, b−j) >ρ b. Then by the definition of b, π(bj, b−j) = π , ˜0 ∗ ∗ ˜ ˜0 so that πj(bj, b−j) = vj(x ) − bj(x ). Letx ˜ = x(bj, b−j). We claim that ∗ ˜ ˜0 ∗ ˜ ∗ x˜ 6= x . To see this, suppose otherwise. Then πj(bj, b−j) = vj(x )−bj(x ) = ∗ ∗ ˜0 ˜ ˜0 vj(x ) − bj(x ), which implies the contradiction πj(bj, b−j) = πj(bj, b−j). ∗ ∗ The previous paragraph demonstrates both thatx ˜ 6= x and that ˜bj(x ) = ∗ bj(x ). Lemma 31 therefore implies that ˜bj(˜x) > bj(˜x). Furthermore, be- ˜0 ˜ ˜0 ˜ cause πj(bj, b−j) < πj(bj, b−j), it must be the case that bj(˜x) ≤ vj(˜x) − ∗ ∗ vj(x ) + bj(x ). Next, define ( ˜bj(x) if x =x ˜ ˆbj(x) = bj(x) if x 6=x ˜

60 Observe the following about ˆbj:

∗ ∗ • ˆbj(˜x) = ˜bj(˜x) ∈ [bj(˜x), vj(˜x) − vj(x ) + bj(x )]. ˜0 ∗ • As previously argued, x(bj, b−j) = x .

∗ ∗ ∗ • Since ˜bj(x ) = bj(x ), we obtain that ˜bj and ˆbj agree on x as well asx ˜. ˜ ˜0 ∗ ˆ ˜0 ∗ Because x(bj, b−j) =x ˜ 6= x , Lemma 31 implies that x(bj, b−j) 6= x .

ˆ ˜0 These three facts, together with Lemma 33(ii) demonstrate that πj(bj, b−j) > ˜0 ˜00 ˆ ˜00 ˜00 πj(bj, b−j). Now suppose that b−j were such that πj(bj, b−j) < πj(bj, b−j). ˜00 By Lemma 33(i) and the first of the above facts, we must have x(bj, b−j) 6= ∗ ˜ ˜ ˜00 ˜ x . Then by the definition of b, b ≥ρ (bj, b−j). Since bj >ρ bj, we must ˜ ˜00 ˜0 ˜00 then have b−j >ρ b−j, and so b−j >ρ b−j. This, together with the fact that ρ has full support, implies that ˆbj lexicographically outperforms bj, and so bj ∈/ BRj(ρ−j). This contradicts (ρ, b) being a lexicographic Nash equilibrium. To prove the second part of the claim, it suffices to show that πj(bj, ˜b−j) 6= ∗ πj(˜bj, ˜b−j). By the definition of ˜b, π(˜bj, ˜b−j) 6= π . On the other hand, be- ∗ cause (bj, ˜b−j) >ρ (˜bj, ˜b−j), the definition also implies π(bj, ˜b−j) = π . The result then follows from Lemma 32(ii).

˜0 ˜ ˜0 ˆ ˜0 Claim: For all b−2 ≥ρ b−3, π2(b2, b−2) ≤ π2(b2, b−2).

Proof of Claim. Observe the following about ˆb2: ˜0 ˜ ˜ ˜0 ˜ • Because b−2 ≥ρ b−3 and b2 >ρ bj,(b2, b−2) >ρ b. Then by the deﬁni- ˜ ˜0 ∗ tion of b, x(b2, b−2) = x .

∗ ∗ • By the deﬁnition of ˆb2, ˆb2(ˆx) = b2(ˆx) + γz2 ≤ v2(ˆx) − v2(x ) + b2(x ).

• We claim that ˆb2(ˆx) ≥ b2(ˆx). By the definition of ˆb2, it will suffice ∗ ∗ to show b2(ˆx) − b2(x ) ≤ v2(ˆx) − v2(x ). Suppose it were instead the ∗ ∗ case that b2(ˆx) − b2(x ) > v2(ˆx) − v2(x ). By the definition of ˆb1, ∗ ∗ ˆb1(ˆx) − ˆb1(x ) ≥ v1(ˆx) − v1(x ) − γ. In addition we know

N ∗ ∗ ∗ X ∗ v0(ˆx)−v0(x )+v1(ˆx)−v1(x )+v2(ˆx)−v2(x )+ [bn(ˆx)−bn(x )] ≥ δ. n=3

61 Combining these three inequalities yields

N ∗ X ∗ v0(ˆx) − v0(x ) + ˆb1(ˆx) − ˆb1(ˆx) + [bn(ˆx) − bn(x )] > δ − γ. n=2

δ ˆ ∗ Since γ < 2 , the above inequality implies that x(b1, b−1) 6= x . Lemma 33(ii) then implies that π1(ˆb1, b−1) > π1(b1, b−1), which contradicts b being a Nash equilibrium.

The claim follows from the above three facts, in conjunction with Lemma 33(ii).

Claim: ˆb2 >ρ ˜bj.

Proof of Claim. This follows directly from the previous two claims and the fact that ρ respects within-and-across-person preferences. (This is the step where extended proper equilibrium, as opposed to merely proper equilibrium, is required.)

Claim: b1 ∈/ BR1(ρ−1).

Proof of Claim. Observe the following about ˆb1:

∗ ∗ • ˆb1(ˆx) ∈ [b1(ˆx), v1(ˆx) − v1(x ) + b1(x )] for the same reasons that ˆb2(ˆx) ∈ ∗ ∗ [b2(ˆx), v2(ˆx) − v2(x ) + b2(x )], which was established in the proof of an earlier claim.

∗ ∗ • By the deﬁnition of ˆb1 and ˆb2, ˆbn(ˆx) − ˆbn(x ) ≥ vn(ˆx) − vn(x ) − γ for n ∈ {1, 2}. In addition we know

N ∗ ∗ ∗ X ∗ v0(ˆx)−v0(x )+v1(ˆx)−v1(x )+v2(ˆx)−v2(x )+ [bn(ˆx)−bn(x )] ≥ δ. n=3 Combining these inequalities yields

N ∗ X ∗ v0(ˆx)−v0(x )+ˆb1(ˆx)−ˆb1(ˆx)+ˆb2(ˆx)−ˆb2(ˆx)+ [bn(ˆx)−bn(x )] ≥ δ−2γ. n=3

δ ˆ ˆ ∗ Since γ < 2 , the above inequality implies that x b1, b2, b−{1,2} 6= x .

62 ˆ ∗ • We claim that x(b1, b2, b−{1,2}) = x . To see why, suppose other- ˆ wise. Lemma 31 would then imply that x(b1, b2, b−{1,2}) =x ˆ. Because ∗ ∗ ˆb2(ˆx) ∈ [b2(ˆx), v2(ˆx) − v2(x ) + b2(x )], Lemma 33(ii) would then imply π2(ˆb2, b−2) > π2(b), which would contradict b being a Nash equilibrium. ˆ ˆ These facts, together with Lemma 33(ii) demonstrate that π1 b1, b2, b−{1,2} > ˆ ˜0 ˆ ˜0 π1 b1, b2, b−{1,2} . Now suppose that b−1 were such that π1(b1, b−1) < ˜0 π1(b1, b−1). By Lemma 33(i) and the ﬁrst of the above facts, we must ˜0 ∗ ˜ ˜ ˜0 have x(b1, b−1) 6= x . Then by the deﬁnition of b, b ≥ρ (b1, b−1). Con- ˜ ˜0 ˆ ˜ ˆ ˜ sequently b−1 ≥ρ b−1. Additionally, because b2 >ρ bj,(b2, b−{1,2}) >ρ b−1. ˆ ˜0 By transitivity, (b2, b−{1,2}) >ρ b−1. This, together with the fact that ρ has full support, implies that ˆb1 lexicographically outperforms b1, and so b1 ∈/ BR1(ρ−1).

The previous claim constitutes a contradiction to (b, ρ) being a lexicographic Nash equilibrium.

References

Bernheim, B. Douglas and Michael D. Whinston, “Menu Auctions, Resource Allocation, and Economic Inﬂuence,” The Quarterly Journal of Economics, 1986, 101 (1), 1–32.

Blume, Lawrence, Adam Brandenburger, and Eddie Dekel, “Lex- icographic Probabilities and Equilibrium Reﬁnements,” Econometrica, 1991, 59 (1), 81–98.

Day, Robert and Paul Milgrom, “Optimal Incentives in Core-Selecting Auctions,” in Nir Vulkan, Alvin E. Roth, and Zvika Neeman, eds., Hand- book of Market Design, Oxford University Press, 2013.

Edelman, Benjamin, Michael Ostrovsky, and Michael Schwarz, “In- ternet Advertising and the Generalized Second-Price Auction: Selling Bil- lions of Dollars Worth of Keywords,” American Economic Review, 2007, 97 (1), 242–259.

Kakutani, Shizuo, “A Generalization of Brouwer’s Fixed Point Theorem,” Duke Mathematical Journal, 1941, 8 (3), 416–427.

63 Monderer, Dov and Lloyd S. Shapley, “Potential Games,” Games and Economic Behavior, 1996, 14 (1), 124–143.

Myerson, Roger B., “Reﬁnements of the Nash Equilibrium Concept,” International Journal of Game Theory, 1978, 7 (2), 73–80.

Varian, Hal R., “Position Auctions,” International Journal of Industrial Organization, December 2007, 25 (6), 1163–1178.