Towards a Universal Test Suite for Combinatorial Auction Algorithms
Total Page：16
File Type：pdf, Size：1020Kb
Towards a Universal Test Suite for Combinatorial Auction Algorithms Kevin Leyton-Brown Mark Pearson Yoav Shoham Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science Stanford University Stanford University Stanford University Stanford, CA 94305 Stanford, CA 94305 Stanford, CA 94305 [email protected] [email protected] [email protected] ABSTRACT equipment, in which a bidder values a particular TV at x General combinatorial auctions—auctions in which bidders and a particular VCR at y but values the pair at z>x+ y. place unrestricted bids for bundles of goods—are the sub- An agent with substitutable valuations for two copies of the ject of increasing study. Much of this work has focused on same book might value either single copy at x, but value algorithms for finding an optimal or approximately optimal the bundle at z<2x. In the special case where z = x (the set of winning bids. Comparatively little attention has been agent values a second book at 0, having already bought a { } paid to methodical evaluation and comparison of these al- first) the agent may submit the set of bids bid1 XOR bid2 . gorithms. In particular, there has not been a systematic By default, we assume that any satisfiable sets of bids that discussion of appropriate data sets that can serve as uni- are not explicitly XOR’ed is a candidate for allocation. We versally accepted and well motivated benchmarks. In this call an auction in which all goods are distinguishable from paper we present a suite of distribution families for generat- each other a single-unit CA. In contrast, in a multi-unit CA ing realistic, economically motivated combinatorial bids in some of the goods are indistinguishable (e.g., many iden- five broad real-world domains. We hope that this work will tical TVs and VCRs) and bidders request some number of yield many comments, criticisms and extensions, bringing goods from each indistinguishable set. This paper is primar- the community closer to a universal combinatorial auction ily concerned with single-unit CA’s, since most research to test suite. date has been focused on this problem. However, when ap- propriate we will discuss ways that our distributions could be generalized to apply to multi-unit CA’s. 1. INTRODUCTION 1.1 Combinatorial Auctions 1.2 The Computational Combinatorial Auc- tion Problem Auctions are a popular way to allocate goods when the amount that bidders are willing to pay is either unknown or In a combinatorial auction, a seller is faced with a set unpredictably changeable over time. The rise of electronic of price offers for various bundles of goods, and his aim is commerce has facilitated the use of increasingly complex to allocate the goods in a way that maximizes his revenue. auction mechanisms, making it possible for auctions to be (For an overview of this problem, see [8].) This optimization applied to domains for which the more familiar mechanisms problem is intractable in the general case, even when each are inadequate. One such example is provided by combina- good has only a single unit. Because of the intractability of torial auctions (CA’s), multi-object auctions in which bids general CA’s, much research has focused on subcases of the name bundles of goods. These auctions are attractive be- CA problem that are tractable; see [22] and more recently cause they allow bidders to express complementarity and [25]. However, these subcases are very restrictive and there- substitutability relationships in their valuations for sets of fore are not applicable to many CA domains. Other research goods. Because CA’s allow bids for arbitrary bundles of attempts to define mechanisms within which general CA’s goods, an agent may offer a different price for some bundle will be tractable (achieved by various trade-offs including of goods than he offers for the sum of his bids for its disjoint bid withdrawal penalties, activity rules and possible ineffi- subsets; in the extreme case he may bid for a bundle with ciency). Milgrom [15] defines the Simultaneous Ascending the guarantee that he will not receive any of its subsets. An Auction mechanism which has been very influential, partic- example of complementarity is an auction of used electronic ularly in the recent FCC spectrum auctions. However, this approach has drawbacks, discussed for example in [6]. In the general case there is no substitute for a completely un- restricted CA. Consequently, many researchers have recently Permission to make digital or hard copies of all or part of this work for begun to propose algorithms for determining the winners of personal or classroom use is granted without fee provided that copies are a general CA, with encouraging results. This wave of re- not made or distributed for profit or commercial advantage and that copies search has given rise to a new problem, however. In order bear this notice and the full citation on the first page. To copy otherwise, to to test (and thus to improve) such algorithms, it has been republish, to post on servers or to redistribute to lists, requires prior specific necessary to use some sort of test suite. Since general CA’s permission and/or a fee. EC’00, October 17-20, 2000, Minneapolis, Minnesota. have never been widely held, there is no data recording the Copyright 2000 ACM 1-58113-272-7/00/0010 ..$5.00 bidding behavior of real bidders upon which such a test suite may be built. In the absence of such natural data, we are 2.3.1 Which goods left only with the option of generating artificial data that First, each of the distributions for generating test data is representative of the sort of scenarios one is likely to en- discussed above has the property that all bundles of the counter. The goal of this paper is to facilitate the creation same size are equally likely to be requested. This assumption of such a test suite. is clearly violated in almost any real-world auction: most of the time, certain goods will be more likely to appear together 2. PAST WORK ON TESTING CA than others. (Continuing our electronics example, TVs and VCRs will be requested together more often than TVs and ALGORITHMS printers.) 2.1 Experiments with Human Subjects 2.3.2 Number of goods One approach to experimental work on combinatorial auc- Likewise, each of the distributions for generating test data tions uses human subjects. These experiments assign valu- determines the number of goods in a bundle completely in- ation functions to subjects, then have them participate in dependently from determining which goods appear in the auctions using various mechanisms [3, 12, 7]. Such tests can bundle. While this assumption appears more reasonable it be useful for understanding how real people bid under differ- will still be violated in many domains, where the expected ent auction mechanisms; however, they are less suitable for length of a bundle will be related to which goods it contains. evaluating the mechanisms’ computational characteristics. (For example, people buying computers will tend to make In particular, this sort of test is only as good as the sub- long combinatorial bids, requesting monitors, printers, etc., jects’ valuation functions, which in the above papers were while people buying refrigerators will tend to make short hand-crafted. As a result, this technique does not easily bids.) permit arbitrary scaling of the problem size, a feature that is important for characterizing an algorithm’s performance. 2.3.3 Price In addition, this method relies on relatively naive subjects Next, there are problems with the pricing1 schemes used to behave rationally given their valuation functions, which by all four techniques. Pricing is especially crucial: if prices may be unreasonable when subjects are faced with complex are not chosen carefully then an otherwise hard distribution and unfamiliar mechanisms. can become computationally easy. In Sandholm [24] prices are drawn randomly from either 2.2 Particular Problems [0, 1] or from [0,g], where g is the number of goods requested. A parallel line of research has examined particular prob- The first method is clearly unreasonable (and computation- lems to which CA’s seem well suited. For example, re- ally trivial) since price is unrelated to the number of goods searchers have considered auctions for the right to use rail- in a bid—note that a bid for many goods and for a small road tracks [5], real estate [19], pollution rights [13], airport subset of the same bid will have exactly the same price on time slot allocation [21] and distributed scheduling of ma- expectation. The second is better, but has the disadvan- chine time [26]. Most of these papers do not suggest holding tage that average and range are parameterized by the same an unrestricted general CA, presumably because of the com- variable. putational obstacles. Instead, they tend to discuss alterna- In Boutilier et al.[4] prices of bids are distributed normally tive mechanisms that are tailored to the particular problem. with mean 16 and standard deviation 3, giving rise to the None of them proposes a method of generating test data, same problem as the [0, 1] case above. nor does any of them describe how the problem’s difficulty In Fujishima et al.[10] prices are drawn from [g(1−d),g(1+ scales with the number of bids and goods. However, they d)], d =0.5. While this scheme avoids the problems de- still remain useful to researchers interested in general CA’s scribed above, prices are simply additive in g and are unre- because they give specific descriptions of problem domains lated to which goods are requested in a bundle, both unre- to which CA’s may be applied.