Some Preliminary Results on Competition Between Markets for Automated Traders

Some Preliminary Results on Competition between Markets for Automated Traders Jinzhong Niu and Kai Cai Simon Parsons and Elizabeth Sklar Department of Computer Science Dept of Computer and Information Science Graduate Center Brooklyn College City University of New York City University of New York 365, 5th Avenue 2900 Bedford Avenue New York, NY 10016, USA Brooklyn, NY 11210, USA {jniu,kcai}@gc.cuny.edu {parsons,sklar}@sci.brooklyn.cuny.edu

Abstract trading strategy 3 ZI-C — which bids randomly but avoids making a loss — and showed that it generates high efficiency Real market institutions, stock and commodity exchanges for solutions (Gode & Sunder 1993). (Cliff & Bruten 1997) then example, do not occur in isolation. Company stock is frequently listed on several stock exchanges, and futures ex- provided an adaptive trading strategy called zero intelligence changes make it possible for dealers in a particular commod- plus (ZIP), and showed that it outperformed ZI-C, generating ity to offset their risks by trading options in that commod- high efficiency outcomes and converging to the equilibrium ity. While there has been extensive research into agent-based price. This led to the suggestion that ZIP embodies the mini- trading in individual markets, there is little work on agents mum intelligence required by traders. Subsequent work has that trade in multiple market scenarios. Our work seeks to led to the development of further trading strategies, includ- address this imbalance, providing an analysis of the behavior ing that proposed by (Roth & Erev 1995), and that suggested of trading agents that are free to move between a number of by (Gjerstad & Dickhaut 1998), commonly referred to as parallel markets, each of which has different properties. GD. This work on trading strategies is only one facet of the re- Introduction search on auctions. Gode and Sunder’s results suggest that The market mechanisms known as auctions, are widely used the structure of the auction mechanisms plays an important to solve real-world resource allocation problems, and in role in determining the outcome of an auction, and this is structuring stock or futures exchanges like the New York further bourne out by the work of (Walsh et al. 2002) (which Stock Exchange (NYSE) and the Chicago Mercantile Ex- also points out that results hinge on both auction design and change (MCE). When well designed (Klemperer 2002), auc- the mix of trading strategies used). For example, if an auctions achieve desired economic outcomes like high allocation is strategy-proof traders need not bother to conceal their tive efficiency whilst being easy to implement. Research private values, and in such auctions complex trading agents on auctions originally interested economists and mathemati- are not required. cians. They view auctions as games of incomplete informa- Despite the variety of this work, it has one common theme tion and have successfully applied traditional analytic meth- — it all studies single markets. In contrast, real market in- ods from game theory to some kinds of auctions (Maskin & stitutions, like the stock and commodity exchanges men- Riley 1985; Vickrey 1961). The high complexity of other tioned above, do not occur in isolation. Company stock is auction types, especially double-sided auctions (Friedman frequently listed on several stock exchanges. Indian compa- 1993) (DAs)1, however makes it difficult to go further in this nies, for example, can be listed on both the National Stock direction (Madhavan 1992; Satterthwaite & Williams 1993). Exchange (NSE) and the Bombay Stock Exchange (BSE) As a result, researchers turned to experimental ap- (Shah & Thomas 2000). US companies may be listed on proaches. For example, (Smith 1962) showed that in con- both the NYSE, NASDAQ and, in the case of larger firms, tinuous double auctions (CDAs)2, even a handful of human non-US markets like the London Stock Exchange (LSE). traders can lead to high overall efficiency, and transaction Such multiple markets for the same goods induce com- prices can quickly converge to the theoretical equilibrium. plex interactions. The simplest example of this is the work With real trade increasingly contracted by automated of arbitrageurs who exploit price differences between markets to buy low in one and sell high in another, thus evening “program traders”, experimental work has followed suit. 4 Gode and Sunder (1993) introduced the zero intelligence the prices between markets. More complex dynamics oc- Copyright c 2007, Association for the Advancement of Artificial 3In a recent paper (Sunder 2004), Sunder reveals that they came Intelligence (www.aaai.org). All rights reserved. up with these simple strategies in the face of demands from stu- 1In DAs, both competing sellers and buyers can make offers, dents whom they had challenged to create automated strategies, in contrast to the most common auction mechanisms, such as the saying that “Our motivation for the ZI-C strategy was part jest: it English auction, where only buyers bid. was sure to lose to the student strategies, but we could still save 2A CDA is a continuous DA in which any trader can accept an face with such an obviously simple and silly strategy”. offer and make a deal any time during the auction period. 4In addition, futures exchanges make it possible for dealers in

19 cur when markets compete, as when the NSE opened and areinthosemarkets. proceeded to claim much of the trade volume from the estab- We allow markets to levy charges on traders, as real lished BSE (Shah & Thomas 2000), or when the newly cre- markets do. In doing this, our work has a different focus ated Singapore International Monetary Exchange (SIMEX) from the other work on market mechanisms we have men- did the same to Japanese markets for index futures on Nikkei tioned. That work is focused on how the performance of 225 (Shah 1997) in the late 1980s. These changes took place traders helps achieve economic goals like high efficiency over a long period of time, but inter-market dynamics can (Gode & Sunder 1993) and trading near equilibrium (Cliff & have much shorter timescales, as was the case in the flow Bruten 1997), or how traders compete amongst themselves between the CME and the NYSE during the global stock mar- to achieve high profits (Tesauro & Das 2001). In contrast, ket crash of 1987 (Miller et al. 1988). This kind of interac- we are interested in competition between markets,andwhat tion between markets has not been widely studied, least of the movement of traders is when they are faced with a vari- all using automated traders. ety of markets. The work described in this paper starts to address this imbalance between experimental work and what happens in Experimental Setup the real world, providing an analysis of scenarios in which The experiments we carried out explore how traders move trading agents choose between a number of parallel mar- between markets of different properties and what effect their kets, while the markets simultaeously decide how to profit movement has on the profits of those markets. from the traders. In common with much work in computa- tional economics (Friedman 1998), the strategies used both Traders by traders to choose between markets, and markets to decide how to charge traders, are very simple — the idea is Our traders have two tasks. One is to decide how to make that using more sophisticated strategies might obscure our offers. The mechanism they use to do this is their trading view of what happening in the complex setting of double strategy. The other task is to choose market to make offers auction markets. in. The mechanism for doing this is their market selection strategy. The trading strategies are: Background • ZI-C: (Gode & Sunder 1993) which picks offers randomly but ensures the trader doesn’t make a loss. To experiment with multiple markets, we used a variant of the Java Auction Simulator API (JASA)5. JASA provides • GD: (Gjerstad & Dickhaut 1998) which estimates the the ability to run continuous double auctions populated by probability of an offer being accepted from the distribu- traders that use a variety of trading strategies, and has been tion of past offers, and chooses the offer which maximises used for a variety of work in analysing auctions, for example its expected utility. (Niu et al. 2006; Phelps et al. 2006). Auctions in JASA fol- The market selection strategies are: low the usual pattern for work on automated trading agents, • T running for a number of trading days, with each day being r: the trader randomly picks a market; and broken up into a series of rounds. A round is an opportunity • T: the trader treats the choice of market as an n-armed for agents to make offers to buy or sell6, and we distinguish bandit problem which it solves using an -greedy ex- different days because at the end of a day, agents have their ploration policy (Sutton & Barto 1998). A T trader inventories replenished. As a result, every buyer can buy chooses what it estimates to be the best market, in terms goods every day, and every seller can sell every day. Days of daily trading profit, with probability 1 − , and ran- are not identical because agents are aware of what happened domly chooses one of the remaining markets otherwise. the previous day. Thus it is possible for traders to learn, over may remain constant or be variable over time, depending the course of several days, the optimal way to trade. upon the value of the parameter α (Sutton & Barto 1998). We run a number of JASA markets simultaneously, allow- If α is 1, remains constant, while if α takes any value in ing traders to move between markets at the end of a day. (0, 1), will reduce over time. In practice this means that traders need a decision mecha- • Tτ : the trader uses the softmax exploration policy (Sutton nism that picks which market to trade in and we have im- & Barto 1998). A Tτ trader does not treat all markets plemented several — these are discussed below. Using this other than the best exactly the same. If it does not choose approach, agents are not only learning how best to make of- the best market, it weights the choice of remaining market fers, which they will have to do anew for each market, but so that it is more likely to choose better markets. The they are also learning which market is best for them. Of parameter τ in the softmax strategy controls the relative course, which market is best will depend partly on the prop- importance of the weights a trader assigns markets, and erties of different markets, but also on which other agents similarly to , it may be fixed or have a variable value that α a particular commodity to offset their risks by trading options — is controlled by . commitments to buy or sell at a future date at a certain price — in Thus all our traders use simple reinforcement learning to de- that commodity, and provide further opportunities for arbitrage. cide which market to trade in7, basing their choice on the 5http://sourceforge.net/projects/jasa/ 6Offers to buy are also called bids, and offers to sell are also 7Though we have results, not presented here, which suggest that called asks. Both are called shouts. more complex forms of reinforcement learning, like the Roth-Erev

20 expected profit suggested by prior experience, and making profit. In contrast, Figure 1(c) and 1(d), when traders pick no use of any other information that may be available about markets based on their personal profits, they move towards the markets. As mentioned above, we deliberately chose this the market with lowest fixed costs. While markets with high simple decision mechanism in order to make the comparison charges make initial windfall profits, the trend is for the between markets as clear as possible. lower charging market to gain greater cumulative profit as the number of trading days increases. Markets Figures 2(a)–2(d) show that these results are robust While we can set up markets to charge traders in a variety of against the ability of traders to make sensible trades since ways, we have concentrated on charging traders a proportion broadly the same results are observed when some or all of of the surplus on a transaction in which they are involved — the traders make their bidding decisions randomly. Fig- that is a proportion of the difference between what the buyer ures 3(a)–3(d) test the sensitivity of the results to the strategy bids and the seller asks. We focus on this because it mirrors that traders use to learn which market to choose. Decreasing the case of the competition between the NSE and the BSE over time (Figures 3(a) and 3(b)) does not seem to have (Shah & Thomas 2000) where the BSE, had a much higher much effect, but switching to the softmax strategy reduces charge on transactions than the new market. the attractiveness of the lowest charging market since traders We experimented with four basic charging mechanisms, can still make good profits in higher charging markets. one fixed and three simple adaptive mechanisms: Finally, for the experiments with only fixed charges, Fig- • ures 4(a)–4(d) show that the results obtained so far are very Fixed charging rates, typically 20%, 40%, 60% and 80% sensitive to the length of time agents have to learn about the of the surplus on a transaction. markets. When some traders start learning afresh every day, • Pricecutting (PC): since traders will, all else being equal, simulating traders leaving and entering the markets (4(c) and prefer markets with lower charges, a pricecutting market 4(d)), the lowest charging market might still capture most of will reduce its charge until it is 80% of the charge of the the traders, but it captures less of them, and the remaining lowest charging market. markets attract enough traders to have the same profit pro- • Bait and switch (B&S): the market cuts its charge until file as when there is no learning (Figures 1(a) and 1(b)). it captures 30% of the traders, then slowly increases its Thus, for the fixed charge markets, provided that there is charge (adjusting its charge downward again if its market no turnover of traders, it is a winning strategy to undercut share drops below 30%). the charges of the other markets. • Zero intelligence (ZIP): a version of the ZIP strategy for Homogeneous markets markets. The market adjusts its charge to be just lower Turning to the adaptive charging strategies, we first tested than that of the market that is the most profitable. If it is them against copies of themselves. In these experiments we the most profitable market, it raises its charges slightly. ran four copies of each kind of market against each other Again, our choice of market strategies was driven by the with different initial profit charges (20%, 40%, 60% and desire to first establish the relative performance of simple 80%). In each experiment we also provided a “null” market charging policies, and thus the basic structure of the problem which made no charges and executed no trades — the idea of competing markets, before trying more complex policies. of this is to allow traders who cannot trade profitably with a Each of the experiments is setup in the following way. mechanism for not trading — and, for completeness, carried The experiment is run for 200 or 400 trading days, with ev- out the same experiment with the fixed price markets. For ery day being split into 10 rounds, each of which is one sec- all of these experiments, and all subsequent experiments, we ond long. The markets are populated by 100 traders, evenly used traders that made bids using GD, selected markets using split between buyers and sellers. Each trader is permitted an -greedy policy, and continued learning for all 400 days. to buy or sell at most one unit of goods per day, and each The results of these experiments can be seen in Figures 5(a) trader has a private value for that good which is drawn from to 6(d). The fixed price markets, in Figures 5(a) and 5(b), a uniform distribution between $50 and $150. attract less traders in the presence of the null market, but make similar profits (since the traders who tend to the null Results market do not often trade). The price cutting markets, in Results are given in Figures 1(a) to 9(d). These show values Figures 5(c) and 5(d), get into a price war which, unlike the averaged over 100 runs of each experiment. myopic pricebots from (Greenwald & Kephart 1999), they do not have the intelligence to get out of, and the bait-and- Fixed charge markets switch markets (Figures 6(a) and 6(b)) are similarly unable to generate a significant profit. The ZIP markets, Figures 6(c) The first set of experiments explore the properties of markets and 6(d), adjusting their profit margins to fit what the traders with fixed charges. Figures 1(a) and 1(b) show that traders will allow, manage to do better, but generate nowhere near that pick markets randomly have no discernable pattern of as much profit as the fixed price markets do. movement between markets, just as we would expect. As a result, the market with the highest charges makes the most Heterogeneous markets approach (Roth & Erev 1995) do not perform significantly differ- While the homogenous market experiments give some idea ently. of market performance, it is more interesting to examine

21 Many Price Cutting Many Bait and Switch Many Zero Intelligence 1-Price Cutting Profit 0.8 – 84.1 6502.2 – 6043.6 stdev. 7.5 – 105.6 1527.1 – 2159.7 relationship < > 1-Bait and Switch Profit 82.0 – 0.7 6545.7 – 5743.8 stdev. 56.7 – 6.8 2325.0 – 1581.8 relationship > > 1-Zero Intelligence Profit 2289.6 – 0.8 1773.5 – 166.9 stdev. 1118.9 – 8.5 633.0 – 264.8 relationship > >

Table 1: Results of one-to-many experiments. For each experiment, the table gives the cumulative profit of the “one” strategy followed by the cumulative profit of the best of the “many”, and an indication of whether the “one” is greater or less than the “many” at the 90% confidence level (determined by a t-test).

Many Price Cutting Many Bait and Switch Many Zero Intelligence 1-Price Cutting Profit 0–7.2 1727.5 – 1475.3 stdev. 0 – 33.5 438.8 – 610.6 relationship < > 1-Bait and Switch Profit 5.9–0 2048.0 – 1397.7 stdev. 40.2 – 0 829.3 – 432.1 relationship > > 1-Zero Intelligence Profit 206.1 – 0 147.2 – 70.2 stdev. 173.4 – 0 54.4 – 227.6 relationship > >

Table 2: Results of one-to-many experiments over the latter days of the run. For each experiment, the table gives the cumulative profit of the “one” strategy over the last 100 days of the experiment followed by the cumulative profit of the best of the “many” and an indication of whether the “one” is greater or less than the “many” at the 90% confidence level (determined by a t-test). how the adaptive charging strategies work in competition this intention, outperforming PC both when one bait-and- against one another. To explore this, we carried out a se- switch takes on multiple price-cutters, and when a single ries of mixed market experiments along the lines of the trad- price-cutter competes against multiple bait-and-switch mar- ing strategy work of (Tesauro & Das 2001). For each of the kets. However, as is the case with PC, when there is more four charging strategies, we ran an experiment in which all than one market using B&S, they may end up cutting charges but one market used that strategy and the remaining market in a futile attempt to increase market share and hence do not used another strategy, carrying out one such “one-to-many” make much profit — this is what happens when there are experiment for each of the other strategies. In other words, many bait-and-switch markets running against a single zero- we tested every “one-to-many” combination. For all these intelligence market. experiments, we measured the cumulative profit of a market The zero-intelligence strategy, designed to get out of using the charging strategies, and ran the markets alongside price wars by increasing charges when it can, performs well the same null market as before. against both PC and B&S markets when it is in the minor- Table 1 gives the results of “one-to-many” experiments, ity. When there is only one price-cutter or bait-and-switch giving the cumulative profits of the “one” market against the against many zero-intelligence markets, the PC and B&S best performing “many” market for each combination of the may outperform ZIP. However, even when this is the case, adaptive markets. The table also indicates which profit is as Figure 8(b) and Figure 9(b) show, ZIP can still make more the greater at 90% confidence (as determined by a t-test). profit than the other market strategies in the short run (before “>” means the “one” market is better than the best “many” 200 days have elapsed). market at 90% confidence and “<” means the best “many” The results in Table 1 are cumulative over the entire 400 market is better. The day by day results are also given in Fig- days of the experiment. Since the early days of the exper- ures 7(a)–9(d). The results show that one price-cutting mar- iment often contain a lot of noise from the initial explo- ket is effective against many zero-intelligence markets, since ration of the traders, it is interesting to also look at the profits it can capture more traders, as Figure 8(a) shows. In such a over the just the later stages of the experiments, when trader case, both types of market generate good profits. However, movement has settled down. Such results are presented in when there is more than one price-cutter, such markets get Table 2. These results suggest that when it is in the major- into a price war and drive their charges down to zero. ity, the zero intelligence strategy is clearly outperformed by The bait-and-switch strategy was envisaged as a more so- both a single price-cutter and a single bait-and-switch mar- phisticated version of PC, one that exploited its market share ket. Since Figures 8(a) and 9(a) suggest that there is no by increasing charges on traders it had attracted through low longer much movement of traders at this point, the results charges. The results in Table 1 suggest that B&S achieves in Table 2 simply reinforce those in Table 1.

22 29 10000 80 4000

9000 28 70 3500 8000 60 3000 27 7000 50 2500 6000 26 5000 40 2000 25 4000 Cumulative Profit Cumulative Profit 30 1500

Daily Number of Traders 24 3000 Daily Number of Traders 20 1000 2000 23 10 500 1000

22 0 0 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 1: Baseline experiments. GD traders, (a) and (b) with random market selection, (c) and (d) with T market selection ( =0.1, α =1).M0.2: dashed line with solid dots; M0.4: solid line; M0.6: dashed line; M0.8: dotted line.

70 8000 70 3500

60 7000 60 3000

6000 50 50 2500

5000 40 40 2000 4000 30 30 1500

Cumulative Profit 3000 Cumulative Profit

Daily Number of Traders 20 Daily Number of Traders 20 1000 2000

10 1000 10 500

0 0 0 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 2: Robustness experiments. (a) and (b) show ZIC traders, and (c) and (d) show a mixture of GD and ZIC traders, all traders use T market selection ( =0.1, α =1). M0.2: dashed line with solid dots; M0.4: solid line; M0.6: dashed line; M0.8: dotted line.

Related work the price wars induced by myopic price-setting, look similar to some of ours, the scenario we are considering is consid- In our experiments, market performance depends on the mix erably more complex. For one thing, the traders in our sce- of market strategies being considered. This suggests that, as nario — the analogs of the buyers in (Greenwald & Kephart is the case for trading strategies (Tesauro & Das 2001), it 1999) — learn rather than making the same market choice at may be hard to ﬁnd a dominant strategy for deciding mar- every trading opportunity. Secondly, and more importantly, ket charges, though such a conclusion must wait until mar- the markets in (Greenwald & Kephart 1999) have prices set ket strategies have been investigated further. This is particu- by the merchants, while in our case the prices are determined larly important since the strategies that we have considered by the traders. As a result, when traders pick a market in our were, quite intentionally, about the simplest we could imag- scenario, they do not know for sure if they will even be able ine (starting with simple strategies seemed a good way to to trade, much less what prices good will change hands at. understand the problem we are considering). From the perspective of the markets, it is possible to attract As mentioned above, there has been little work on the many traders who, because of their value for the commodity problem of choosing between multiple markets. Our work being traded, do not end up trading. We are in the process of is similar to (Ladley & Bullock 2005), but differs in that our investigating the effect of these subtleties. work assesses the impact of different market charges while (Ladley & Bullock 2005) are concerned with the information available to traders. (Ladley & Bullock 2005) are also Conclusions concerned with markets that are spatially separated, so that This paper has described some of our initial work exam- traders’ access to trading partners is limited by their loca- ining the dynamics of trading when agents can choose be- tion. This is similar to the concern of (Gaston & desJardins tween different markets. While we are wary of drawing too 2005). In comparison, our traders are able to ﬁnd any part- many conclusions from our results, because we are still at ner, but the mobility of traders means that they can be sepa- a very preliminary stage in our investigation, we can dis- rated temporally rather than spatially. tinguish some broad trends. These show that, even when Our work also has similarities to that of (Greenwald & they are limited in their ability to make good trades and Kephart 1999). In the latter, shoppers choose between dif- limited in their learning about markets, traders will gravi- ferent merchants, and the merchants set prices that depend tate to the lowest charging markets rather quickly, and, as on the prices set by other merchants. While some of the re- a result, markets with lower charges generate higher prof- sults obtained in (Greenwald & Kephart 1999), especially its. However, the advantages of low charges are somewhat

23 brittle. The advantages evaporate, for example, when not all ceedings of the Workshop on Trading Agent Design and traders are experienced, and it appears that the best charging Analysis. strategies are both adaptive and, like the simple “zero intel- Madhavan, A. 1992. Trading mechanisms in securities ligence” and “bait-and-switch” strategies that we introduce, markets. The Journal of Finance 47(2):607–641. quick to increase charges when they can. Clearly there are Maskin, E. S., and Riley, J. G. 1985. Auction theory with many other possible charging strategies, and it remains to be private values. American Economic Review 75:150–155. seen whether these conclusions hold when other strategies are tested. Miller, M. H.; Hawke, J. D.; Malkiel, B.; and Scholes, M. Our future, and, indeed, current, work is aimed at fur- 1988. Findings of the Committee of Inquiry Examining the ther untangling the behavior of competing markets. First, Events Surrounding October 19, 1987. Technical report, we want to repeat the existing experiments over longer pe- Chicago Mercantile Exchange. riods, ensuring that the results we have are representative Niu, J.; Cai, K.; Parsons, S.; and Sklar, E. 2006. Reduc- of what happens in the steady state, after all start-up ef- ing price fluctuation in continuous double auctions through fects are removed. Second, we want to try to optimise the pricing policy and shout improvement. In Proceedings of simple adaptive strategies — the behavior of each is deter- the 5th International Conference on Autonomous Agents mined by some simple parameters (for example the market and Multi-Agent Systems. share that the bait-and-switch market looks to capture), and Phelps, S.; Marcinkiewicz, M.; Parsons, S.; and McBurney, it seems likely that suitable adjustment of these parameters P. 2006. A novel method for automated strategy acquisition can improve performance. Third, we aim to investigate ad- in n-player non-zero-sum games. In Proceedings of the ditional market strategies with the aim of discovering one 5th International Conference on Autonomous Agents and that is dominant, moving from the “one-to-many” analysis Multi-Agent Systems. performed here to the kind of evolutionary game theoretic Roth, A. E., and Erev, I. 1995. Learning in extensive- analysis used in (Walsh et al. 2002). form games: Experimental data and simple dynamic mod- Acknowledgments The authors are grateful for financial els in the intermediate term. Games and Economic Behav- support received from the National Science Foundation un- ior 8:164–212. der grant NSF IIS-0329037. Satterthwaite, M. A., and Williams, S. R. 1993. The Bayesian theory of the k-double auction. In Friedman, D., References and Rust, J., eds., The Double Auction Market: Institutions, Cliff, D., and Bruten, J. 1997. Minimal-intelligence agents Theories and Evidence. Cambridge, MA: Perseus Publish- for bargaining behaviours in market-based environments. ing. 99–123. Technical report, Hewlett-Packard Research Laboratories. Shah, A., and Thomas, S. 2000. David and Goliath: Friedman, D. 1993. The double auction institution: A Displacing a primary market. Global Financial Markets survey. In Friedman, D., and Rust, J., eds., The Double 1(1):14–21. Auction Market: Institutions, Theories and Evidence.Cam- Shah, A. 1997. Competing exchanges: The international bridge, MA: Perseus Publishing. 3–25. dimension. Business Standard. Friedman, D. 1998. Evolutionary economics goes main- Smith, V. L. 1962. An experimental study of com- stream: A review of “The Theory of Learning in Games”. petitive market behaviour. Journal of Political Economy Journal of Evolutionary Economics 8(4):423–432. 70(2):111–137. Gaston, M. E., and desJardins, M. 2005. Agent-organized Sunder, S. 2004. Markets as artifacts: Aggregate efficiency networks for dynamic team formation. In Proceedings of from zero-intelligence traders. In Augier, M., and March, the 4th International Conference on Autonomous Agents J. G., eds., Models of a Man: Essays in memory of Herbert and Multiagent Systems. A. Simon. Cambridge MA: MIT Press. Gjerstad, S., and Dickhaut, J. 1998. Price formation in Sutton, R. S., and Barto, A. G. 1998. Reinforcement learn- double auctions. Games and Economic Behavior 22:1–29. ing: An introduction. Cambridge, MA: MIT Press. Gode, D. K., and Sunder, S. 1993. Allocative efficiency Tesauro, G., and Das, R. 2001. High-performance bidding of markets with zero-intelligence traders: Market as a par- agents for the continuous double auction. In Proceedings tial substitute for individual rationality. Journal of Political of the 3rd ACM Conference on Electronic Commerce. Economy 101(1):119–137. Vickrey, W. 1961. Counterspeculation, auctions, and com- Greenwald, A. R., and Kephart, J. O. 1999. Shopbots and petitive sealed tenders. Journal of Finance 16:8–27. pricebots. In Proceedings of the 16th International Joint Walsh, W.; Das, R.; Tesauro, G.; and Kephart, J. O. 2002. Conference on Artificial Intelligence. Analyzing complex strategic interactions in multi-agent Klemperer, P. 2002. How (not) to run auctions: The Eu- systems. In Proceedings of Workshop on Game-Theoretic ropean 3G telecom auctions. European Economic Review and Decision-Theoretic Agents. 46:829–845. Ladley, D., and Bullock, S. 2005. Who to listen to: Ex- ploiting information quality in a ZIP-agent market. In Pro-

24 70 4000 45 3000

60 3500 40 2500

3000 35 50 2000 2500 30 40 2000 25 1500 30

Cumulative Profit 1500 20 Cumulative Profit 1000 Daily Number of Traders 20 Daily Number of Traders 1000 15

500 10 500 10

0 0 5 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 3: Learning experiments. GD traders, (a) and (b) with T traders ( =1, α =0.95), (c) and (d) with Tτ traders (τ =1, α =0.95). M0.2: dashed line with solid dots; M0.4: solid line; M0.6: dashed line; M0.8: dotted line.

80 4000 50 3500

70 3500 45 3000

60 3000 40 2500

50 2500 35 2000 40 2000 30 1500 Cumulative Profit 30 1500 Cumulative Profit Daily Number of Traders Daily Number of Traders 25 1000 20 1000

10 500 20 500

0 0 15 0 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 200 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 4: Population experiments. GD traders, all traders use T market selection ( =0.1, α =1). In (a) and (b), all traders learn continuously through the experiment. In (c) and (d), 10% of the traders re-start learning every day. M0.2: dashed line with solid dots; M0.4: solid line; M0.6: dashed line; M0.8: dotted line.

50 7000 60 0.8

45 6000 0.7 50 40 0.6 5000 35 40 0.5 30 4000 30 0.4 25 3000 Cumulative Profit Cumulative Profit 0.3 20 20 Daily Number of Traders 2000 Daily Number of Traders 0.2 15 10 1000 10 0.1

5 0 0 0 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 5: Homogeneous markets: ﬁxed price markets and pricecutting markets. GD traders, all traders use T market selection ( =0.1, α =1). (a) and (b) are 4 homogeneous ﬁxed price markets, (c) and (d) are 4 homogeneous pricecutting markets. The crossed line denotes traders that choose not to enter any market. Market with 20% initial charge: dashed line with solid dots; Market with 40% initial charge: solid line; Market with 60% initial charge: dashed line; Market with 80% initial charge: dotted line.

50 100 40 900

45 90 35 800 40 80 700 30 35 70 600 25 30 60 500 25 50 20 400 20 40

Cumulative Profit 15 Cumulative Profit 300 Daily Number of Traders 15 30 Daily Number of Traders 10 200 10 20 5 5 10 100

0 0 0 0 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 Day Day Day Day (a) Number of traders each day (b) Cumulative proﬁt (c) Number of traders each day (d) Cumulative proﬁt

Figure 6: Homogeneous markets: bait-and-switch markets and zero intelligence markets. GD traders, all traders use T market selection ( =0.1, α =1). (a) and (b) are 4 homogeneous bait-and-switch markets, (c) and (d) are 4 homogeneous zero intelligence markets. The crossed line denotes traders that choose not to enter any market. Market with 20% initial charge: dashed line with solid dots; Market with 40% initial charge: solid line; Market with 60% initial charge: dashed line; Market with 80% initial charge: dotted line.

25 60 90 60 90

80 80 50 50 70 70

40 60 40 60

50 50 30 30 40 40 Cumulative Profit Cumulative Profit 20 30 20 30 Daily Number of Traders Daily Number of Traders

20 20 10 10 10 10

Figure 7: Heterogeneous markets: pricecutting against bait-and-switch. GD traders, all traders use T market selection ( =0.1, α =1). In (a) and (b) one pricecutting market (solid line) competes with 3 bait-and-switch markets, in (c) and (d), one bait- and-switch market (solid line) competes with 3 pricecutting markets. The grey line denotes traders that choose not to enter any market.

50 7000 60 2500

45 6000 50 2000 40 5000 35 40 1500 30 4000 30 25 3000 1000 Cumulative Profit Cumulative Profit 20 20 Daily Number of Traders 2000 Daily Number of Traders 15 500 10 1000 10

Figure 8: Heterogeneous markets: pricecutting against zero intelligence. GD traders, all traders use T market selection ( =0.1, α =1). In (a) and (b) one pricecutting market (solid line) competes with 3 zero intelligence markets, in (c) and (d), one zero intelligence market (solid line) competes with 3 pricecutting markets. The grey line denotes traders that choose not to enter any market.

50 7000 50 1800

45 45 1600 6000 40 40 1400 35 5000 35 1200 30 30 4000 1000 25 25 3000 800 20 20 Cumulative Profit Cumulative Profit 600 Daily Number of Traders 15 2000 Daily Number of Traders 15 400 10 10 1000 5 5 200

Figure 9: Heterogeneous markets: bait-and-switch against zero intelligence. GD traders, all traders use T market selection ( =0.1, α =1). In (a) and (b) one bait-and-switch market (solid line) competes with 3 zero intelligence markets, in (c) and (d), one zero intelligence market (solid line) competes with 3 bait-and-switch markets. The grey line denotes traders that choose not to enter any market.