Download PDF 1249 KB

11.22.17 WorldQuant Perspectives

Shall We Play a Game?

The study of game theory can provide valuable insights into how people make decisions in competitive environments, including everything from playing tic-tac-toe or chess to buying and selling stocks.

WorldQuant, LLC 1700 East Putnam Ave. Third Floor Old Greenwich, CT 06870 www.weareworldquant.com WorldQuant Shall We Play a Game? Perspectives 11.22.17

JOHN VON NEUMANN WAS A MAN WITH MANY INTERESTS, FROM theory of corporate takeover bids and the negotiations of Britain formulating the mathematics of quantum mechanics to and the European Union over the application of Article 50. One of developing the modern computer. Luckily for the birth of modern the most spectacular applications was in the distribution of large game theory, he also had a passion for poker, an interest that portions of the electromagnetic spectrum to commercial users culminated in the monumental work Theory of Games and in an auction organized by the U.S. government in 1994. Experts Economic Behavior, written in collaboration with economist in game theory were able to maximize both the government Oskar Morgenstern. Their focus was on cooperative games — revenue — amounting in the end to a staggering $10 billion — that is, games in which players can form coalitions or, in the and the efficient allocation of resources to the frequency buyers. words of Fields medalist John Milnor, “sit around a smoke-filled (A similar auction carried out by the New Zealand government in room and negotiate with each other.” Game theory was a radical 1990 without the help of game theory experts ended up being a departure from the standard view of economics, the so-called total fiasco.) Robinson Crusoe economy, in which consumers’ well-being is In this article we will outline the very basics of noncooperative not affected by their social interactions: Like Crusoe, alone on game theory with a view toward financial applications, starting a deserted island where he could interact only with nature, the from the ubiquitous prisoner’s dilemma and concluding with players in this idealized economy interact only with prices. But more-realistic games that describe important aspects of modern as von Neumann observed, “Real life consists of bluffing, of little financial markets and model real investors’ behavior. tactics of deception, of asking yourself what is the other man going to think I mean to do, and that is what games are about in Crime and Punishment my theory.” Alice and Bob have committed a serious crime: They stole $10 Von Neumann’s pioneering work had applications to warfare (he from their mom’s wallet, which she keeps in her nightstand. But was one source of inspiration for the iconic wheelchair-bound being amateur thieves, they were not too careful in orchestrating scientist in Stanley Kubrick’s Dr. Strangelove) but found limited their misdemeanor, and their mom, Carol, caught them applications in real world and economic theory. The next greatest trespassing in her bedroom. She immediately realized something revolution in game theory came from John Nash’s analysis of was out of order and checked her wallet, discovering with great noncooperative games, in which the emphasis is on individual consternation that money was missing. As a precautionary behavior. In 1994, Nash was awarded a Nobel Prize in economics measure, she decided to confine Alice and Bob to separate rooms. for the far-reaching applications of his groundbreaking work. Carol has enough evidence to convict both of her children for the Game theory has emerged as a very powerful and versatile lesser crime of trespassing, for which they would be grounded for technique, capable of modeling situations as diverse as the one day. But she lacks evidence for the principal crime, the theft. Being an ingenious woman, Carol offers Alice and Bob a deal. If they both confess to the theft, they will benefit from a reduced sentence and each will be grounded for five days. If neither The greatest revolution confesses, they will be sentenced only for the lesser crime and grounded for only one day. Finally, if Alice confesses while her in game theory came from accomplice remains silent, she will not face any charge and Bob John Nash’s analysis of will be grounded for 20 days. The same scenario applies if Bob noncooperative games, in confesses and Alice remains silent. The siblings must reach their which the emphasis is on decision independently, without communicating with each other. individual behavior. We can summarize Alice and Bob’s available strategies with the following payoff matrix:

Bob symmetrical, Bob will also choose to defect. The scenario in which both players defect is the sought-after Nash equilibrium for the silent confess prisoner’s dilemma.

silent (-1,-1) (-20,0) Although neither player has an incentive to unilaterally change strategy, the Nash equilibrium does not represent the best

Alice possible outcome: Alice and Bob would have been better off by confess (0,-20) (-5,-5) cooperating and remaining silent. But this scenario is not the equilibrium we just found. There is no “right” solution to this little game, hence the dilemma. Figure 1 The prisoner’s dilemma is the most paradigmatic example of a In the matrix the negative numbers represent the number of days non-zero-sum game. In this kind of game, complementary and of punishment. For example, the values in payoff (−5,−5) are the conflicting interests can be present simultaneously. In zero-sum days Alice and Bob would remain grounded if both confessed. games — like tic-tac-toe, chess or “global thermonuclear war” (played by the computer in the movie WarGames) — players From the siblings’ point of view, remaining silent and confessing are purely antagonistic. In these games “wealth” is transferred can be seen as forms of cooperation and defection. To highlight from loser to winner. In the financial world the futures, this interpretation, we relabel the payoff matrix as follows: options and currency markets are all zero-sum games. By contrast, the stock market is a non-zero-sum game because performance is inextricably linked to external factors, such as Bob the overall economic outlook. All investors could profit in a bull market, for example. cooperate defect Life is riddled with examples of the prisoner’s dilemma, from cooperate (-1,-1) (-20,0) countries negotiating on actions to limit global climate change to birds trying to remove ticks from each other’s feathers Alice (cooperation/defection corresponding to a bird agreeing/ defect (0,-20) (-5,-5) refusing to pull off its companion’s ticks). There is a simple and very practical financial application: competition in oligopolistic Figure 2 markets, where optimal quantity and price always depend on choices made by a small number of companies. Let us consider Given the assumption of rationality — and the adage that there is no honor among thieves — what strategies will Alice and Bob choose? To analyze their strategic behaviors, let us introduce the idea of the Nash equilibrium. We are in a Nash equilibrium if Life is riddled with Alice’s choice is optimal for Alice given Bob’s choice and at the same time Bob’s choice is optimal for Bob given Alice’s choice. In examples of the prisoner’s other words, neither player has an incentive to deviate unilaterally dilemma, from countries by playing a different strategy, given the strategy chosen by his or negotiating on actions her opponent. to limit global climate We can find the Nash equilibrium by first considering Alice’s point change to birds trying of view: If her brother defects, she can be punished with either 20 days (if she cooperates) or five days (if she defects). Given these to remove ticks from each outcomes, we can easily guess Alice’s choice: She will decide to other’s feathers. defect by confessing to her mom! Because the game is completely

Copyright © 2017 WorldQuant, LLC WorldQuant Perspectives November 2017 3 traders, areonlymoderately represent theavailableoptionswithfollowing payoffmatrix, would beforced tofollow toprotect itsmarket share. We can Coca-Cola decidedtoreduce itsprice—thatis,defectPepsiCo their carbonatedbeverages artificiallyhigh. Butif,outoftheblue, interest ofbothcolamakers tocooperate andkeep thepricesof the caseofrivalsCoca-ColaCo.andPepsiCo.Itwouldbein Alice isvery passionateaboutactionmovies, whileBobhasa watch theactionfilmor romantic comedyplayingthere. movie theater, butneither canrecall whethertheyplannedto Alice andBob,nolongergrounded, have decidedtomeetata the sexes.” illustration, letusconsiderthegameknown asthe“battleof Games withmultipleequilibriaare common,though.Asan In theprisoner’s dilemmawe foundasingleNashequilibrium. incentive todefect. are againinaprisoner’s dilemma,asbothcompanieshave an profits per year (inarbitrary units).Itiseasytoseethat we in whichtheentriesrepresent theincrease inthecompanies’ WorldQuant Copyright ©2017 WorldQuant, LLC while atthesametimevery learning from experience. Perspectives efficient atrecognizing good atdeductivelogic Most humans,including emerging patternsand Coca-Cola cooperate defect cooperate (5, 5) (7, 0) Figure 3 11.22.17 Shall WePlayaGame? PepsiCo defect (2, 2) (0, 7)

We canrepresent thesituationwithfollowing payoffmatrix: should theydo? is deadandtheycannotcommunicatewitheachother. What alone amongstrangers. Unfortunately,Alice’s cellphonebattery preferences, theywouldrather watchamovie togetherthansit clear partialityforromantic comedies.Despitetheirpersonal unlikely togive youanedgeinyournext game. an explicitconstructive solutionisnotknown, soknowing thisis mixed Nashequilibrium.Given thegames’complexity, however, equilibrium alwaysexists.Gameslike chessandpoker have a must terminateafterafinitenumberof moves —amixedNash Nash proved thatforallfinitegames —thatis,games percent probability (1/5). probability (4/5) andBobgoingtotheromantic movie witha20 corresponds toAlicegoingtheactionmovie withan80percent 1. Forthebattleofsexes,mixedNashequilibrium mixed equilibriainwhichastrategy isplayed withprobability probability. Clearly, allpure equilibriaare particularcasesof but rather choosesamongthepossiblestrategies withacertain equilibrium, aplayer doesnotalwayschoosethesamestrategy type ofequilibrium—a Nash equilibria.Inthebattleofsexes,we alsohave anew All theexampleswe have encountered sofarare known as attending theromantic comedy. Nash equilibria:bothsiblingsattendingtheactionmovie andboth by azero payoff.Itissimpleto verify thatthisgamehastwo unhappiness iftheyendupwatchingdifferent movies isquantified Alice beingfourtimeshappierthanherbrother. Thesiblings’ watching theactionmovie, thepayoffcanbeinterpreted as and Bob’s levels ofhappiness.Forexample,iftheyendup We canthinkoftheentriesinmatrixasmeasures ofAlice’s

Alice romantic action mixed Figure 4 action Nashequilibrium.Inthiskindof (0, 0) (4, 1) WorldQuant Bob Perspectives romantic (1, 4) (0, 0) November 2017 pure

4 WorldQuant Shall We Play a Game? Perspectives 11.22.17

Nice Guys Finish First Although cooperation is often a fact of life, we saw that The stock market is a rational players in the prisoner’s dilemma end up playing selfishly even though mutual cooperation is in their best non-zero-sum game because interest. How, then, can cooperation emerge without being performance is inextricably forced by an external authority? linked to external factors, One way to achieve this is to play multiple rounds of the prisoner’s such as the overall dilemma. This was the insight of American political scientist economic outlook. Robert Axelrod, who in the early 1980s organized prisoner’s dilemma tournaments for which various experts in game theory submitted their computer code. The winner was the simplest bar is overcrowded, all the fun would be lost; you would rather program, a four-line program called TIT-FOR-TAT. It works as stay home. The bar can easily accommodate 60 people. If you follows: In the first round the program always cooperates, and in expect there will be fewer than 60, you decide to go. Other people the successive rounds it simply copies the opponent’s choice from — say, 100 other people — are also interested in going. Everyone the previous move. else shares your preferences, and the only public information is past attendance. What is your optimal strategy? TIT-FOR-TAT clearly values cooperation. If the opponent decides to cooperate in its first move, TIT-FOR-TAT will appreciate the This dilemma, known as the El Farol problem after a bar in Santa kind gesture and cheerfully cooperate in the second move. But Fe, New Mexico, was proposed by economist W. Brian Arthur in it will also penalize unprofitable encounters (hence the name): 1994. Note that in this situation there is no common, global best If the opponent decides to defect in its first move, TIT-FOR-TAT strategy. Indeed, if such a strategy existed, everybody would use will punish the selfish behavior by defecting in the second move. it. As a consequence, if the strategy predicted a crowded bar next Axelrod introduced the concepts of niceness and forgiveness to Friday, nobody would go. Basing your decision on tossing a coin characterize different behaviors; a program is nice if it never wouldn’t help either: On average, only 50 people would show up, defects first. Similarly, a program is forgiving if it tends to resume and the bar would be underutilized. cooperation after its opponent has done so. TIT-FOR-TAT is an example of a nice and forgiving strategy. Remarkably, such The key to solving the El Farol problem turns out to be an efficient programs achieved the highest score in Axelrod’s tournaments — analysis of previous attendance, using methods akin to those cooperation at last! in standard technical analysis. A more rigorous analysis can be done considering a famous variant of this problem, the so-called The games we have considered so far had a small number of minority game. In this game an odd number of players compete perfectly rational players. But most humans, including traders, against one another by choosing between two possible outcomes. are only moderately good at deductive logic while at the same Rather than deciding whether to visit a favorite bar, the option time very efficient at recognizing emerging patterns and here is to buy or sell a stock. The winners of the game are those learning from experience. In the final section we will introduce who end up selecting the side chosen by the minority. Because the a more complicated game that can model the behavior of many number of winners is smaller than the number of losers, this is an interacting traders participating in a real-world market. In this interesting example of a negative-sum game. model players are not perfectly rational but will be able to learn Let us consider the simplest possible case, involving three and improve their strategies through evolution. players: Alice, who has grown up to become a respected trader, and two of her colleagues, who have some time to kill. At each A Minority Game stage of the game, they buy or sell a fictitious stock. If Alice plays It’s Friday night and you want to chill out at your favorite a contrarian strategy — for example, if she sells when the other downtown bar. But after a long week of work, you know that if the two traders buy — she will make money because she will sell at

particles in the visible universe. Therefore, despite its apparent simplicity, the model has a rich dynamic. From the strategy pool Momentum strategies each player is given a certain number S of strategies to trade. essentially follow the Note that the case S = 1 would not be interesting because all leading market sentiment. traders would be forced to always play the very same strategy and there would be no room for learning. Each strategy is awarded a But in an optimal strategy, virtual point for any correct prediction of the market outcome. At it is not wise to always each round Alice trades her best strategy — that is, the one with the highest current virtual score. If her best strategy happens to follow the crowd. correctly predict the next outcome, she will gain $1.

At each round of the game, we can sum all traders’ decisions. This sum represents the excess demand for the underlying stock; a price that has been set higher by the demand of her fellows. The goal of a good strategy is to correctly forecast the minority side. its volatility models the fluctuations of the underlying market. A high-volatility regime will correspond to a noisy and unpredictable In their simplest incarnation momentum strategies essentially market, whereas a low-volatility regime will correspond to a follow the leading market sentiment. But in an optimal strategy, predictable market. Not all strategies are independent, and the it is not wise to always follow the crowd. Even when we correctly number of uncorrelated strategies turns out to be 2M. Note that anticipate the overall sentiment, we will want to enter or exit the the smaller the traders’ memory capacity M, the smaller the market before other traders do. We will transact at a better price number of uncorrelated strategies they can create. To explore the by being ahead of the crowd. It is, then, beneficial to be contrarian, minority game’s remarkably rich dynamics, let us now introduce and the minority game precisely models this scenario because the parameter θ 2M/N, which represents the ratio of the number winners by definition are in the minority group. of uncorrelated strategies to the number of traders. As we vary θ, we have a phase transition from a predictable phase of the market The mathematical tools necessary for the analysis of the minority game are those provided by statistical physics and in particular by to an unpredictable one. the theory of phase transitions, a subject we briefly touched upon in a previous article. When θ is small enough — that is, the number of independent strategies is small compared with the number of players — the Now let us consider the rules of the game in more detail. We have traders end up crowding into the same strategies. The agents N players, with N an odd integer. Each player has finite resources become a crowd as they process the available information in and can process only a finite amount of past information. This the same way and end up using the same best strategy. This information is public and amounts to knowing the history of the phase is, therefore, dominated by herding; because the purpose most recent M winning outcomes. The parameter M, therefore, of the game is to be in the minority, all the traders lose. Without represents the memory capacity of each trader. Note that the a sufficiently diversified pool of strategies, traders behave like assumption of a limited memory is especially relevant in an the proverbial sheep running off a cliff. In this phase players do era in which computational power is never enough to tame the worse than they would have if they’d just tossed a coin. In this exponential growth of Big Data. “worse than random” regime, there is no helpful information to be If we represent sell/buy decisions as −1/1, and memory M = 5, a extracted from history. possible history could look like −1,−1,1,1,1, in which case the first two winning decisions are sells (sellers were a minority) and the Let us now consider the other phase, for a sufficiently large remaining three are buys (buyers were a minority). θ. In this regime traders have a good memory and the number of independent strategies is sufficiently large relative to A strategy is a function that, given history, makes a forecast for the number of players. We are in a “better than random” phase: the next market outcome. The number of possible strategies is Strategies perform better than random coin tosses, and future potentially enormous: For a trading period of only two weeks, market behavior is predictable with some degree of statistical the number of strategies is far larger than the total number of confidence. This period is also known as the inefficient phase

of the market: Arbitrage opportunities exist, and the efficient The minority game is a promising attempt at modeling the trial- market hypothesis is clearly violated. Quite remarkable is the and-error inductive thinking of real traders. It also shows the fact that as the traders’ memory capacity increases, the market’s deep connection between certain applications of game theory and predictability decreases, approaching asymptotically the random statistical physics. Von Neumann himself was very keen to apply toss game. This shows that being omniscient is not beneficial! ideas rooted in statistical physics to the study of large numbers of interacting agents in an economy. After all, game theory has never At the interface of the worse-than-random and better-than- been an ivory tower theory, and from its very early beginnings random phases, we have a very interesting regime in which it has benefited from the interaction with a vast range of fields, the market has minimum volatility and therefore is in its most predictable state; traders have learned to share the limited from economics to political science to biology. Game theory has available resources. This phase corresponds to a cooperative evolved into a mature discipline central to our understanding of regime in which agents can efficiently process historical human behavior and strategic interactions. And perhaps more information and attempt to predict future market movements — pragmatically, the next time you find yourself in an awkward a somewhat surprising revelation considering the intrinsically social dilemma or wonder whether snitching on your friend is a selfish nature of traders. good idea, you know where to look! ◀

References Robert Axelrod and William D. Hamilton. John Nash. “Non-Cooperative Games.” “The Evolution of Cooperation.” Science 211 (1981). Annals of Mathematics 54, no. 2 (1951). Robert Axelrod. The Evolution of Cooperation (Basic Books, 1984). John von Neumann and Oskar Morgenstern. Damien Challet, Matteo Marsili and Yi-Cheng Zhang. Theory of Games and Economic Behavior Minority Games (Oxford University Press, 2005). (Princeton University Press, 1944).

Thought Leadership articles are prepared by and are the property of WorldQuant, LLC, and are circulated for informational and educational purposes only. This article is not intended to relate specifically to any investment strategy or product that WorldQuant offers, nor does this article constitute investment advice or convey an offer to sell, or the solicitation of an offer to buy, any securities or other financial products. In addition, the above information is not intended to provide, and should not be relied upon for, investment, accounting, legal or tax advice. Past performance should not be considered indicative of future performance. WorldQuant makes no representations, express or implied, regarding the accuracy or adequacy of this information, and you accept all risks in relying on the above information for any purposes whatsoever. The views expressed herein are solely those of WorldQuant as of the date of this article and are subject to change without notice. No assurances can be given that any aims, assumptions, expectations and/or goals described in this article will be realized or that the activities described in the article did or will continue at all or in the same manner as they were conducted during the period covered by this article. WorldQuant does not undertake to advise you of any changes in the views expressed herein. WorldQuant may have a significant financial interest in one or more of any positions and/or securities or derivatives discussed.