Fairness Versus Reason in the Ultimatum Game

R EPORTS

omit G4, which gets eliminated anyway. To describe the change in the frequencies x , x , Fairness Versus Reason in the 1 2 and x3 of the strategies G1, G2, and G3, respectively, we use the replicator equation. Ultimatum Game It describes a population dynamics where successful strategies spread, either by cultural 1 1 2,3 Martin A. Nowak, * Karen M. Page, Karl Sigmund imitation or biological reproduction (17). Un- der these dynamics, the reasonable strategy

In the Ultimatum Game, two players are offered a chance to win a certain sum G1 will eventually reach fixation. Populations of money. All they must do is divide it. The proposer suggests how to split the that consist only of G1 and G3 players will sum. The responder can accept or reject the deal. If the deal is rejected, neither converge to pure G1 or G3 populations de- player gets anything. The rational solution, suggested by game theory, is for the pending on the initial frequencies of the two

proposer to offer the smallest possible share and for the responder to accept strategies. Mixtures of G1 and G2 players will it. If humans play the game, however, the most frequent outcome is a fair share. always tend to G1, but mixtures of G2 and G3 In this paper, we develop an evolutionary approach to the Ultimatum Game. players are neutrally stable and subject to We show that fairness will evolve if the proposer can obtain some information random drift. Hence, starting with any mix-

on what deals the responder has accepted in the past. Hence, the evolution of ture of G1, G2, and G3 players, evolution will fairness, similarly to the evolution of cooperation, is linked to reputation. always lead to a population that consists en-

tirely of G1 players (18). Reason dominates The Ultimatum Game is quickly catching up small groups, however: a simple calculation fairness. with the Prisoner’s Dilemma as a prime show- shows that responders should only reject of- Let us now introduce the possibility that piece of apparently irrational behavior. In the fers that are less than 1/nth of the total sum, players can obtain information about previ- past two decades, it has inspired dozens of where n is the number of individuals in the ous encounters. In this case, individuals theoretical and experimental investigations. group (9). A third explanation is based on the have to be careful about their reputation: if The rules of the game are surprisingly simple. idea that a substantial proportion of humans they accept low offers, this may become Two players have to agree on how to split a maximize a subjective utility function differ- known, and the next proposer may think sum of money. The proposer makes an offer. ent from the payoff (10–12). twice about making a high offer. Assume, If the responder accepts, the deal goes ahead. Here, we studied the Ultimatum Game therefore, that the average offer of an h- If the responder rejects, neither player gets from the perspective of evolutionary game proposer to an l-responder is lowered by an anything. In both cases, the game is over. theory (13). To discuss this model, both an- amount a. Even if this amount is very Obviously, rational responders should accept alytically and by means of computer simula- small—possibly because obtaining infor- even the smallest positive offer, since the tions, we set the sum which is to be divided mation on the co-player is difficult or be- alternative is getting nothing. Proposers, there- equal to 1 and assumed that players are equal- cause the information may be considered fore, should be able to claim almost the entire ly likely to be in either of the two roles. Their unreliable by h-proposers—the effect is sum. In a large number of human studies, strategies are given by two parameters p and drastic (19). In a mixture of h-proposers ʦ however, conducted with different incentives q [0,1]. When acting as proposer, the only, the fair strategy, G3 dominates. The in different countries, the majority of propos- player offers the amount p. When acting as whole system is now bistable: depending ers offer 40 to 50% of the total sum, and responder, the player rejects any offer smaller on the initial condition, either the reason- about half of all responders reject offers be- than q. The parameter q can be seen as an able strategy G1 or the fair strategy G3 low 30% (1–6). aspiration level. It is reasonable to assume reaches fixation (Fig. 1). In the extreme The irrational human emphasis on a fair that the share kept by the player acting as case, where h-proposers have full informa- division suggests that players have preferenc- proposer, 1 Ϫ p, should not be smaller than tion on the responder’s type and offer only es which do not depend solely on their own the aspiration level, q. Therefore, only strat- l when they can get away with it, we ob- ϩ Յ payoff, and that responders are ready to pun- egies with p q 1 were considered (14). serve a reversal of the game: G3 reaches ish proposers offering only a small share by The expected payoff for a player using fixation whereas mixtures between G1 and ϭ rejecting the deal (which costs less to them- strategy S1 ( p1,q1) against a player using G2 are neutrally stable. Intuitively, this re- ϭ selves than to the proposers). But how do S2 ( p2,q2) is given (up to the factor 1/2, versal occurs because it is now the respond- Ϫ ϩ these preferences come about? One possible which we henceforth omit) by (i) 1 p1 er who has the initiative: it is up to the Ն Ն Ϫ Ն explanation is that the players do not grasp p2,ifp1 q2 and p2 q1; (ii) 1 p1,ifp1 proposer to react. Ͻ Ͻ Ն Ͻ Ͻ Ϫ that they interact only once. Humans are ac- q2 and p2 q1; (iii) p2,ifp1 q2 and p2 For 0 a h l, G3 risk-dominates Ͻ Ͻ customed to repeated interactions. Repeating q1; and (iv) 0, if p1 q2 and p2 q1. (20): this implies that whenever stochastic the Ultimatum Game is like haggling over a Before studying the full game, with its fluctuations are added to the population (by price, and fair splits are more likely (6–8). continuum of strategies, let us first consider a allowing mutation, for instance, or spatial Another argument is based on the view that so-called minigame with only two possible diffusion), the fair strategy will supersede the allowing a co-player to get a large share is offers h and l (high and low), with 0 Ͻ l Ͻ reasonable one in the long run (Fig. 1). conceding a relative advantage to a direct h Ͻ 1/2 (9, 15). There are four different Let us now study the evolutionary dy- rival. This argument holds only for very strategies (l,l), (h,l), (h,h), and (l,h), which namics on the continuum of all strategies,

we enumerate, in this order, by G1 to G4. G1 S( p,q). Consider a population of n players. is the “reasonable” strategy of offering little In every generation, several random pairs 1Institute for Advanced Study, Einstein Drive, Prince- and rejecting nothing [for the cognoscenti: it are formed. Suppose each player will be 2 ton, NJ 08540, USA. Institute for Mathematics, Uni- is the only subgame perfect Nash equilibrium proposer on average r times and be re- versity of Vienna, Strudlhofgasse 4, A-1090 Vienna, Austria. 3International Institute for Applied Systems of the minigame (16)]. G2 makes a high offer sponder the same number of times. The Analysis, A-2361 Laxenburg, Austria. but is willing to accept a low offer. G3 is the payoffs of all individuals are then summed *To whom correspondence should be addressed. E- “fair” strategy, offering and demanding a up. For the next generation, individuals mail: [email protected] high share. For the sake of exposition, we leave a number of offspring proportional to

www.sciencemag.org SCIENCE VOL 289 8 SEPTEMBER 2000 1773 R EPORTS their total payoff. Offspring adopt the strat- level. Each accepted deal is made known to others in previous encounters, the outcome egy of their parents, plus or minus some a fraction w of all players. Thus, individu- is dramatically different. Under these cir- small random value. Thus, this system in- als who accept low offers run the risk of cumstances, evolutionary dynamics tend to cludes selection and mutation. As before, receiving reduced offers in the future. In favor strategies that demand and offer a fair we can interpret these dynamics as denot- contrast, the costly act of rejecting a low share of the prize. This effect, which does ing biological or cultural reproduction. We offer buys the reputation that one accepts not require the same players to interact observe that the evolutionary dynamics only fair offers. Figure 2 shows that this twice, suffices to keep the aspiration levels lead to a state where all players adopt process can readily lead to the evolution of high. Accepting low offers damages the strategies that are close to the rational strat- fairness. The average p and q values de- individual’s reputation within the group egy, S(0,0). pend on the number of games per individ- and increases the chance of receiving re- Let us now add the possibility that a ual, r, and the fraction w of individuals who duced offers in subsequent encounters. Re- proposer can sometimes obtain information find out about any given interaction. Larger jecting low offers is costly, but the cost is on what offers have been accepted by the r and w values lead to fairer solutions. offset by gaining the reputation of some- responder in the past. We stress that the Hence, evolutionary dynamics, in accor- body who insists on a fair offer. When same players need not meet twice. We as- dance with the predictions of economic game reputation is included in the Ultimatum sume that a proposer will offer, whatever is theory, lead to rational solutions in the basic Game, adaptation favors fairness over rea- smaller, his own p-value or the minimum Ultimatum Game. Thus, one need not assume son. In this most elementary game, infor- offer that he knows has been accepted by that the players are rational utility-maximiz- mation on the co-player fosters the emer- the responder during previous encounters. ers to predict the prevalence of low offers and gence of strategies that are nonrational, but In addition, we include a small probability low aspiration levels. Whatever the evolu- promote economic exchange. This agrees that proposers will make offers that are tionary mechanism—learning by trial and er- well with findings on the emergence of reduced by a small, randomly chosen ror, imitation, inheritance—it always pro- cooperation (21) or of bargaining behavior amount. This effect allows a proposer to motes the same reasonable outcome: low of- (22). Reputation based on commitment and test for responders who are willing to ac- fers and low demands. communication plays an essential role in cept low offers. Hence, p can be seen as a If, however, we include the possibility the natural history of economic life (23). proposer’s maximum offer, whereas q rep- that individuals can obtain some informa- resents a responder’s minimum acceptance tion on which offers have been accepted by

Fig. 1. Fairness dominates in the mini-ultimatum game if proposers have some chance of finding out whether responders might accept a low offer. There are three strategies: the reasonable strategy G1(l,l) offers and accepts low shares; the fair strategy G3(h,h) offers and accepts high shares; the strategy G2(h,l) offers high shares but is willing to accept low shares. If there is Fig. 2. Fairness evolves in computer simula- no information on the respond- tions of the Ultimatum Game, if a sufficiently ϭ er’s type, a 0, then the reasonable strategy G1 dominates the overall dynamics: G1 and G3 are large fraction w of players is informed about bistable, G2 and G3 are neutral, but G1 dominates G2. If there is some possibility of obtaining any one accepted offer. Each player is defined information on the responder’s type, then we assume that h-proposers will reduce their average by an S( p,q) strategy with p ϩ q Յ 1(14). In Ͻ Ͻ Ϫ offers to l -responders by an amount a. For 0 a h l , both G1 and G3 dominate G2. G1 and any one interaction, a random pair of players G3 are still bistable, but the fair strategy has the larger basin of attraction; adding noise or spatial is chosen. The proposer will offer—whatever effects will favor fairness. In the special limit, a ϭ h Ϫ l, which can be interpreted as having full is smaller—either his own p value or the information on the responders type, the game is reversed: G1 and G2 are neutral, whereas G3 lowest amount that he knows was accepted dominates G2; G3 is the only strict Nash solution. The figure shows the flow of evolutionary game by the responder during previous interac- dynamics (17) on the edge of the simplex S3 (18, 19). tions. In addition, there is a small (0.1) probability that the responder will offer his p value minus some random number between 0 Table 1. Payoff matrix for the mini-ultimatum game. and 0.1; this is to test for players who are willing to accept reduced offers. The total population size is n ϭ 100. Individuals repro- G1 G2 G3 G4 duce proportional to their payoff. Offspring G 11Ϫ l ϩ hh l adopt their parent’s p and q values plus a 1 Ϫ ϩ Ϫ ϩ Ϫ G2 1 h l 111h l random number from the interval ( 0.005, G 1 Ϫ h 111Ϫ h ϩ0.005). There are on average r ϭ 50 rounds 3 Ϫ Ϫ ϩ G4 1 l 1 l hh 0 per player per generation in both roles. Equi- librium p and q values are shown averaged over 105 generations. For w ϭ 0 (no infor- Table 2. Payoff matrix for the mini-ultimatum game with information. mation about previous interactions), the p and q values converge close to the rational solution S(0,0); they are not exactly zero G1 G2 G3 G4 because mutation introduces heterogeneity, and the best response to a heterogeneous G 11Ϫ l ϩ h Ϫ ahϪ al 1 population is not S(0,0). For increasing values G 1 Ϫ h ϩ l ϩ a 11Ϫ a 1 Ϫ h ϩ l 2 of w, there is convergence close to the fair G 1 Ϫ h ϩ a 1 ϩ a 11Ϫ h 3 solution, S(1/2,1/2), with q being slightly G 1 Ϫ l 1 Ϫ l ϩ hh 0 4 smaller than p.

1774 8 SEPTEMBER 2000 VOL 289 SCIENCE www.sciencemag.org R EPORTS

References and Notes 16. K. G. Binmore, Fun and Games: A Text on Game which is a saddle on this surface (the surface is 1. W. Gu¨th, R. Schmittberger, B. Schwarze, J. Econ. Theory (Heath, Lexington, MA, 1992). spanned by the edges G1G2G3G4G1). (iv). The corre- Behav. Organ. 3, 376 (1982). 17. J. Hofbauer and K. Sigmund, Evolutionary Games and sponding stable manifolds divide the state space S4 into two regions, a basin of attraction for G and a 2. R. H. Thaler, J. Econ. Perspect. 2, 195 (1988). Population Dynamics (Cambridge Univ. Press, Cam- 1 bridge, 1998). basin of attraction for G3. (v) The line intersects 3. W. Gu¨th and R. Tietze, J. Econ. Psychol. 11, 417 either the face x ϭ 0orx ϭ 0; the intersection with 18. In the minigame, we consider the strategies G (l,l), 4 2 (1990). 1 x ϭ 0 is a saddle point Q(a) on this face. There is no G (h,l), G (h,h), and G (l,h). The matrix of the expect- 4 4. E. Fehr and S. Ga¨chter, Homo Reciprocans and Human 2 3 4 other fixed point on this face. If a varies from 0 to h Cooperation, discussion paper, Institute for Empirical ed payoff values is shown in Table 1. We note first Ϫ l, Q(a) describes an arc from the point Q to the G l h G l l x x Economic Research, University of Zurich (1999); E. that 4( , ) is dominated by 1( , ) so that 4/ 1 point S(1 – l/h, l/h, 0,0). For some values of h and l, Fehr and K. Schmidt, Q. J. Econ. 114, 817 (1999). converges to 0. It follows that all orbits in the interior this arc will cross the edge G G ; at such an inter- S 1 3 5. G. E. Bolton and R. Zwick, Game Econ. Behav. 10,95 of the state space 4 (the simplex spanned by the section, Q(a) ϭ P(a). (vi) If a ϭ h Ϫ l (total knowl- (1995). unit vectors on the xi axes) converge to the boundary ϭ edge of the co-player, full exploitation of his weak- 6. A. E. Roth, in Handbook of Experimental Economics, face, where x4 0. The edge G2G3 (high proposers ness) we get a phase portrait on the G1G2G3-simplex J. H. Kagel and A. E. Roth, Eds. (Princeton Univ. Press, only) consists of fixed points. Those fixed points Ϫ Ϫ Ϫ Ϫ which is exactly the reverse of the phase portrait if Princeton, NJ, 1995); ࿜࿜࿜࿜ , V. Prasknikar, M. between G3 and Q(0,[1 h]/[1 l], [h l]/[1 l],0) ϭ a 0: the Nash equilibria are G3 and the points on Okuno-Fujiwara, S. Zamir, Am. Econ. Rev. 81, 1068 are saturated. They cannot be invaded by low proposers the segment G1S. (vii) Finally, we note that on the (1991). and correspond to Nash equilibria. The other points can G G edge, P(a) separates the basins of attraction: if G G 1 3 7. A. Rubinstein, Econometrica 50, 97 (1982). be invaded by low proposers, 1 and 4. On the edge initially x is larger than 1 Ϫ h ϩ a, then G reaches G G , strategy G dominates. The edge G G is bistable, 1 1 8. G. E. Bolton and A. Ockenfels, Am. Econ. Rev. 90, 166 1 2 1 1 3 fixation. Thus, G has the larger basin of attraction (it with fixed point P(1 Ϫ h,0,h,0). There is one further 3 (2000). is risk-dominant) whenever h Ͻ 1/2 ϩ a. fixed point, R(0,1 Ϫ h ϩ l,0,h – l) which is stable on the 9. S. Huck and J. O¨ chssler, Game Econ. Behav. 28,13 20. H. P. Young, Econometrica 61, 57 (1993); M. Kandori, edge G G but can be invaded by both missing strate- (1999). 2 4 G. J. Mailath, R. Rob, Econometrica 61, 29 (1993); J. gies. The dynamics are simple: orbits can converge Hofbauer, in Game Theory, Experience, Rationality: 10. G. Kirchsteiger, J. Econ. Behav. Organ. 25, 373 G either to 1, which is the only perfect Nash equilibrium Foundations of Social Sciences, Economics and Ethics, (1994). QG of the game, or else to the segment 3. But once W. Leinfellner and E. Ko¨hler, Eds. (Kluwer, Dordrecht, 11. J. Bethwaite and P. Tompkinson, J. Econ. Psychol. 17, there, neutral drift or recurrent mutations introducing 259 (1996). Netherlands, 1998), pp. 245–267. G1 or G4 will inexorably push the state toward QG2 (it 12. D. Kahnemann, J. L. Knetsch, R. Thaler, J. Bus. 59, 21. M. A. Nowak and K. Sigmund, Nature 393, 573 is enough to note that x2/x3 grows whenever x1 or x4 5285 (1986); E. Fehr and K. M. Schmidt, Q. J. Econ. (1998); J. Theor. Biol. 194, 561 (1998); A. Lotem, are positive) so that eventually the reasonable strategy M. A. Fishman, L. Stone, Nature 400, 226 (1999); C. 114, 817 (1999). G 1 reaches fixation. Wedekind and M. Milinsky, Science 288, 850 (2000); 13. J. Maynard Smith, Evolution and the Theory of Games 19. Let us now assume that if a player has a low aspira- A. Zahavi, A. Zahavi, A. Balaban, The Handicap Prin- (Cambridge Univ. Press, Cambridge, 1982). tion level, then this risks to become known to the Ϫ Ն ciple (Oxford Univ. Press, London, 1996). 14. The condition, 1 p q, is equivalent to the proposer, in which case an h-proposer will, with a 22. T. Ellingsen, Q. J. Econ. 112, 581 (1997); H. P. Young, assumption that the individuals do not regard the certain probability, offer a bit less. Suppose the av- J. Econ. Theory 59, 145 (1993); A. Banerjee and J. role of proposer as inferior to the role of responder. erage offer of an h-proposer to an l-responder is h – Weibull, in Learning and Rationality in Economics,A. Therefore, what they demand for themselves when a, which is Ն l. This implies that the payoff matrix is Ͻ Ͻ Ϫ Kirman and M. Salmon, Eds. (Blackwell, Oxford, acting as proposer should not be less than the min- now given by Table 2. For 0 a h l, the fair 1995); K. Binmore, A. Shaked, J. Sutton, Am. Econ. imum amount they expect as responders. This con- strategy G3 dominates G2 on the edge G2G3.Onthe Rev. 75, 1178 (1985). dition is relevant when introducing information into edge G1G3, the system is bistable: the basin of at- 23. From the huge literature on this topic, we quote two the Ultimatum Game. Otherwise, high levels of in- traction of G and G are separated by the point ϭ Ϫ 1 ϩ 3 classics: T. C. Schelling, The Strategy of Conflict (Har- formation can lead to a reversal of the game. If the P(a) (1 h a,0,h – a, 0). The game dynamics vard Univ. Press, Cambridge, MA, 1960); R. Frank, proposer has perfect knowledge of the responder’s q have the following properties: (i) The ratio x2x4/x1x3 Passions within Reason (Norton, New York, 1988). value, then it is in fact the responder who makes the remains constant. (ii) There exists a line of fixed 24. Support from the Leon Levy and Shelby White Initi- offer. In this case, evolutionary dynamics lead to ϭ ϩ␮ points in the interior S4, of the form xi mi (for atives Fund; the Florence Gould Foundation; the J. strategies close to S(1,1); that is, proposers have to ϭ ϭ Ϫ␮ ϭ ␮ i 1,3) and xi mi (for i 2,4) where is a Seward Johnson, Sr. Charitable Trusts; the Ambrose offer almost the full amount. The condition 1 Ϫ p Ն real parameter and (m ,m ,m ,m ) ϭ (1/S)[a(1 Ϫ h), Ϫ Ϫ Ϫ 1 2Ϫ 3 4 ϭ ϩ Monell Foundation; and the Alfred P. Sloan Founda- q avoids this, perhaps unrealistic, complication. l(1 h), l(h l a), a(h l-a )] with S (l a)(1 tion is gratefully acknowledged. 15. J. Gale, K. Binmore, L. Samuelson, Game Econ. Behav. Ϫ l Ϫ a). (iii) This line intersects all invariant surfaces ϭ Ͼ 8, 56 (1995). of the form x2x4 Kx1x3 (for K 0) in a fixed point 8 June 2000; accepted 5 July 2000