A Service of Leibniz-Informationszentrum econstor Wirtschaft Leibniz Information Centre Make Your Publications Visible. zbw for Economics Duersch, Peter; Oechssler, Jörg; Schipper, Burkhard C. Working Paper Once Beaten, Never Again: Imitation in Two-Player Potential Games Discussion Paper Series, No. 529 Provided in Cooperation with: Alfred Weber Institute, Department of Economics, University of Heidelberg Suggested Citation: Duersch, Peter; Oechssler, Jörg; Schipper, Burkhard C. (2012) : Once Beaten, Never Again: Imitation in Two-Player Potential Games, Discussion Paper Series, No. 529, University of Heidelberg, Department of Economics, Heidelberg, http://dx.doi.org/10.11588/heidok.00013440 This Version is available at: http://hdl.handle.net/10419/127338 Standard-Nutzungsbedingungen: Terms of use: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Documents in EconStor may be saved and copied for your Zwecken und zum Privatgebrauch gespeichert und kopiert werden. personal and scholarly purposes. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle You are not to copy documents for public or commercial Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich purposes, to exhibit the documents publicly, to make them machen, vertreiben oder anderweitig nutzen. publicly available on the internet, or to distribute or otherwise use the documents in public. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, If the documents have been made available under an Open gelten abweichend von diesen Nutzungsbedingungen die in der dort Content Licence (especially Creative Commons Licences), you genannten Lizenz gewährten Nutzungsrechte. may exercise further usage rights as specified in the indicated licence. www.econstor.eu University of Heidelberg Department of Economics Discussion Paper Series No. 529 Once Beaten, Never Again: Imitation in Two-Player Potential Games Peter Duersch, Jörg Oechssler, and Burkhard C. Schipper June 2012 Once Beaten, Never Again: Imitation in Two-Player Potential Games Peter Duerschy Jörg Oechssler z Burkhard C. Schipperx May 7, 2012 Abstract We show that in symmetric two-player exact potential games, the simple decision rule “imitate-if-better” cannot be beaten by any strategy in a repeated game by more than the maximal payo¤ di¤erence of the one-period game. Our results apply to many interesting games including examples like 2x2 games, Cournot duopoly, price competition, public goods games, common pool resource games, and mini- mum e¤ort coordination games. Keywords: Imitate-the-best, learning, exact potential games, symmetric games, relative payo¤s, zero-sum games. JEL-Classi…cations: C72, C73, D43. Some of the material was previously circulated in a companion paper “Unbeatable Imitation”. We thank Carlos Alós-Ferrer, Chen Bo, Drew Fudenberg, Alexander Matros, Klaus Ritzberger, Karl Schlag, and John Stachurski for interesting discussions. Seminar audiences at Australian National University, Melbourne University, Monash University, UC Davis, UC San Diego, the Universities of Heidelberg, Konstanz, Wien, and Zürich, the University of Queensland, the University of Oregon, Calpoly, at the International Conference on Game Theory in Stony Brook, 2009, the Midwestern Economic Theory Con- ference in Evanston 2010, and at the Econometric Society World Congress 2010 in Shanghai contributed helpful comments. yDepartment of Economics, University of Heidelberg, Email: [email protected] zDepartment of Economics, University of Heidelberg, Email: [email protected] xDepartment of Economics, University of California, Davis, Email: [email protected] 1 Introduction Suppose you play a repeated symmetric game against one opponent. Suppose further that you know that this opponent uses a very simple learning algorithm, namely the rule “imitate-if-better”,the rule that simply prescribes to mimic the action of the other player if and only if the other player received a higher payo¤ in the previous period. Since the rule is deterministic, you therefore know exactly what your opponent will do in all future periods and how he will react to your actions. The question we pose in this paper is whether you can use this knowledge to exploit the imitator in the sense hat you achieve a higher payo¤ than the imitator. We show that surprisingly there are meaningful classes of games like all symmetric 2x2 games, Cournot duopoly, Bertrand duopoly, public goods games, common pool re- source games, minimum e¤ort coordination games, and Diamond’ssearch, where this is essentially not possible. To be precise, we call imitation “essentially unbeatable” if in the repeated game there exists no strategy of the opponent with which the opponent can obtain, in total, over an in…nite number of periods, a payo¤ di¤erence that is more than the maximal payo¤ di¤erence for the one–period game. To gain some intuition for this, consider the game of “chicken” presented in the following payo¤ matrix. swerve straight swerve 3; 3 1; 4 straight 4; 1 0; 0 Suppose that initially the imitator starts out with playing “swerve”. What should a forward looking opponent do? If she decides to play “straight”, she will earn more than the imitator today but will be copied by the imitator tomorrow. From then on, the imitator will stay with “straight” forever. If she decides to play “swerve” today, then she will earn the same as the imitator and the imitator will stay with “swerve”as long as the opponent stays with “swerve”. Suppose the opponent is a dynamic relative payo¤ maximizer. In that case, the dynamic relative payo¤ maximizer can beat the imitator at most by the maximal one-period payo¤ di¤erential of 3. Now suppose the opponent maximizes the sum of her absolute payo¤s. The best an absolute payo¤ maximizer can do is to play swerve forever. In this case the imitator cannot be beaten at all as he receives the same payo¤ as his opponent. In either case, imitation comes very close to the top–performing heuristics and there is no evolutionary pressure against such an heuristic. 1 These results extend results from our recent paper Duersch, Oechssler, and Schipper (2011a), in which we show that imitation is subject to a money pump (i.e., can be exploited without bounds) if and only if the relative payo¤ game is of the rock-paper- scissors variety. The current results are stronger because they show that imitation can only be exploited with a bound that is equal to the payo¤ di¤erence in the one–period game. However, this comes at the cost of restricting ourselves to the class of symmetric two-player exact potential games. But as mentioned above, many economically relevant games satisfy this property. Exact potential games have been introduced by Monderer and Shapley (1996) and are studied widely in learning in games (see Sandholm, 2010). The behavior of learning heuristics has previously been studied mostly for the case when all players use the same heuristic. For the case of imitate-the-best,1 Vega-Redondo (1997) showed that in a symmetric Cournot oligopoly with imitators, the long run out- come converges to the competitive output if small mistakes are allowed. This result has been generalized to aggregative quasisubmodular games by Schipper (2003) and Alós- Ferrer and Ania (2005). Huck, Normann, and Oechssler (1999), O¤erman, Potters, and Sonnemans (2002), and Apesteguia et al. (2007, 2010) provide some experimental evi- dence in favor of imitative behavior. In contrast to the above cited literature, the current paper deals with the interaction of an imitator and a possibly forward looking, very ratio- nal and patient player. Apart from experimental evidence in Duersch, Kolb, Oechssler, and Schipper (2010) and our own paper Duersch at al. (2011a) we are not aware of any work that deals with this issue. For a Cournot oligopoly with imitators and myopic best reply players, Schipper (2009) showed that the imitators’long run average payo¤s are strictly higher than the best reply players’average payo¤s.2 A recent paper by Feldman, Kalai, and Tennenholtz (2010) has a similar but comple- mentary objective to ours. They study whether a strategy which they call “copycat”can be beaten in a symmetric two-player game by an arbitrary opponent who may have full knowledge of the game and may play any history dependent strategy. Remarkably, the copycat strategy can nearly match the average payo¤ of the opponent. Yet, their copy- cat rule is far more sophisticated than our imitation rule as it entails …nding a (possibly 1 For the two-player case, imitate-the-best and imitate-if-better are almost equivalent, the di¤erence being that the latter speci…cally prescribes a tie-breaking rule (for the case of both players having equal payo¤s in the previous round). Since we use imitate-if-better only in the two-player case, we do not need to specify what happens if more than one other player is observed. 2 In a related paper, Hehenkamp and Kaarbøe (2008) study a Cournot duopoly with one imitator and one myopic optimizer in a rapidly changing environment with strategic substitutes. If the optimizer can adjust decisions as fast as some parameters of the game change (like the demand), then it is possible that the optimizer is better o¤ than the imitator if strategic substitutes are su¢ ciently weak. 2 mixed) minmax strategy in the auxiliary zero–sum game in each round. On the other hand, the possible opponents are less omniscient than the opponents in our setting. In particular, the opponents in their setting cannot perfectly predict the imitator’s action in the next round. Somewhat farther related is the literature on playing games by a population with het- erogeneous learning rules. For instance, Droste, Hommes, and Tuinstra (2002) consider a large population of …rms matched randomly into a Cournot duopoly. A fraction of …rms play a best response to the average output in the previous period whereas others perfectly forecast the output of either decision rule, play a best response to the forecast but bear an extra information cost.