Learning and Convergence to Nash in Games with Continuous Action Sets∗ Sebastian Bervoetsy Mario Bravoz and Mathieu Faurex November 9, 2016 Abstract We analyze convergence to Nash equilibria in games that have continuous action sets, when players use a simple learning process. We assume that agents are unsophis- ticated and know nothing about the game. They only observe their own payoff after actions have been played and choose their next action according to how their last pay- offs have varied. We also assume that players update their actions simultaneously, and we allow for heterogeneous patterns of interactions. We show that despite such limited information, in all games with concave payoff functions, convergence to Nash occurs much more easily than in discrete games. Further, we show that in ordinal potential games, convergence to Nash occurs with probability one. We also show that in games with strategic complements, the process converges with probability one to a Nash equi- librium that is stable. Finally we focus our attention on network games, where we get even stronger results. We believe that these results argue strongly for the use of Nash equilibrium in economics. Keywords: Learning, Nash equilibrium, Potential games, Strategic complements ∗We wish to thank Y. Bramoull´e,P. Mertikopoulos, N. Allouch, F. Dero¨ıan,M. Belhaj, and participants of the IHP Game Theory seminar in Paris and participants of the Conference on Network Science in Economics in Stanford for useful conversations. The authors thank the French National Research Agency (ANR) for their financial support through the program ANR 13 JSH1 0009 01. Part of this work was completed while the authors were hosted at the Institute for Mathematical Sciences, National University of Singapore, during the Workshop on Stochastic Methods in Game Theory in 2015. yAix-Marseille University (Aix-Marseille School of Economics), CNRS and EHESS zUniversidad de Santiago de Chile, Departamento de Matem´aticay Ciencia de la Computaci´on xAix-Marseille University (Aix-Marseille School of Economics), CNRS and EHESS 1 1 Introduction The concept of Nash equilibrium is central to economics. However, computing Nash equi- libria requires agents to possess an amount of information and sophistication that may seem unrealistic. This has motivated a large literature on learning, which aims at investigating whether agents with limited information and sophistication playing a repeated game, will eventually converge to a Nash equilibrium. Attention has mainly been devoted to games with finite strategy sets. However, many variables appearing in economic models, such as prices, effort, time allocation give rise to games with continuous action sets, about which little is known in terms of learning. We tackle this issue by proposing a natural learning process for continuous games in contexts where agents have as little information as possible on the game they are playing, and almost no sophistication. We analyze the convergence of this learning process, which yields some general insights. Contrary to discrete games, where convergence to Nash is generally difficult to obtain even for two- or three-player games, we show that convergence is easier to obtain with continuous action sets, for very general classes of games and with arbitrary numbers of players. Actually, convergence is possible in any game, provided the payoff functions are strictly concave in one's own action. In addition, while convergence to Nash occurs in many cases in ordinal potential games, it is guaranteed in games with strategic complements. In that case we also prove that the process will converge to Nash equilibria that are stable. In the learning process we consider, a continuous game is repeated across periods that are divided into two stages. First, all players simultaneously move a small distance away from their current action, in a direction chosen at random. This distance decreases over time and goes to zero as the periods unfold. We call this the exploration stage. Second, players observe the payoff associated with their exploration, and compare it with the previous payoff. If the payoff has increased (resp. decreased), players move in the same (resp. opposite) direction, with an amplitude proportionate to the variation in payoff. We call this the updating stage. Players receive a new payoff and a new exploration stage begins. This learning process is very simple: it requires no sophistication from the agents, who only need to compare their own payoff in the last and the current periods. Nor does it require any information to be made available to agents, as they do not know what mechanism produced these payoffs. They know nothing about their payoff function, who they are playing against or what actions their opponents have chosen. They may actually not even know that they are playing a game. There is a stochastic part to the process in the exploration stage. We analyze its (random) 2 set of accumulation points, called the limit set1, by resorting to stochastic approximation theory. This theory tells us that the long-run behavior of the stochastic process is driven by some underlying deterministic dynamic system. We thus start by identifying the deterministic dynamic system that underlies our specific stochastic learning process, and use its asymptotic behavior to establish the properties of the limit set of the learning process. In fact, to understand the behavior of the limit set of our learning process, we only need to relate it to the set of stationary points of the deterministic dynamic system. We show that Nash equilibria are always stationary points, although other points may also be stationary. Before detailing our results, we should point out that two cases may arise. While in many economic applications, the set of Nash equilibria consists of isolated points, in some cases continua of equilibria may appear. Here, to be as general as possible, we treat both cases. When equilibria are isolated we will talk about convergence in the usual sense of convergence to a point; when they are in a continuum we will talk about convergence to sets. Likewise, when discussing stability we will refer to linear stability in the case of isolated equilibria and to the notion of attractors in the case of continua. Naturally, results for the general case are less precise than results for the specific case of isolated equilibria. Therefore in each section we will state general results in the form of propositions, and results for isolated points in the form of theorems. Our first set of results (section 3) holds for any game that is strictly concave. We prove that, although some stationary points are non-Nash, the limit set of our process is never contained in this set. Further, we show that if some set of Nash equilibria is stable, then the process will converge to it with positive probability. To be able to make such a statement at this level of generality is surprising. In the case of isolated points these results get more precise: the process will never converge to stationary points that are non-Nash, and it will converge to a Nash equilibrium that is stable with positive probability. Thus, agents playing any (strictly concave) game that they know nothing about, and following a simple updating process, can end up playing a stable Nash equilibrium. Given how difficult it is to obtain convergence to Nash in discrete games, we believe that these findings are striking. In sections 4 and 5, we provide more precise convergence results by focusing on two broad classes of games. We start with ordinal potential games, for which we obtain a striking improvement. First, the limit set of our learning process is always contained in the set of stationary points of the dynamics. When equilibria are isolated, this implies that the process converges to a Nash equilibrium with probability one. This is a very positive result. 1By abuse of language, we will say that the process converges to its limit set, whether it is a point or a non trivial set. It is non convergent when the process goes to infinity. 3 Second, although convergence to unstable equilibria cannot be ruled out, it is still possible to qualitatively distinguish between equilibria that are stable and those that are not, using the properties of the potential function. Finally, we focus on games with strategic complements, a class of games that has also been extensively studied for their ability to model wide-ranging economic situations. However, unlike for potential games, the literature on games with strategic complements does not provide any general method that can help determine whether our stochastic process converges to Nash. This is why convergence results for standard learning processes with discrete actions are hard to obtain in these games (see Bena¨ımand Faure (2012)). However, for our learning process we are able to prove a very tight result: under some weak restrictions on the pattern of interactions, our process converges to a Nash equilibrium with probability one in games with strategic complements. Moreover, we prove that this equilibrium is necessarily stable. Thus, learning not only leads to Nash, it also serves as a selection device between stable and unstable equilibria. Last, we look into games played on networks (section 6). These games have become popular in economics over the last twenty years, as they consider heterogeneous interactions that are described by an underlying network. As we show, this framework allows us to interpret our different validity conditions as a simple condition on the bipartiteness of the network. It also enables us to obtain slightly tighter results. Related Literature Our paper employs a method based on stochastic approximation theory, initially intro- duced in Ljung (1977) and often referred to as the ODE method, which consists in approx- imating a random process by a mean ordinary differential equation.