A Class of Population Dynamics for Reaching Epsilon-Equilibria: Engineering Applications

German´ Obando, Julian Barreiro-Gomez, and Nicanor Quijano

Abstract— This document proposes a novel class of popula- The contribution of this work is the deduction of a class of tion dynamics that are parameterized by a nonnegative scalar population dynamics that are parameterized by a nonnegative . We show that any rest point of the proposed dynamics escalar  and converge to the set of -equilibria of the corresponds to an -equilibrium of the underlying population game. In order to derive this class of population dynamics, underlying game. Indeed, we formally show that the set our approach is twofold. First, we use an extension of the of epsilon-equilibria and the rest points of the proposed - pairwise comparison revision protocol and the classic mean dynamics are equal. We use two approaches to derive the dynamics for well-mixed populations. This approach requires -dynamics. On the one hand, we employ a parameterized full-information. Second, we employ the same revision protocol version of the pairwise comparison protocol and the mean and a version of the mean dynamics for non-well-mixed popula- tions that uses only local information. Furthermore, invariance dynamics for well-mixed populations [4]. This approach properties of the set of allowed population states are analyzed, generates a version of the -dynamics that requires full- and stability of the -equilibria is formally proven. Finally, two information of the game to evolve. On the other hand, we engineering examples based on the −dynamics are presented: use the same modified pairwise comparison protocol and the A control scenario in which noisy measurements should be mean dynamics for non-well-mixed populations [5] to obtain mitigated, and a humanitarian engineering application related to wealth distribution in poor societies. distributed -dynamics that only use local information. Due to the advantages of finding -equilibria instead of I.INTRODUCTION Nash equilibria, -dynamics enlarge the spectrum of ap- plications of population games, which have been widely In game theory, the -equilibrium is a relaxation of the employed for solving optimization and control problems [6], widely known concept of Nash equilibrium [1]. While in a [7]. For instance, since a controller based on -dynamics can Nash equilibrium a player has no incentives to unilaterally deal with imperfect information, it is suitable to be applied change its strategy, in an -equilibrium players can have in situations where the measurements of systems’ outputs incentives to choose other strategies. Nonetheless, these have noise. To illustrate the usefulness of -dynamics, we incentives are not higher than a nonnegative threshold . apply them in two problems: i) The design of a model- There are a couple of issues that motivate the development free controller for water allocation in an m-tanks system of techniques to find a relaxed version of Nash equilibria. that includes noisy measurements; ii) the design of wealth First, -equilibrium is used in stochastic games or games with distribution policies in poor communities. The latter problem imperfect information, where Nash equilibrium is rapidly is discussed in [8]. changing. Under this class of games, it can be desirable The organization of this document is as follows: Section II that players’ decisions converge to a rest point that is presents the preliminary concepts associated with population close to the Nash equilibrium instead of having permanent dynamics, -equilibria, and graphs. Section III introduces the variations of the players’ decisions due to Nash equilibrium deduction of the -dynamics. Afterwards, some properties of changes. Second, there are some cases in which finding a the -dynamics are presented in IV. Section V derives the Nash equilibrium is computationally expensive [2]. In these distributed version of the -dynamics. Moreover, Sections VI cases, convergence to -equilibria reduces the computational and VII present applications of the -dynamics in a control burden. Although algorithms for reaching -equilibria have problem with noisy measurements, and the formulation of a been proposed for strategic games (e.g., see [3]), similar wealth distribution policy, respectively. Finally, Section VIII methods have been largely unexplored for population games. draws come concluding remarks. n G. Obando, J. Barreiro-Gomez, and N. Quijano are with Departa- Notation: Throughout this manuscript, we use R+ to mento de Ingenier´ıa Electrica´ y Electronica,´ Universidad de los Andes, denote the set of vectors in Rn that have nonnegative entries. Carrera 1 No 18A-10, Bogota,´ Colombia ge-oband,j.barreiro135, n n { Similarly, R denotes the set of vectors in R with strictly nquijano @uniandes.edu.co. ++ } positive entries. On the other hand, we use [ ] to denote the J. Barreiro-Gomez is also with the Automatic Control Department, · + Universitat Politecnica` de Catalunya, Institut de Robotica` i Informatica` projection on R++, i.e., [ ]+ = max(0, ). Industrial (CSIC-UPC), Llorens i Artigas, 4-6, 08028 Barcelona, Spain; and · · with the project ECOCIS (Ref. DPI2013-48243-C2-1-R) from the Spanish II.PRELIMINARIES Ministry of Science and Education. This work was supported in part by Colciencias Colfuturo, under A. Population Games Grants 528 and 6172, and in part by the project ALTERNAR, BPIN A population game is a model that describes the strategic 20130001000089, OCAD-Fondo de CTel-SGR–Colombia, under Grant Acuerdo 005 de 2013. Agencia` de Gestio´ d’Ajust Universitaris i de Recerca interaction among individuals in a population. Mathemati- AGAUR also supports J. Barreiro-Gomez. cally, a population game is characterized by: a population mass X R+; a set of strategies = 1, . . . , n ; a III.EPSILON-DYNAMICS population∈ state x = [x , . . . , x ] thatS defines{ the portion} 1 n > A. Revision Protocols and Mean Dynamics of the population mass playing each strategy, i.e., x ∆, where ∈ Nash equilibria and -equilibria predict the outcome of a n population game. Different from these static notions, there n ∆ = x R : xi = X ; (1) exists a framework that is capable to model not only the ∈ + ( i=1 ) outcome of a population game, but also the evolution of X the population state along the time. This framework is and a set of fitness functions , for all fi : ∆ R i = called population dynamics, and its fundamental input is the . Given a population state , the7→ fitness value 1, . . . , n x fi(x) concept of revision protocol. captures the payoff of the individuals playing the ith strategy. A revision protocol is a set of rules that describes the For notational convenience, we define the fitness vector timing and results of the individuals decisions. Specifically, f(x) := [f (x), . . . , f (x)] . 1 n > a revision protocol establishes conditions under which an Under the assumption that the individuals of the popula- individual changes its strategy. Formally, a revision protocol tion are rational, i.e., they try to maximize their payoff, the is defined as follows, expected output of a population game is given by a Nash Definition 3: Revision protocol (adapted from [4]). A equilibrium. revision protocol, denoted by ρ = [ρij], is a map ρ : Definition 1: Nash equilibria (adapted from [4]). x n n n ∗ R ∆ R × that is characterized by the conditional ∆ is a Nash equilibrium if each used strategy entails the∈ × 7→ + switch rates ρij : i, j . Given a population state x and maximum benefit for the players who chose it. Equivalently, { ∈ S} a fitness vector f(x), ρij(f(x), x) captures the switch rate the set of Nash equilibria is given by NE = x∗ ∆ : th th { ∈ from the i strategy to the j strategy. x∗ > 0 f (x∗) f (x∗), i, j = 1, . . . , n . ♦ i ⇒ i ≥ j ∀ } ♦ A population game and a revision protocol define the A relaxed version of the Nash equilibirum concept is the following dynamics, -equilibrium [1], which is defined as follows: Definition 2: -equilibria. Let  R+. The set of - x˙ i = xjρji(f(x), x) xi ρij(f(x), x), i , (2) ∈ − ∀ ∈ S equilibria is given by E = x ∆ : x > 0 f (x) j j  i i X∈S X∈S f (x) , i, j = 1, . . . , n .{ ∈ ⇒ ≥ j − ∀ } ♦ which are called mean dynamics [4]. The first term of (2) While in a Nash equilibrium an individual has no incen- is related to the individuals that switch to the ith strategy tives to unilaterally change its strategy, in an -equilibrium from other ones, while the second term is associated with the the individuals can have incentives to choose other strategies, individuals that change strategy i by another strategy. but these incentives are not higher than a threshold . Notice ∈ S  Summarizing, (2) models the dynamics of a population that, if , an -equilibrium is a Nash equilibrium.  = 0  involved in a strategic game that is using a given revision protocol. In this regard, different revision protocols produce B. Graphs different population dynamics models.

Using the framework developed in [5], the structure of B. Epsilon-Dynamics a population involved in a strategic game can be modeled by means of a graph = , , where the set of nodes Let us define the following revision protocol, is associated with theG available{V E} strategies, i.e., = , V V S ρ (f(x), x) = [f (x) (f (x) + )] , i, j , (3) and the set of edges describes the encounter ij j − i + ∀ ∈ S probability between individualsE ⊂ V × playing V different strategies, where  R+. The constant  acts as a switching threshold. i.e., if (i, j) , then it is possible that an individual playing ∈ th ∈ E That is, an individual playing the i strategy that adopts the strategy i can be matched with an individual playing strategy revision protocol (3) might prefer to keep strategy i even j. In contrast, if (i, j) / , then individuals playing strategies ∈ E if strategy j entails a higher payoff. This individual only i and j cannot encounter each other. Throughout this paper, switches to the jth strategy if the difference between the we only consider undirected graphs, i.e., if (i, j) , then payment perceived by using the jth strategy and the payment (j, i) . ∈ E ∈ E perceived by using its current strategy exceeds the value A well-mixed population is such that the encounter prob- of . Individuals’ self-confidence, change aversion, opinion ability of individuals playing any two strategies from is S inertia, and other kind of biases can be modeled by protocol the same no matter what the strategies are. Therefore, well- (3). mixed population are represented by complete graphs. On Replacing the revision protocol (3) in the mean dynamics the other hand, in non-well-mixed populations, the encounter (2), we obtain the proposed -dynamics, probability between individuals depends on the strategies that these individuals are playing. Hence, non-well-mixed x˙ = x [f (x) (f (x) + )] i j i − j + populations are described by non-complete graphs. In [5], j X∈S (4) non-well-mixed populations are employed to model local x [f (x) (f (x) + )] , i . − i j − i + ∀ ∈ S information structures. j X∈S Notice that, if  = 0, the revision protocol (3) becomes B. Convergence to Epsilon-Equilibria a pairwise comparison protocol [4], and the -dynamics be- Depending on the revision protocol, population dynamics come the classic Smith dynamics. Nevertheless, the addition converge to rational outputs of games. In fact, several classic of a parameter  > 0 provides different characteristics to population dynamics, including best response, Smith, and the behavior of a population that evolves according to the projection dynamics converge to Nash equilibria. In this -dynamics compared to a population under the Smith dy- section, we show that the proposed -dynamics converge to namics. These characteristics are studied in the next section. -equilibria. IV. EPSILON-DYNAMICS PROPERTIES First, let us prove that there is an equivalence between rest A. Simplex Invariance points of the -dynamics and -equilibria of the underlying population game. First, let us show that the -dynamics satisfy simplex invariance, which is a fundamental property of population Proposition 2: Rest points. Let  R+. The population  ∈ dynamics. The invariance of the simplex ∆, defined in (1), state x ∆ is an equilibrium point of the -dynamics (4) if and only∈ if x E . guarantees that the population mass remains constant, and ∈  n Proof: that the population state x belongs to R+ for all time, i.e., the amount of individuals playing any strategy is always “ ”: Let ˆ be the set of strategies that are used at • ⇒ S ⊆ S ˆ  ˆ nonnegative. equilibrium, i.e., = i : xi > 0 ; and let 0 Proposition 1: Simplex invariance. Assume that the ini- be the set of strategiesS { that∈ S are not used} at equilibrium,S ⊆ S tial population state x(0) ∆. If x(t) evolves according to i.e., ˆ = i : x = 0 . Notice that, since x ∆ ∈ S0 { ∈ S i } ∈ the -dynamics (4), then x(t) ∆, for all t 0. by assumption, then ˆ ˆ0 = . Proof: Notice that x(t)∈ ∆ if x(t≥) satisfies two First, let us investigateS ∪ theS characteristicsS of the state ∈ conditions (cf. Equation (1)): i) mass conservation, i.e., variables related to ˆ0. According to (4), we have that ; and ii) nonnegativeness, i.e., ,   S  ˆ i xi(t) = X xi(t) 0 j ˆ xj [fi(x ) (fj(x ) + )]+ = 0, i 0. Thus, i ∈S . Therefore, our proof is divided into two steps:≥ ∈S − ∀ ∈ S P∀ ∈ S P   ˆ ˆ 1. Mass conservation: Since i xi(0) = X by as- fi(x ) fj(x ) + , i 0 and j . (5) sumption, to prove mass conservation∈S it is sufficient to show ≤ ∀ ∈ S ∈ S P Now, let us characterize the state variables related to ˆ. that the quantity i xi(t) is positively invariant. Notice ∈S Without loss of generality, assume that ˆ = n, n S that i x˙ i = j xj i [fi(x) (fj(x) + )]+ ∈S P ∈S ∈S − − 1, . . . , k + 1, k , and f (x) f (xS) { − i xi j [fj(x) (fi(x) + )]+. Rewriting the n n 1 ∈SP ∈S P − P  }  ≥ − ≥ · · · ≥ right-hand side of this mathematical equation, we obtain fk+1(x ) fk(x ). Once this is made, we proceed by P P contradiction.≥ Assume that f (x) > f (x) + . Under that i x˙ i = j xj i [fi(x) (fj(x) + )]+ n k ∈S ∈S ∈S − − this scenario, we can use the property stated in (5) j xj i [fi(x) (fj(x) + )]+. Thus, i x˙ i = 0. ∈SP ∈S P − P ∈S to obtain    , and This implies that i xi(t) is positively invariant. Hence, j xj [fn(x ) (fj(x ) + )]+ > 0 P P ∈S P ∈S  − . Notice that i xi(t) = X, for all t 0. xn j [fj(x ) (fn(x ) + )]+ = 0 ∈S P ≥ this implies∈S P that the− right hand side of (4) is different P P  2. Nonnegativeness: Let us prove that xi(t) 0, for all from zero, which is a contradiction since x is a rest ≥   i , and for all t 0. We proceed by contradiction. point by assumption. Therefore, fk(x ) +  fn(x ). ∈ S ≥ n ≥ Since the initial state x(0) belongs to R+, let us as- A consequence of this fact is that sume that there exists a set 0 associated with the S ⊆ S first state variables that become negative, i.e., we assume max(fi) min(fi) . (6) i ˆ − i ˆ ≤ that, for some ta > 0, the following conditions hold: i) ∈S ∈S  x (t ) < 0 : k 0 ; and ii) x (t) 0 : k 0 , for Combining (5) and (6) we can conclude that x E. { k a ∈ S } { k ≥ ∈ S\S } ∈ all t < ta.   Notice that the above assumption implies that there exist “ ”: Since x E, then fi(x )  • ⇐  ∈  − ≤ tb < ta and l 0 such that xl(tb) = 0, x˙ l(tb) < fj(x ), i, j such that xj > 0. Thus, ∈ S ∀  ∈ S  0, and xj(tb) 0, for all j . This is a contra- j xj [fi(x ) (fj(x ) + )]+ = 0, i . More- ≥ ∈ S ∈S −   ∀ ∈ S diction since, according to the -dynamics (4), x˙ l(tb) = over, notice that xi j [fj(x ) (fi(x ) + )]+ = P ∈S −  j xj(tb)[fl(x(tb)) (fj(x(tb)) + )]+ 0. 0, i . This equality holds since, if xi > 0, then ∈S − ≥ ∀ ∈ S  P If the population mass is associated with a physical quan- fi(x ) fj(x ) . Therefore, the right hand side of P ≥ −  tity (e.g., if the -dynamics are used to model an engineering (4) is equal to zero. This implies that x is a rest point problem), then simplex invariance plays a fundamental role of the -dynamics. for guaranteeing the satisfaction of certain constraints on that Having shown that any equilibrium point of the - quantity. For instance, if the population mass is associated dynamics is an -equilibrium, let us make some assumptions with the energy applied by a control system, simplex invari- on the characteristics of the population game that guarantee ance guarantees the limitation of that energy (see, e.g., [9], that the -dynamics converge to the set of -equilibria. [10], [7], [11]). Benefits of simplex invariance are exploited Assumption 1: The fitness functions satisfy the following in some applications that are discussed later in this paper. conditions, for all i , and x ∆: ∈ S ∈ f only depends on the population playing the ith of scenarios, full information dependency is undesirable (due • i strategy, i.e., xi (when this is the case, we write fi(xi) to, for instance, privacy issues, scale of systems, limitations to make the dependency explicit). on the communication infrastructure, etc.). Therefore, we use f is differentiable. Moreover, its derivative satisfies the procedure proposed in [5] to relax the full information • i dfi (x ) < 0. dependency of the -dynamics. The core of the framework dxi i Assumption 1 is not very restrictive. Notice that this as-♦ developed in [5] lies on the mean dynamics for non-well- sumption implies that the fitness functions are monotonically mixed populations. Since the structure of a non-well-mixed decreasing, which is a common characteristic in several population can be described by an undirected graph = scenarios, e.g., coordination games [4], biological models , , its mean dynamics are given by G [12], congestion games [13], and so forth. {S E} Next, we provide our main result on convergence of the x˙ = x ρ (f(x), x) x ρ (f(x), x), i , i j ji − i ij ∀ ∈ S -dynamics. j i j i X∈N X∈N Theorem 1: Convergence to -equilibria. Let Assump- (10) where is the set of strategies that can interact with the ith tion 1 hold, and let E be a subset of int∆, where int∆ := i n N n is the interior of the simplex . strategy following the graph , i.e., i = j :(i, j) x R++ : i=1 xi = X ∆ G N { ∈ S ∈ The∈ proposed -dynamics (4) asymptotically converge to the . Replacing the revision protocol (3) in the mean-dynamics  P forE} non-well-mixed populations (10), we obtain a version of set of -equilibria E of the underlying population game. Proof: First, notice that since the set of -equilibria the -dynamics that only uses local information to evolve. belongs to the interior of the simplex ∆ by assumption, These distributed -dynamics are described as follows, this set is characterized by E = x ∆ : maxi fi(xi) { ∈ − x˙ i = xj [fi(x) (fj(x) + )] mini fi(xi)  . To study the stability properties of E, let − + ≤ } j i us consider the following Lyapunov function candidate X∈N (11) xi [fj(x) (fi(x) + )]+, i . V (x) = max fi(xi) min fi(xi). (7) − − ∀ ∈ S i − i j i ∈S ∈S X∈N Clearly, V (x) 0, for all x ∆. Indeed, V (x) = 0 Let us show that, although the distributed -dynamics do not ≥ ∈  only at the Nash equilibrium, i.e., at x∗ ∆ such that preserve all the appealing properties of (4), they still exhibit ∈ fi(xi∗) = fj(xj∗), for all i, j . The derivative of (7) along a desirable behavior. the trajectories of the -dynamics∈ S (4) is given by. V˙ (x) = V˙ (x) V˙ (x), A. Simplex Invariance M − m where VM (x) := maxi fi(xi), and Vm(x) := mini fi(xi). Distributed -dynamics satisfies simplex invariance, as it Let us define the sets ΩM = i : fi(xi) = VM (x) , and is stated in the following proposition. Ω = i : f (x ) = V (x) . Using{ the Clarke’s generalized} Proposition 3: Simplex invariance. Assume that the ini- m { i i m } gradient [14], notice that V˙M (x) and V˙m(x) are given by tial population state x(0) ∆. If x(t) evolves according to the distributed -dynamics∈ (11), then x(t) ∆, for all t 0. ˙ dfi ∈ ≥ VM (x) = λi (xi)x ˙ i : λi R+, λi = 1 , Proof: Let = [aij] be the adjacency matrix related dxi ∈ A (i ΩM i ΩM ) to the graph = , , i.e., aij = 1 if (i, j) , and ∈X ∈X (8) a = 0 if (i,G j) / {S. WeE} recall that only undirected∈E graphs ij ∈ E ˙ dfi are considered. Therefore, aij = aji. Vm(x) = λi (xi)x ˙ i : λi R+, λi = 1 . dxi ∈ 1. Mass conservation: Since xi(0) = X by (i Ωm i Ωm ) i ∈X ∈X (9) assumption, it is sufficient to show∈S that the quantity dfi P If i ΩM and x / E, then (xi)x ˙ i = i xi(t) is positively invariant under (11). Notice that ∈ ∈ dxi ∈S dfi x˙ = a x [f (x) (f (x) + )] (xi) xj [fi(x) (fj(x) + )] < 0 since i i j i ij j i j + dxi j + P ∈S ∈S ∈S − − dfi ∈S − ajixi [fj(x) (fi(x) + )] . Since aij = aji, (xi) < 0 by assumption . Therefore, V˙M < 0, for i j + dxi P Pthe∈S right-hand∈S P side ofP the− latter equation can be rewritten all x / E. Furthermore, if i Ωm and x / E, then P P dfi ∈ dfi
∈ ∈ as x˙ i = aijxj [fi(x) (fj(x) + )] (xi)x ˙ i = xi [fj(x) (fi(x) + )] > 0. i j i + dxi − dxi j − + ∈S ∈S ∈S − − ˙ ∈S ˙ j i aijxj [fi(x) (fj(x) + )]+. This implies that Hence, Vm > 0, for all x / E. Thus, V (x) < 0, for P∈S ∈S P P − P ∈ i x˙ i = 0. Thus, i xi(t) is positively invariant. In all i / E. Additionally, notice that E is invariant under P ∈S P ∈S ∈ ˙ conclusion, i xi(t) = X, for all t 0. the -dynamics (cf. Proposition 2) and V (x) = 0, for all P ∈S P ≥ x E. Therefore, according to the LaSalle’s invariance P 2. Nonnegativeness: To prove that x (t) 0, for all i , principle∈ [15], we can conclude that every solution of (4) i and for all t 0. We can employ the same≥ arguments∈ used S converges to E as t + . ≥ → ∞ in the proof of Proposition 1. V. DISTRIBUTED EPSILON-DYNAMICS Notice that the statement of Proposition 3 does not im- The -dynamics described in (4) require full information to pose special constraints on the graph . Therefore, simplex evolve. This fact is because they are derived from the mean- invariance is guaranteed under any populationG structure that dynamics for well-mixed populations (2). In a large number can be described by an undirected graph. m x X B. Convergence Analysis i=1 i ≤ Given a population structure described by the undirected P x1 x2 x3 x4 ··· xm graph = , , the equilibrium points x∗ of the dis- h1 h2 h3 h4 hm tributedG-dynamics{S E} are characterized by the following set, ··· qout,1 qout,2 qout,3 qout,4 qout,m ˆ E = x∗ ∆ : xi∗ > 0 fi(xi∗) fj(xi∗) , j i . { ∈ ⇒ ≥ − ∀ ∈ N(12)} Fig. 1. Case study composed by m-tanks. Therefore, although any -equilibrium is a rest point of the distributed -dynamics, a rest point of these dynamics is not consider that there is an additive noise in the measurements necessarily an -equilibrium (unless i = , for all i , N S ∈ S of each tank level. This noise is denoted by d [ d d¯]. i.e, unless the graph is complete). This is the main difference i ∈ − The dynamics of the ith tank are given by h˙ = x K h , of the distributed -dynamics compared to the -dynamics i i− i i that use full information (4). where hi is the water level, xi is the inflow, and Ki is a If is connected, then we can make a better characteriza- constant factor characterizing the outflow. G We compare the performance of the closed-loop system tion of the equilibrium set Eˆ. To do this, let d(i, j) be the distance between nodes i and j of (the distance between when the tanks inflows are controlled via the Smith dynamics two nodes is equal to the number ofG edges in a shortest path (i.e., the -dynamics with  = 0) and the -dynamics with that connects these nodes). Moreover, let D be the diameter  > 0. This is done by taking as strategies the tanks, and as population mass the available water flow X. Hence, xi of the graph , which is defined as D = maxi,j d(i, j). G ∈S (i.e., the water inflow allocated to the ith tank) corresponds For connected graphs, it can be shown that Eˆ ED, where ⊆ to the proportion of the players mass that choose the ith ED is the set of D-equilibria of the population game, i.e., strategy. We propose a case study composed by m = 10 E = x ∆ : x > 0 f (x) f (x) D, i, j tanks, with a constant available water flow given by X = 25, D { ∈ i ⇒ i ≥ j − ∀ ∈ S} and noise for each measurement in the range [ 0.5, 0.5]. Next, we provide a convergence result for the distributed We assume that there is a local controller per valve,− which -dynamics. guarantees that each inflow reaches the reference xi imposed Proposition 4: Convergence. Let Assumption 1 hold, and by the population dynamics. The limited available water flow ˆ let E be a subset of int∆. If the undirected graph that establishes a constraint over all the inflows. This constraint describes the population structure is connected, thenG the is managed by adding a nonnegative slack variable xm+1, distributed -dynamics asymptotically converge to the set i.e., we add an strategy , and obtain according to the ˆ (m + 1) E ED . m+1 ⊆ simplex invariance that i=1 xi = X (cf. Proposition 1). Proof: ˆ m In order to study the stability properties of E, Thus, x X. we use the Lyapunov function candidate defined in i=1 i ≤ V (x) The fitness function associatedP with the ith tank is selected (7), and employ the same procedure described in the proof P m m as the level error, i.e., fi = r hi , where hi = hi +di is the of Theorem 1 to obtain that V˙ (x) = V˙M (x) V˙m(x), where − ˙ ˙ − measured level. Thus, more water inflow is allocated to those VM (x) and Vm(x) are given in (8) and (9), respectively. tanks with higher error. Figure 2(a) shows the evolution of dfi Notice that, if i ΩM , then (xi)x ˙ i = ∈ dxi the tanks levels controlled via the Smith dynamics. Notice dfi (x ) x [f (x) (f (x) + )] 0. Therefore, dxi i j i j i j + that the disturbances in the measurements affect the values of ∈N − ≤ dfi V˙M 0. Furthermore, if i Ωm, then (xi)x ˙ i = the fitness functions. Consequently, the performance of this P dxi dfi ≤ ∈ xi [fj(x) (fi(x) + )] 0. Hence, V˙m 0. controller exhibits permanent oscillations that could have a − dxi j i − + ≥ ≥ Thus, V˙ (x)∈N 0. Additionally, since is connected by negative effect on the systems actuators. P ≤ G assumption, V˙ (x) = 0 only if x Eˆ. Therefore, we can 3 3 use the LaSalle’s invariance principle∈ [15] to conclude that every solution of (11) converges to Eˆ as t + . 2 2 Under the conditions given in the statement→ of Proposition∞ 1 1 height [m] height [m] 4, a simpler characterization of the set ED is as follows, 0 0 0 100 200 300 400 500 0 100 200 300 400 500 ED = x ∆ : max fi(x) min fi(x) D (13) time [s] time [s] ∈ i − i ≤  ∈S ∈S  (a) (b) In the following sections, we illustrate the applicability of Fig. 2. Evolution of the tanks levels under: (a) Smith dynamics ( = 0), the proposed -dynamics in some engineering problems. and (b) -dynamics ( = 0.5).

VI.CONTROLWITH NOISY MEASUREMENTS In order to mitigate the problem caused by the noisy Figure 1 shows a case study composed by m tanks. The measurements, we implement the -dynamics. The noise control objective is to maintain all the tank levels at the same included in the measurements is neglected as long as the reference r, while a physical constraint given by a limited value of  d¯. This is because the possible increment of the amount of water flow X must be satisfied. Moreover, we fitness functions≥ do not overtake the threshold that makes the 50 50 players (water inflow) want to change them strategies (tanks) 40 40 when they are in an -equilibrium. Therefore, the distribution 30 30 of population mass among the strategies (i.e., allocation of 20 20 wealth [$] wealth [$] the water flow among the tanks) remains constant. Figure 10 10 0 0 2(b) shows the performance of the -dynamics with  = 0.5. 4 2 0 2 3 1 1 10− 10− 10 10 10− 10− 10 It can be seen the significant improvement and the mitigation time [days] time [days] of the noise under these dynamics. (a) (b) Fig. 3. Evolution of the wealth of each individual under: (a) Distributed - VII.COMMUNITY WEALTH DISTRIBUTION dynamics ( = 0.02), and (b) -dynamics with full information ( = 0.02). Consider a community composed by n individuals. Let th xi > 0 be the wealth of the i individual. Moreover, condition of Nash equilibria). Stability of the -equilibrium assume that the ith individual can only interact with a and invariance of the set of allowed population states (sim- set of individuals in the community (its neighbors). These plex) have been formally proven. Furthermore, a case study “allowed” interactions are described by an undirected graph with imperfect information (a more realistic scenario than = , , where: is the set of nodes, which are related those scenarios considered in other related works) has shown toG the{V individualsE} ofV the community, i.e., = 1, . . . , n ; the enhancement of the closed-loop performance with an - and is the set of edges, which are relatedV to the{ interaction} dynamics-based controller, which is capable to mitigate noisy linksE between individuals, i.e., if (i, j) , then the ith measurements in comparison to other population-dynamics- individual can interact with the jth individual.∈ E Our objective based controllers that converge to a Nash equilibrium. Be- is to design a wealth distribution policy that mitigates the sides, we have taken advantage of the properties of the - wealth gap in the considered community. dynamics to develop a model of wealth distribution in poor Previous works have employed classic consensus protocols communities. [8]. However, the scenario in which all the individuals ACKNOWLEDGMENTS achieve the same wealth is not realistic. Hence, instead The authors would like to thank Professor K. M. Passino of using consensus algorithms, we propose to apply the for his valuable inputs. distributed -dynamics with fi(xi) = xi, for all i . REFERENCES Notice that this is equivalent to use the− revision protocol∈ S [1] I. Menache and A. Ozdaglar, Network games: Theory, models, and ρ = [x (x + )] , i.e., the ith individual only donates ij i − j + dynamics, vol. 4. Morgan & Claypool Publishers, 2011. part of his/her wealth to the jth individual if the wealth gap [2] C. Daskalakis, P. Goldberg, and C. Papadimitriou, “The complexity between these two people is greater than a threshold  > 0. of computing a Nash equilibrium,” pp. 71–78, ACM Press, 2006. [3] H. Bosse, J. Byrka, and E. Markakis, “New algorithms for approximate In order to illustrate the performance of the proposed nash equilibria in bimatrix games,” Theoretical Computer Science, method, let us consider a community composed by 100 indi- vol. 411, pp. 164–173, 2010. viduals, where interactions among individuals are described [4] W. H. Sandholm, Population games and evolutionary dynamics. Cam- bridge, Mass. MIT Press, 2010. by a path graph. Initially, the wealths of individuals are [5] J. Barreiro-Gomez, G. Obando, and N. Quijano, “Distributed pop- distributed as follows: x1(0) = $0.5, x2(0) = $1, x3(0) = ulation dynamics: Optimization and control applications,” IEEE Transactions on Systems, Man, and Cybernetics: Systems. DOI: $1.5, ... , x99(0) = $49.5, x100 = $50 (i.e., xi(0) = $0.5i). 10.1109/TSMC.2016.2523934, 2016. Figure 3(a) shows the evolution of the wealth of each [6] E. Ramirez-Llanos and N. Quijano, “A population dynamics approach individual if the community applies a wealth distribution for the water distribution problem,” International Journal of Control, policy based on the -dynamics (11) with  = $0.02. Notice vol. 83, no. 9, pp. 1947–1964, 2010. [7] G. Obando, A. Pantoja, and N. Quijano, “Building Temperature Con- that the wealth gap decreases along the time and the wealth trol based on Population Dynamics,” IEEE Transactions on Control distribution reaches an equilibrium x∗ that belongs to the Systems Technology., vol. 22, no. 1, pp. 404–412, 2014. [8] K. Passino, Humanitarian engineering. Creating technologies that help set ED (cf. Equation (13)), where D = 100. This fact is people. Bede Publishing, Columbus, Ohio, 2nd ed., 2015. because maxi xi∗ mini xi∗ < $2, i.e., the wealth gap [9] J. I. Poveda and N. Quijano, “Shahshahani gradient-like extremum ∈S − ∈S decreases to a value that is lower than D. On the other seeking,” Automatica, vol. 58, pp. 51–59, 2015. hand, Figure 3(b) shows the evolution of the individuals’ [10] J. Barreiro-Gomez, N. Quijano, and C. Ocampo-Martinez, “Con- strained distributed optimization: A population dynamics approach,” wealth under the same scenario described above, but using Automatica. DOI: 10.1016/j.automatica.2016.02.004, 2016. a complete graph, i.e., individuals have full information. In [11] A. Pantoja and N. Quijano, “A population dynamics approach for the this case, evolution of the wealth distribution is faster, and dispatch of distributed generators,” IEEE Transactions on Industrial Electronics, vol. 58, pp. 4559–4567, 2011. the wealth gap decreases to a value equal to , i.e., an - [12] N. Quijano and K. M. Passino, “The ideal free distribution: theory equilibrium is reached. and engineering application,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 1, pp. 154–165, VIII.CONCLUSIONS 2007. [13] H. Tembine, E. Altman, R. El-Azouzi, and Y. Hayel, “Evolutionary We have deduced a centralized and distributed class of games in wireless networks,” IEEE Transactions on Systems, Man, population dynamics parameterized by a scalar term  as and Cybernetics, Part B: Cybernetics, vol. 40, no. 3, pp. 634–646, 2010. an extension of the classic pairwise comparison revision [14] F. Clarke, Optimization and nonsmooth analysis, vol. 5. Siam, 1990. protocol and the mean dynamics. As a result, the -dynamics [15] H. K. Khalil, Nonlinear Systems. Upper Saddle River, NJ: Prentice have been obtained, which converge to -equilibria (a relaxed Hall, 2002.