Networks of conforming or nonconforming individuals tend to reach satisfactory decisions

Pouria Ramazia,1, James Riehla, and Ming Caoa,1

aEngineering and Technology Institute Groningen (ENTEG), Faculty of Mathematics and Natural Sciences, University of Groningen, 9747 AG Groningen, The Netherlands

Edited by Kenneth W. Wachter, University of California, Berkeley, Berkeley, CA, and approved September 20, 2016 (received for review June 23, 2016) Binary decisions of agents coupled in networks can often be classified networks (11). The idea behind best-response dynamics is sim- into two types: “coordination,” where an agent takes an action if ple: in a network of interacting agents, each agent takes the enough neighbors are using that action, as in the spread of social action that results in the best cumulative outcome against its norms, innovations, and viral epidemics, and “anticoordination,” neighbors. This framework is frequently considered in a game- where too many neighbors taking a particular action causes an agent theoretic context (12–15), but its impact is further broadened by to take the opposite action, as in traffic congestion, crowd disper- the fact that when there are two available actions for each agent, sion, and division of labor. Both of these cases can be modeled using it is equivalent to a “linear-threshold model” (16). This comes linear-threshold–based dynamics, and a fundamental question is with an alternative yet also intuitive interpretation: when a suf- whether the individuals in such networks are likely to reach decisions ficient fraction of my neighbors are taking action A, I will also with which they are satisfied. We show that, in the coordination take action A. The reverse is also possible: if too many of my case, and perhaps more surprisingly, also in the anticoordination neighbors are taking action A, I will switch to action B. We call case, the agents will indeed always tend to reach satisfactory deci- agents of the former type “coordinating” and the latter, “anti- sions, that is, the network will almost surely reach an equilibrium coordinating,” according to the standard coordination and anti- state. This holds for every network topology and every distribution coordination games, which are also sometimes referred to as of thresholds, for both asynchronous and partially synchronous de- games of strategic complements and strategic substitutes, re- cision-making updates. These results reveal that irregular network spectively (17). These switches may be considered voluntary as in topology, population heterogeneity, and partial synchrony are not changing an opinion, preference, or habit, or involuntary, as SOCIAL SCIENCES sufficient to cause cycles or nonconvergence in linear-threshold dy- getting infected by virus or defaulting on a loan. In general, the namics; rather, other factors such as imitation or the coexistence of thresholds of the agents need not be the same, and this is equiv- coordinating and anticoordinating agents must play a role. alent to agents having asymmetric payoffs in the corresponding matrix game. Depending on the application, this can represent evolutionary | network games | best-response dynamics | differences in the willingness to follow a crowd, susceptibility to linear-threshold model | equilibrium convergence infection, perceived value of interactions, and many other indi- vidual characteristics (18). Variation in thresholds can thus be he sharing of resources, division of labor, and dispersion of thought of as heterogeneity of a population. Tcrowds represent a few examples of collective behaviors that The linear-threshold model was first motivated and dis- can emerge on networks of interacting agents when the adoption cussed by Mark Granovetter (18) in the context of fully con- of a particular action by too many individuals deters others from nected networks, with an example of how the incitement of riots ’ adopting that action. On the other hand, when individuals taking in crowds depends critically on the distribution of individual an action makes it more likely that others will adopt that action, thresholds that will cause each person to join the riot. Since behaviors such as the spread of social norms, technological in- then, a rich literature has developed around questions related to novations, and viral infections can occur. In either case, various stability and convergence of best-response dynamics in matrix factors are thought to contribute to the emergence of such col- lective behaviors, including network topology, interaction payoffs, Significance update rules, and population diversity (1–3). In particular, when agents can take one of a finite number of states at a given time, whether by voluntary decisions or involuntary reactions, the Many real-life decisions where one out of two actions must be fundamental questions include (i) whether the agents will eventu- chosen can be modeled on networks consisting of individuals ally settle on a fixed state/action, (ii) which distribution of actions who are either coordinating, that is, take an action only if the network will converge to, and (iii) how long it will take for the sufficient neighbors are also doing so, or anticoordinating, that network to converge. Although most of the literature focuses on the is, take an action only if too many neighbors are doing the second and especially third questions, particularly for homogeneous opposite. It is not yet known whether such networks tend to reach a state where every individual is satisfied with his de- populations where individuals share the same utility function (4–7), cision. We show that indeed any network of coordinating, and the first question is perhaps more fundamental because it deals with any network of anticoordinating individuals always reaches a convergence at the agent level. Undoubtedly, this action settling is satisfactory state, regardless of how they are connected, how crucial to a wide range of decision-making populations, from col- different their preferences are, and how many simultaneous lective nest selection by ants (8) to stabilization of financial markets decisions are made over time. (9). Moreover, without establishing such convergence, answers to

questions ii and iii cannot be used to design policies at the local or Author contributions: P.R., J.R., and M.C. designed research, performed research, and neighborhood level (10), which may help to improve cost efficiency wrote the paper. by targeting influential nodes in a complex network. The authors declare no conflict of interest. “ ” Best-response dynamics is one of the most widely used This article is a PNAS Direct Submission. models to study the types of problems mentioned above; this is 1To whom correspondence may be addressed. Email: [email protected] or m.cao@ due in part to its rational nature as well as its broad applicability. rug.nl. Indeed, recent experimental studies have shown that humans This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. often use myopic best responses when making decisions in social 1073/pnas.1610244113/-/DCSupplemental.

www.pnas.org/cgi/doi/10.1073/pnas.1610244113 PNAS Early Edition | 1of6 Downloaded by guest on September 30, 2021 games on well-mixed populations (19, 20). Convergence of ho- k, and denote the number of neighbors of agent i playing A and B A B mogeneous well-mixed populations under more general non- at time k by ni ðkÞ and ni ðkÞ, respectively. When there is no ambi- matrix games has been established in ref. 21 in the context of guity, we may sometimes omit the time k for compactness of nota- congestion games, using potential functions. The convergence tion. The total payoffs to each agent i at time k are accumulated A + B properties of networks with arbitrary structure has also been over all neighbors, and are therefore equal to aini ðkÞ bini ðkÞ = A + B = investigated under various conditions, particularly in the ho- when xiðkÞ A,orcini ðkÞ dini ðkÞ when xiðkÞ B. mogeneous (symmetric) case (22–24). Recently in ref. 17, a In asynchronous (myopic) best-response dynamics, one agent at combination of mean-field approximations and simulations a time becomes active and chooses a single action to play against were used to show that synchronous best-response dynamics all neighbors. The active agent at time k updates at time k + 1to in symmetric coordination or anticoordination games tend to the strategy that achieves the highest total payoff, that is, is the always converge to Nash equilibria. However, in the case of best response, against the strategies of its neighbors at time k: asymmetric coordination games, it was shown in ref. 25 that 8 < A + B > A + B synchronous best-response dynamics may not converge to a A,ifaini bini cini dini + = A + B < A + B single action but rather to a cycle of alternating actions. Sym- xiðk 1Þ : B,ifaini bini cini dini . A + B = A + B metry thus seems to be a significant factor in the convergence of zi,ifaini bini cini dini the threshold model, but we will show that asynchrony perhaps plays an even more important role, because regardless of the In the case that strategies A and B result in equal payoffs, both symmetry, the network always converges in the asynchronous strategies are best responses and we use the notation zi, which is case. In addition to making the convergence more likely, com- defined to be A, B,orxiðkÞ, to allow for several possibilities for pared with synchronous models, asynchronous dynamics can this equality case. Note that we do not require all agents to have provide a more realistic model of the timeline over which the same zi. That is, when both A and B are best responses, some independent agents make decisions and receive information, agents may choose A, others may choose B, and others may keep and they are particularly suitable when the payoff dynamics can their current strategy; however, the agents cannot change their be thought of as fast compared with the update dynamics. For choice of zi over time. example, decisions on matters such as which product to buy, It is convenient to rewrite these dynamics in terms of the number which political party to vote for, or which traffic route to take of neighbors playing each strategy. Let degi denote the total may be the culmination of many individual interactions. Nev- number of neighbors of agent i. We can simplify the conditions B = − A ertheless, our results do not require that only one agent can above by using the fact that ni degi ni and rearranging terms: update at a time; in fact, we show that even small asynchronous A + − A > A + − A perturbations to fully synchronous dynamics lead to equilibrium aini bi degi ni cini di degi ni A − + − > − convergence. ni ðai ci di biÞ degiðdi biÞ δ A > γ In this paper, we show that every network consisting of anti- ini idegi, coordinating agents with asynchronous best-response dynamics δ d − + − γ d − “<” will eventually reach an equilibrium state, even if each agent has where i ai ci di bi and i di bi. The cases of and a different threshold. Moreover, we show that the same result “=” can be handled similarly. Consider the case when δi ≠ 0, and τ dγ =δ “ ” holds for networks of coordinating agents. As a corollary, we let i i i denote a threshold for agent i. Depending on the establish the existence of pure-strategy Nash equilibria in both sign of δi, we have two possible types of best-response update cases for arbitrary networks and arbitrary payoffs. On the rules. If δi > 0, the update rule is given by the following: question of convergence time, we show that, in such networks, 8 < A > τ the total number of strategy switches is no greater than six times A if ni ðkÞ i degi + = A < τ [1] the number of edges in the network. In the case of partial synchrony, xiðk 1Þ : B if ni ðkÞ i degi . A = τ when a random number of agents can update simultaneously, zi if ni ðkÞ i degi we show that the network still almost surely reaches an equi- librium. It follows that irregular network topology, population These agents switch to strategy A if a sufficient fraction of neigh- heterogeneity, and synchrony of decisions between two or more bors are using that strategy, and likewise for strategy B. On the agents (as long as random asynchronous updating is not com- other hand, for δi < 0, if a sufficient fraction of neighbors are pletely excluded) are not sufficient to cause nonconvergence or playing A, the agent will switch to B, and vice versa. This update cycles in best-response dynamics; rather, possible causes include rule is given by the following: the occasional use of non–best-response strategies, randomiza- 8 < A < τ tion, or a mixture of coordinating and anticoordinating agents. A if ni ðkÞ i degi + = A > τ [2] Indeed, we provide a small example demonstrating the pos- xiðk 1Þ : B if ni ðkÞ i degi . A = τ sibility of cycles in networks containing both coordinating and zi if ni ðkÞ i degi anticoordinating agents. Inthecasethatτi ∉ ½0,1, it is straightforward to show that there Asynchronous Best-Response Dynamics exists a strictly dominant strategy, and the update rule Eq. 1 or Consider an undirected network G = ðV, EÞ where the nodes 2 is equivalent to one in which τi ∈ f0,1g and zi ∈ fA, Bg.The V = f1, ..., ng correspond to agents and each edge in the set same holds for when δi = 0. Agents for which δi ≥ 0arecalled E ⊆ V × V represents a 2-player game between neighboring agents. coordinating and can be modeled by Eq. 1.Agentsforwhich Each agent i ∈ V chooses pure strategies from a binary set Sd δi ≤ 0 are called anticoordinating and can be modeled by Eq. 2. fA, Bg and receives a payoff upon completion of game according Therefore, every agent can be described as a coordinating or an to the matrix: anticoordinating agent (or both). Let ΓdðG, τ, f+, −, ± gÞ denote a “network game,” which AB consists of the network G, a vector of agent thresholds τ = ⊤ A ai bi ðτ1, ..., τnÞ , and one of +, −,or±, corresponding to the cases of , ai, bi, ci, di ∈ R. B ci di all coordinating, all anticoordinating, or a mixture of both types of agents, respectively. The dynamics in Eqs. 1 and 2 are in the Thedynamicstakeplaceoverasequenceofdiscretetime form of the standard linear-threshold model (18). An equilib- k = 0,1,2, .... Let xiðkÞ ∈ S denote the strategy of agent i at time rium state in the threshold model is a state in which the number

2of6 | www.pnas.org/cgi/doi/10.1073/pnas.1610244113 Ramazi et al. Downloaded by guest on September 30, 2021 of A-neighbors of each agent does not violate the threshold that will cause a switch to B. Similarly, when playing B, the agent would cause them to change strategies. For example, in a net- needs at least 2 A-neighbors to remain playing B, whereas 1 or ^ work of anticoordinating agents in which zi = B for all i, this fewer will cause a switch to A. Now consider an agent i whose ∈ = A < τ = means that for each agent i V, xiðkÞ A implies ni ðkÞ idegi threshold is 1 2 but who has one additional neighbor who always = A ≥ τ and xiðkÞ B implies ni ðkÞ idegi. Note that this notion of plays A, for a total of 5 neighbors, as shown in Fig. 1B.When equilibrium is equivalent to a pure-strategy in playing A, this agent can tolerate up to 2 A-neighbors before the corresponding network game. switching to B, and as a B-agent needs at least 3 A-neighbors, whereas 2 or fewer will cause a switch to A.Notice,however, Convergence Results that with respect to the original 4 neighbors, the dynamics of We investigate the equilibrium convergence properties of the agents i and ^i are indistinguishable. It turns out that whenever 1 2 agent-based threshold models in Eqs. and that we defined in τi < 1=2, by adding a sufficient number of fixed A-neighbors, the previous section. we can always construct a dual node^i with threshold 1=2 whose Before providing the main results, we precisely define the dynamics are equivalent to the dynamics of the original node i. nature of the asynchronous dynamics. We require only that at Moreover, if τi > 1=2, we can achieve the same result by adding any given time step, each agent is guaranteed to be active at fixed B-neighbors. To ensure that the added nodes do not change k some finite future time.∞ Let i denote the agent who is active strategies, we simply add two opposite strategy neighbors to k at time k and let ði Þk=0 denote a sequence of active agents. We these nodes (Fig. 1C). It is then straightforward to show that say that such a sequence is “persistent” if for every agent j ∈ V the strategies of all added nodes remain constant. We now for- and every time k ≥ 0, there exists some finite later time k′ > k at ′ malize this argument for arbitrary networks of anticoordinat- which agent j is again active (ik = j). ing agents, using some techniques similar to those that have already proven useful in studying the convergence of synchronous net- Assumption 1. Every activation sequence driving the dynamics in works (25). 1 2 ^ Eq. or Eq. is persistent. We define the “augmented network game” Γ^dðG, ð1=2Þ1, − Þ Remark 1. ^ ^ In stochastic settings, Assumption 1 holds almost surely based on Γ as follows. Let G = ðV^, EÞ.Definea“V-block” whenever agents activate infinitely many times with probability ^ as a triplet of nodes fv1, v2, v3g ⊆ V along with the edges one, for example, if each agent activates at a rate determined by ^ ffv1, v2g, fv1, v3gg ⊆ E. For each agent i ∈ V, we introduce a “dual a Poisson process. ” ^∈ ^ = agent i V with the same initial strategy, that is, x^ið0Þ xið0Þ, We divide the convergence analysis into two main parts cor- ^ SOCIAL SCIENCES and with z^ = z . Corresponding to each dual agent i, there are m i i ^ i responding to the cases of anticoordinating and coordinating number of V-blocks in G such that the v -node of each block is agents. In what follows, we use 1 to denote the n-dimensional ^ 1 connected to i, with mi being defined as follows: if τi = 1=2, then vector containing all ones. We refer the reader to SI Appendix for m = 0; otherwise, m depends on which one of the following the detailed proofs of the results. i i three conditions on τi holds: 8 All Agents Are Anticoordinating − τ τ ∈ Z > jð1 2 iÞdegij i degi Theorem 1. Every network of anticoordinating agents who update > <> + asynchronously under Assumption 1 will reach an equilibrium state − − ∃ ∈ Z r < τ < r 1 = jdegi r 1j r 2 : i degi in finite time. mi , > 2 2 The sketch of the proof is as follows. We begin by showing > :> r r + 1 that an arbitrary network game Γ = ðG, ð1=2Þ1, − Þ consisting of jdeg − rj ∃r ∈ 2Z + 1: ≤ τi deg < i 2 i 2 anticoordinating agents in which the threshold of each agent is = 1 2 will reach an equilibrium in finite time. Then we extend the where 2Z and 2Z + 1 denote the set of even and odd numbers, result to a heterogeneous-threshold network game Γ = ðG, τ, − Þ respectively. If τ < 1=2, then the initial strategies of each V-block i ^ by constructing a homogeneous-threshold “augmented” network connected to the dual agent i are x ð0Þ = A and x ð0Þ = x ð0Þ = B; ^ v1 v2 v3 game Γ^ = ðG, ð1=2Þ1, − Þ that is dynamically equivalent to Γ.We τ > = = = = ^ and if iP 1 2, then xv1ð0Þ B and xv2ð0Þ xv3ð0Þ A. In total, V complete the proof by showing that if the augmented network has n + n m agents, the thresholds of all of which are set to ^ ^i=1 i game Γ reaches an equilibrium, then Γ does as well. The fol- 1=2. For E, in addition to the edges for the V-blocks, there is an ^ ^ lowing lemma establishes convergence of the homogeneous- edge between any two dual agents i and j in V^, if and only if there is threshold network game ðG, ð1=2Þ1, − Þ. an edge between i and j in V. Next, we show that if whenever an agent in G activates, its dual ^ Lemma 1. Every network of anticoordinating agents who update in G also activates (while neglecting the time steps that a V-block asynchronously under Assumption 1, with τi = 1=2 for each agent i ∈ V, will reach an equilibrium state in finite time. The proof of the lemma revolves around the following po- tential function: A B C nAðkÞ − n^A if x ðkÞ = A Φ ðkÞ = i i i , i ^A + − A = ni 1 ni ðkÞ if xiðkÞ B ^A where ni denotes the maximum number of A-neighbors of agent i that will not cause agent i to switch to B when playing A. The proof follows from the fact that the function is lower bounded and decreases every time an agent in the network switches strategies. To motivate our approach for extending this result to an ar- ^ Fig. 1. (A and B) Agents i and i have thresholds τi = 1=3 and τ^ = 1=2, but ^ i bitrary distribution of thresholds, consider for example an agent because an extra A-neighbor has been added to i, they are dynamically i with 4 neighbors whose threshold is 1=3. When playing A,this equivalent. (C) To ensure that the added node does not switch, two B agent can tolerate up to 1 A-neighbor (Fig. 1A), but 2 or more neighbors are added who also will not switch, forming a V-block.

Ramazi et al. PNAS Early Edition | 3of6 Downloaded by guest on September 30, 2021 agent is active), then the dynamics of each node in G are the same Corollary 4. Every network of all coordinating or all anticoordinating ^ as the dynamics of its dual node in G (again while neglecting the agents will reach an equilibrium state after no more than 6jEj time steps that a V-block agent is active). Then Theorem 1 can be agent switches. proven accordingly. This implies that agents cannot switch an arbitrary number As a result of Theorem 1, we have the following corollary, of times before the network reaches equilibrium. It follows which to the best of our knowledge has not been shown in the that when agent activation times are independent and iden- literature to date. tically distributed, the upper bound on the expected time to reach equilibrium is linear in the number of edges in the Corollary 1. Every network of anticoordinating agents admits a pure- network. strategy Nash equilibrium. Synchronous and Partially Synchronous Updating All Agents Are Coordinating So far, any network of all coordinating or all anticoordinating Theorem 2. Every network of coordinating agents who update agents is shown to reach an equilibrium state, as long as the asynchronously under Assumption 1 will reach an equilibrium state agents update asynchronously. However, the importance of in finite time. asynchronous updating to the convergence results remains an The proof of Theorem 2 follows similar steps as the anti- open problem. In this section, we show that, although full syn- coordinating case. The key difference is that the potential function chrony may not always result in convergence, the results indeed becomes the following: still hold for partial synchrony, in which a random number of 8 agents update at each time step. < A − A = ni ni ðkÞ if xiðkÞ A ΦiðkÞ = , [3] Synchronous Updating. We show that networks in which updates : nAðkÞ − nA + 1if x ðkÞ = B i i i are fully synchronous may never reach an equilibrium state, by presenting counterexamples with only two agents. A where ni is defined as the minimum number of A-neighbors First, suppose that both agents are anticoordinating and start required for an A-playing agent to continue playing A. The max- from the strategy vector ðA, AÞ. The agents update synchro- imum number of A-neighbors that a B agent can tolerate before nously, that is, at each time step, both agents activate and update A − switching to A is then given by ni 1. We refer the reader to SI their strategies according to the update rule in Eq. 2.Therefore, Appendix for a complete proof of the theorem. the dynamics will be deterministic, and the following transitions Theorem 2 also leads to a corollary on existence of equilibria will occur on the strategies of the agents: ðA, AÞ → ðB, BÞ → ðA, AÞ, for networks of coordinating agents, but in this case we confirm resulting in a cycle of length 2. an already known result (e.g., ref. 26). Now suppose that both agents are coordinating and they start from the strategy vector ðA, BÞ. Following the update rule in Eq. Corollary 2. Every network of coordinating agents admits a pure- 1, the following transitions take place under the synchronous strategy Nash equilibrium. updating: ðA, BÞ → ðB, AÞ → ðA, BÞ, again resulting in a cycle of The following corollary follows directly from Theorem 1 length 2. and Theorem 2, and the fact that a network of homogeneous The above examples prove that equilibrium convergence is no agents can always be described as all coordinating or all longer guaranteed if the agents update in full synchrony. How- anticoordinating. ever, it is known that any network game governed by synchro- Corollary 3. Every network of homogenous agents who update nous best-response dynamics will reach a cycle of length at most asynchronously under Assumption 1 will reach an equilibrium state 2, even when both coordinating and anticoordinating agents in finite time. coexist in the network (27).

Coordinating and Anticoordinating Agents Coexist. Given the pre- Partially Synchronous Updating. To understand what happens in vious two results, the natural question arises of what we can say the case of partially synchronous updates, we need to relax the about convergence when both coordinating and anticoordinating assumption that only one agent can update at a given time. We agents are present in a network. Although there may be partic- must therefore decouple the activation sequence from the dis- crete-time dynamics, and consider the activations to occur in ular configurations that converge, we can demonstrate that this ∞ continuous time. Let tj denote a sequence of times and will not hold in general with a simple example on a network ∞ f gj=1 j j j consisting of only two agents connected to each other, namely a fi gj=1, a sequence of agents such that agent i activates at time t . path of length one, that is, Vdf1,2g, Edfð1,2Þ, ð2,1Þg. Suppose Fig. 2 shows the relationship of the discrete time steps with the that agent 1 is coordinating, agent 2 is anticoordinating, and their agent activation sequence. thresholds are both equal to 1=2. Given their respective update It is now possible that multiple agents update between con- rules in Eqs. 1 and 2, agent 1 will always switch to match agent 2 secutive time steps. In particular, all agents who are active in the while agent 2 will always switch to oppose the strategy of agent 1. time interval ½k, k + 1Þ update at time k + 1 based on the state of In other words, the following transitions will occur with proba- the network at time k. Let Ak denote the set of all agents who are bility one: ðA, AÞ → ðA, BÞ → ðB, BÞ → ðB, AÞ → ðA, AÞ, resulting in a neverending cycle. This formulation is equivalent to a repeated version of the well-known game. From this, we can conclude that convergence is not a trivial result to be expected in all cases, and can only be guaranteed when all agents are either coordinating or anticoordinating. Fig. 2. In the partially synchronous update model, multiple agents can Convergence Time. The following corollary follows from the fact Φ activate in a single discrete time step. All agents who are active in the that the potential function ðkÞ is bounded from above and absolute time interval ½k, k + 1Þ update at time k + 1 based on the in- below and decreases by at least one every time an agent switches formation at time k. The colors red, blue, and green represent three dif- strategies. ferent agents.

4of6 | www.pnas.org/cgi/doi/10.1073/pnas.1610244113 Ramazi et al. Downloaded by guest on September 30, 2021 C for particular nonrandom activation sequences, as discussed in AB D the following.

Zero-Probability Nonconvergence. We now show that Theorem 3 holds only almost surely. In other words, there exist activation sequences generated under Assumption 2 that do not result in equilibrium convergence; however, the probability of such sequences happening is zero. Consider the network game Γ = ðG, ð1=2Þ1, + Þ where G and the initial strategies xð0Þ are as in Fig. 3A. First, we show that if any single agent activates when the strategy state equals xð0Þ, then there exists a finite sequence of agent activations that return the state to xð0Þ. Consider the case when agent 1 activates exclusively at k = 0 (Fig. 3A), and Fig. 3. (A–D) Strategy evolution in the network game Γ = ðG, ð1=2Þ1, + Þ hence, switches to B at k = 1. Then if agents 2, 3, 4, 6 activate at α = under the activation subsequence 1 ðf1g, f2,3,4,6g, f1,2,3,4,6gÞ.Red k = 1 and agents 1, 2, 3, 4, 6 at k = 2, the strategy state returns circles indicate those nodes that are activated at the corresponding x k = x = x time step. to ð0Þ at 3, that is, ð3Þ ð0Þ; the process is shown in Fig. 3 B–D. Denote the corresponding activation subsequence by α1 = ðf1g, f2,3,4,6g, f1,2,3,4,6gÞ. Similarly, as shown in Fig. 4, active during the time interval ½k, k + 1Þ. We can express the there exists an activation sequence that returns the state of = partially synchronous dynamics as follows: the system to xð0Þ when agent 2 activates at k 0. The corre- α = sponding activation subsequence would be 2 ðf2g, f1,3,4,5,6g, xbrðkÞ if i ∈ A f1,2,3,4,5,6gÞ. Moreover, due to the symmetric distribution of x ðk + 1Þ = i k , i x ðkÞ otherwise the strategies xð0Þ in the network, the same can be shown for i when agents 3, 4, or 6 are activated at k = 0 (which are similar to br agents 1, 2, and 5, respectively). Denote the corresponding ac- where xi ðkÞ denotes the best response of agent i to the strategies tivation subsequences by α3, α4, and α6. Now consider the event of its neighbors at time k. To provide a general framework α α for independent stochastic activation sequences, we make the X made by 1 and 2 as follows: following assumption. = ∞ ∈ α α SOCIAL SCIENCES X ðxiÞi=1, xi f 1, 2g. Assumption 2. The interactivation times for each agent are drawn from mutually independent probability distributions with support Any activation sequence in X can be generated under Assump- tion 2 and has the property that all agents activate exclusively on R≥ . 0 infinitely many times. However, no sequence in X results in con- One standard model that satisfies Assumption 2 is to use ex- Γ ponential distributions with mean 1=λ. This represents the case in vergence of the network game . However, this does not contra- dict Theorem 3 because P½X = 0, under Assumption 2. which each agent updates according to a Poisson clock with rate λ, and the expected number of agents updating in one unit of Concluding Remarks λ time is n. We have shown that arbitrary networks consisting of all coordi- From the previous sections, we know that the best-response nating or all anticoordinating agents who update with asynchronous dynamics do not necessarily converge to an equilibrium when best responses will reach equilibrium in finite time. Moreover, when the updating is synchronous, yet do converge when the updating updates are partially synchronous, we have shown that the network is asynchronous. Therefore, the natural question arising is the still almost surely reaches an equilibrium under mild conditions on following: how much asynchrony do the partially synchronous the independence and randomness of agent updates. For the case of best-response dynamics need for convergence? It turns out that, anticoordinating agents, these results have important implications in even under the relatively mild assumption on the partially syn- social contexts where individuals prefer an action only if a small chronous updates in Assumption 2, we still have convergence to enough portion of neighbors are using that action, for example, an equilibrium state almost surely. deciding which route to take to avoid traffic congestion, vol- unteering for a dangerous but important public service posi- Theorem 3. Every network of all coordinating or all anticoordinating tion, contributing money or time toward a crowd-sourced agents who update with partially synchronous dynamics that satisfy project, etc. For coordinating agents, the results apply to social Assumption 2 almost surely reaches an equilibrium state in finite contexts where each agents prefer an action only if a sufficient time. To prove the theorem, we model the network game as a Markov chain and show that it is absorbing (28). We refer the reader to SI Appendix for a detailed proof. A B C D Note that one technically equivalent but perhaps less practical modeling of the partially synchronous updates would be to simply assume that at each time step k = 0,1, ...,aran- dom number of random agents activate simultaneously and then update at k + 1. In other words, there is a fixed proba- bility that a particular group of agents activate simultaneously at every time step. Then Theorem 3 still holds because the probability that each agent activates asynchronously is boun- ded below by a positive constant. From a broader point of view, Theorem 3 holds whenever random asynchronous acti- vations are not completely excluded from the partially syn- Fig. 4. (A–D) Strategy evolution in the network game Γ = ðG, ð1=2Þ1, + Þ chronous dynamics. Convergence, however, may not be achieved under the activation subsequence α2 = ðf2g, f1,3,4,5,6g, f1,2,3,4,5,6gÞ.

Ramazi et al. PNAS Early Edition | 5of6 Downloaded by guest on September 30, 2021 number of neighbors are using that action, for example, the spread in the update dynamics, or a mixture of coordinating and anti- of social behaviors, technological innovations, viral infections. Our coordinating agents. These results also open the door to charac- results suggest that, in both cases, no matter how different the terizing the equilibria and investigating possibilities for payoff- individuals are, which neighbors affect their decisions, or how based incentive control of the network. many simultaneous decisions are made, everyone will tend to settle on a particular action with which they are satisfied. This ACKNOWLEDGMENTS. This work was supported in part by European Re- means that the presence of cycles or nonconvergence must result search Council Grant ERCStG-307207 and Dutch Technology Foundation from other factors such as imitation or other unmodeled effects Grant STW-vidi-14134.

1. Meloni S, Arenas A, Moreno Y (2009) Traffic-driven epidemic spreading in finite-size 16. Kleinberg J (2007) Cascading behavior in networks: Algorithmic and economic issues. scale-free networks. Proc Natl Acad Sci USA 106(40):16897–16902. Algorithmic Game Theory 24:613–632. 2. Funk S, Gilad E, Watkins C, Jansen VA (2009) The spread of awareness and its impact 17. Cimini G, Castellano C, Sánchez A (2015) Dynamics to equilibrium in network games: – on epidemic outbreaks. Proc Natl Acad Sci USA 106(16):6872 6877. individual behavior and global response. PLoS One 10(3):e0120343. 3. Skyrms B, Pemantle R (2000) A dynamic model of social network formation. Proc Natl 18. Granovetter M (1978) Threshold models of collective behavior. Am J Sociol 83(6): – Acad Sci USA 97(16):9340 9346. 1420–1443. 4. Young HP (2011) The dynamics of social innovation. Proc Natl Acad Sci USA 108(Suppl 19. Ramazi P, Cao M (2015) Analysis and control of strategic interactions in finite het- 4):21285–21291. erogeneous populations under best-response update rule. 54th IEEE Conference on 5. Kreindler GE, Young HP (2013) Fast convergence in evolutionary equilibrium selec- Decision and Control (CDC). (IEEE, Piscataway, NJ), pp 4537–4542. tion. Games Econ Behav 80:39–67. 20. Alós-Ferrer C (2003) Finite population dynamics and mixed equilibria. Int Game 6. Alós-Ferrer C, Neustadt I (2010) Best-response dynamics in a birth-death model of – evolution in games. Int Game Theory Rev 12(02):197–204. Theory Rev 5(03):263 290. 7. Montanari A, Saberi A (2010) The spread of innovations in social networks. Proc Natl 21. Chien S, Sinclair A (2007) Convergence to approximate Nash equilibria in congestion Acad Sci USA 107(47):20196–20201. games. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete 8. Masuda N, O’shea-Wheller TA, Doran C, Franks NR (2015) Computational model of Algorithms (Society for Industrial and Applied Mathematics, Philadelphia), pp collective nest selection by ants with heterogeneous acceptance thresholds. R Soc 169–178. Open Sci 2(6):140533. 22. Morris S (2000) Contagion. Rev Econ Stud 67(1):57–78. 9. Carfì D, Musolino F (2012) A coopetitive approach to financial markets stabilization 23. Lelarge M (2012) Diffusion and cascading behavior in random networks. Games Econ and risk management. Advances in Computational Intelligence (Springer, Berlin), pp Behav 75(2):752–775. – 578 592. 24. López-Pintado D (2006) Contagion and coordination in random networks. Int J Game 10. Riehl JR, Cao M (2017) Towards optimal control of evolutionary games on networks. Theory 34(3):371–381. IEEE Trans Automat Contr, in press. 25. Adam EM, et al. (2012) On the behavior of threshold models over finite networks. 11. Mäs M, Nax HH (2016) A behavioral study of “noise” in coordination games. J Econ 51st IEEE Conference on Decision and Control (CDC) (IEEE, Piscataway, NJ), pp Theory 162:195–208. 2672–2677. 12. Sandholm WH (2010) Population Games and Evolutionary Dynamics (MIT Press, Cambridge, MA). 26. Alós-Ferrer C, Netzer N (2010) The logit-response dynamics. Games Econ Behav 68(2): – 13. Kreindler GE, Young HP (2014) Rapid innovation diffusion in social networks. Proc 413 427. Natl Acad Sci USA 111(Suppl 3):10881–10888. 27. Adam EM, Dahleh MA, Ozdaglar A (2012) On threshold models over finite networks. 14. Ramazi P, Hessel J, Cao M (2015) How feeling betrayed affects cooperation. PLoS One arXiv:1211.0654. 10(4):e0122205. 28. Grinstead CM, Snell JL (2012) Introduction to Probability (American Mathematical 15. Skyrms B, Avise JC, Ayala FJ (2014) In the light of evolution VIII: Darwinian thinking in Society, London). the social sciences. Introduction. Proc Natl Acad Sci USA 111(Suppl 3):10781–10784. 29. Monderer D, Shapley LS (1996) Potential games. Games Econ Behav 14(1):124–143.

6of6 | www.pnas.org/cgi/doi/10.1073/pnas.1610244113 Ramazi et al. Downloaded by guest on September 30, 2021