Essays on Networks and Hierarchies

Home , Trigger strategy

The Pennsylvania State University The Graduate School Department of Economics

ESSAYS ON NETWORKS AND HIERARCHIES

A Dissertation in Economics by Myeonghwan Cho

°c 2008 Myeonghwan Cho

Submitted in Partial Fulﬁllment of the Requirements for the Degree of

Doctor of Philosophy

August 2008 The dissertation of Myeonghwan Cho was reviewed and approved∗ by the following:

Kalyan Chatterjee

Distinguished Professor of Economics and Management Science

Dissertation Advisor, Chair of Committee

Edward J. Green

Professor of Economics

James S. Jordan

Professor of Economics

Anthony M. Kwasnica

Associate Professor of Business Economics

Robert C. Marshall

Professor of Economics

Head of the Department of Economics

∗ Signatures are on ﬁle in the Graduate School. Abstract

This dissertation consists of four chapters.

Chapter 1 studies a repeated prisoner’s dilemma game in a network in which each agent interacts with his neighbors and cannot observe the actions of other agents who are not directly connected to him. If there is global information processing through a public randomization and global communication, it is not difficult to construct a sequential equilibrium which supports cooperation and satisfies a refinement, called stability, which requires that cooperation resumes after any history. Here, we allow agents to locally communicate with their neighbors and show that it is possible to construct such an equilibrium without global information processing. The role of local communication is to resolve the discrepancy of agents’ expectations on their neighbor’s future actions.

Chapter 2 considers the same environment as does Chapter 1, except that agents are not allowed to communicate with their neighbors but instead can observe a public randomization. By introducing a public randomization, we also can construct a sequential equilibrium for sufficiently patient agents, which supports cooperation and in which cooperation eventually resumes after any history. In addition, if there is a small possibility of mistakes, the equilibrium we construct can result in a more efficient outcome than the trigger strategy equilibrium, even though they produce the same payoffs in the limit.

Chapter 3 studies a situation in which agents can make a binding agreement both on the amount of local public goods and on the structure of networks through which they share the benefits of public goods. An agent enjoys the benefit of public goods produced by others who are (directly or indirectly) connected to him. There is a cost to maintain a link as well as to produce a public good. Since agents can choose the amount of public goods, the value of a link is endogenously determined. In the analysis, we first consider two different models of sequential bargaining games through which a contract on allocations is established. In the first model, we allow agents to propose a pure allocation and show that there is no symmetric stationary perfect equilibrium for sufficiently patient agents. In the

iii second model, agents are allowed to propose a distribution on allocations. As a result, we find a symmetric stationary perfect equilibrium in which probabilistic choices are made on an equivalent class of allocations. Next, we characterize core allocations, which consist of a minimally connected network and an effort profile in which at most one agent does not produce the maximum amount of public good.

Chapter 4 studies the problem of organizational design in a setting where workers diﬀer in their abilities and each worker is required to be monitored to prevent shirking. The need to monitor workers generates a hierarchical structure in an organization, which can be represented as a rooted tree. The value of an organization is determined by the wage cost, the total number of productive tasks carried out, and the length of chain of supervision in the organizational hierarchy. In this setting, we construct and characterize an eﬃcient structure of hierarchy which maximizes the value of the organization. In addition, we show that a hierarchy with the optimal size has a form of balanced rooted tree.

iv Table of Contents

List of Figures vii

List of Tables viii

Acknowledgments ix

1 Cooperation in the Repeated Prisoner’s Dilemma Game with Local In- teraction and Local Communication 1 1.1 Introduction ...... 1 1.2 The Model ...... 5 1.3 Strategy σ∗ ...... 8 1.4 Belief System µ ...... 13 1.5 Sequential Equilibrium ...... 15 1.6 Discussion ...... 17 1.7 Conclusion ...... 20 Appendix A: Proof of Lemma 1.3 ...... 22 Appendix B: Actions in period s ≥ t ...... 25 Appendix C: Proof of Proposition 1.1 ...... 28

2 Public Randomization in the Repeated Prisoner’s Dilemma Game with Local Interaction 34 2.1 Introduction ...... 34 2.2 The Model ...... 36 2.3 Sequential Equilibrium ...... 40 2.4 Stability and Robustness of σp ...... 45 2.5 Concluding Remarks ...... 51 Appendix: Proof of Proposition 2.1 ...... 52

v 3 Endogenous Formation of Networks for Local Public Goods 57 3.1 Introduction ...... 57 3.2 Models ...... 60 3.3 Sequential Bargaining Games ...... 62 3.3.1 Non-Existence of Symmetric Stationary Perfect Equilibrium with Pure Proposals ...... 65 3.3.2 Existence of Symmetric Stationary Perfect Equilibrium with Mixed Proposals ...... 68 3.4 Allocations in the Core ...... 74 3.5 Concluding Remarks ...... 80 Appendix A: Proofs in Section 3.3.1 ...... 81 Appendix B: Proofs in Section 3.3.2 ...... 83 Appendix C: Proofs in Section 3.4 ...... 86

4 Eﬃcient Structure of Organization with Heterogeneous Workers 90 4.1 Introduction ...... 90 4.2 Model ...... 93 ∗ 4.3 Eﬃcient Structure Ga ...... 97 4.4 Optimal Size of Organization ...... 104 4.5 Characterization of Structure Rule ...... 107 4.6 Conclusion ...... 109 Appendix: Proof of Proposition 4.6 ...... 110

References 113

vi List of Figures

1.1 σ∗-path conditioning on history h2 ...... 12 1.2 histories ht and hˆt ...... 14 1.3σ ˆ-path conditioning on h3 ...... 18 ∗ 3 ∗ 3 1.4 αN (σ ; h ) and αN (σ ; h˜ ) ...... 19 ∗ 2 1.5 αN (σ ; h ) ...... 20 1.6 Assumption of small l ...... 21 1.7 Assumption of small g ...... 21

3.1 (G, x) and (G0, x0) ...... 76 0 0 3.2 (x1, x2) satisfying NBi = 0 ...... 77 3.3 Examples of (G, x) for Proposition 3.6 ...... 79

4.1 Counting up and down rank systems ...... 96 ∗ 4.2 Example of Ga ...... 97 0 4.3 Example of Ga and Ga for Lemma 4.1 ...... 99 00 0 4.4 Example of Ga, Ga, and Ga for Lemma 4.2 ...... 100 4.5 Counting down rank system ...... 103 4.6 Example for the proof of Proposition 4.5 ...... 107 4.7 Structure ruleϕ ¯ ...... 109 4.8 Example for the proof of Lemma 4.6 ...... 112

vii List of Tables

1.1 Payoﬀs in prisoner’s dilemma game with communication ...... 6

2.1 Payoﬀ of prisoner’s dilemma game ...... 37

viii Acknowledgments

I am deeply indebted to Professor Kalyan Chatterjee for his invaluable advice and encouragement during the process of developing this research. Without his guidance and support, I would not have reached this point. I also would like to thank Professor Edward Green, Professor James Jordan, Professor Vijay Krishna, Professor Anthony Kwasnica, Profes- sor Marek Pycia, Professor Neil Wallace, and participants at the 18th Stony Brook Game Theory conference and PET07 conference for their helpful comments. In particular, I am grateful to Professor John Riew for his encouragement. The ﬁnancial support of the Robert C. Daniels Graduate Fellowship, the Will E. Mason Memorial Endowment Award, and the David W. and Carolyn P. Grow Graduate Fellowship are gratefully acknowledged. Finally and personally, I would like to thank my parents for their continued love and support throughout my entire life and Sun Young for being with me through the last year at Penn State.

ix Chapter 1

Cooperation in the Repeated Prisoner’s Dilemma Game with Local Interaction and Local Communication

1.1 Introduction

From the results called the folk theorem, it is well known that any outcome that Pareto dominates a Nash equilibrium (or, an individually rational outcome) in a stage game can be supported as an equilibrium in infinitely repeated games for sufficiently patient agents. The earliest works on this theorem, such as Friedman (1971) and Fudenberg and Maskin (1986), require that each agent perfectly monitors the actions chosen by all the other agents. After those works, many studies focus on situations in which monitoring is not perfect. For example, Kandori (1992b) and Ellison (1994) study repeated games with a random matching process, where agents are matched randomly in each period and each agent observes only the outcome of his own match. In this paper, we are interested in another kind of situation in which monitoring is not perfect, in that each agent observes only the actions of a subset of agents and his payoff depends only on his actions and the actions chosen by the agents in this subset. More precisely, agents are located in a network and each agent observes only the actions chosen by his neighbors, who are agents directly connected to him. In addition, in each period, each agent plays prisoner’s dilemma games against his neighbors and his stage game payoff is the sum of payoffs from the prisoner’s dilemma games against all of his neighbors. Furthermore, in each period, each agent has to choose only one action against his neighbors. So, he cannot play cooperation with one of his neighbors while playing defection against another neighbor. An example of this situation is local competition and collusion between adjacent car

1 dealers. That is, car dealers are located along the road and two neighboring car dealers play a Bertrand competition for consumers located between them. They can cooperate with each other by choosing a monopolistic price, or defect by choosing a competitive price. Since car dealers cannot tell consumers apart, each dealer has to choose the same price against all of his neighbors. Since the price is a private offer to consumers, a dealer may not figure out other dealers’ prices which do not affect his profit directly. Furthermore, if there is a cost to see the prices of other dealers, a dealer may not want to pay the cost to see prices which are irrelevant to his profit.1 For this environment, the goal of this paper is to construct a sequential equilibrium for sufficiently patient agents, which supports cooperation and satisfies a refinement that cooperation resumes after any history. Kandori (1992b) introduces (global) stability as a desirable property of equilibria in infinitely repeated games and it requires that continuation payoffs eventually go back to the original payoffs after any history. If an equilibrium satisfies the refinement that cooperation resumes after any history, it is stable with Kandori (1992b)’s notion. As he pointed out, stability allows agents to test various actions in order to learn the social norm. In addition, if there is a small possibility of mistakes, a payoff in a stable equilibrium may be higher than a payoff in an equilibrium which is not stable, although they yield the same payoff in the limit as the probability of mistakes goes to zero. Furthermore, if we consider an evolutionary process in which pairs of players from a population are randomly matched and mutant strategies occasionally emerge, a trigger strategy with permanent punishment, which is not stable, is invaded by a strategy with finite-period punishment. As a result, the trigger strategy will disappear in the evolutionary process.2 Indeed, it is not difficult to construct an unstable sequential equilibrium supporting cooperation. Consider the trigger strategy that observing a deviation causes a permanent punishment. If the loss from being cheated is small enough, the trigger strategy cannot be a sequential equilibrium since an agent is reluctant to punish a defector for fear of being punished by other neighbors. However, the trigger strategy can be a sequential equilibrium for agents who are not sufficiently patient, and Ellison (1994) provides the idea of constructing a sequential equilibrium for sufficiently patient agents by modifying the trigger strategy. His modification of the trigger strategy is a sequential equilibrium which supports cooperation, but cooperation never resumes after a mistake to play defection.

1Another example for our model is one in which there is production and sharing of local public goods. Each agent decides whether or not to produce his public good whose benefit is shared with his neighbors. The individual benefit from the public good is smaller than its cost, even though the social benefit from the public good is greater than its cost. Thus, a dominant strategy for each agent is not to produce, but it does not yield an efficient outcome. 2I thank Drew Fundenberg for making me aware of this point. Fudenberg and Maskin (1990) define evolutionarily stable strategies as strategies which cannot be invaded by any mutant strategy that is initially rare in the evolutionary process. The trigger strategy with permanent punishment is not evolutionarily stable in their notion.

2 The usual way to obtain an equilibrium in which cooperation resumes after any history is to have a punishment of fixed finite length. That is, if an agent observes his neighbor playing defection, he plays defection in order to punish the defector for a finite number of periods. However, local observability may cause a discrepancy in his neighbors’ expectations about when he resumes cooperation. If there is such a discrepancy in some history, the agent whose neighbors have different expectations about his future actions may not be able to satisfy his neighbors’ expectations in any period. This discrepancy in expectations brings a difficulty in constructing a sequential equilibrium in which cooperation resumes after any history. If agents share common information through perfect monitoring, public randomization, or global communication, they can reach an agreement on when they resume cooperation after it breaks down. So, it is not difficult to construct a stable sequential equilibrium.3 In this paper, we introduce local communication and show the possibility of constructing a stable sequential equilibrium without global information processing. Here, local communication means that, in each period, each agent sends a message only to his neighbors without any cost and messages do not affect his payoffs directly. The message can be interpreted as car dealer’s advertising banners with slogans such as “On Sale.” In the equilibrium we construct, an agent starts a finite-period defection phase if there was a surprise in the previous period. Here, a surprise means a deviation from expectation, not a deviation from strategy. Because an agent cannot figure out whether his neighbor’s defection is a deviation or a consequence of punishing other neighbors, it is not possible to construct an equilibrium in which punishment occurs based on deviation. In this paper, we first construct the expectations of agents about their neighbors’ actions, and then construct an equilibrium in which each agent chooses his action based on whether or not his neighbors follow his expectations. The role of local communication in the equilibrium is to enable an agent to resolve the discrepancy of his neighbors’ expectations by informing them when he starts a finite-period defection phase.

Related literature

Since the earliest works on the folk theorem which assume perfect monitoring, there have been many studies concerned with situations involving imperfect monitoring. Green and Porter (1984) and Fudenberg et al. (1994) investigate the situation in which agents do not observe the actions chosen by the other agents, but only a public signal whose distribution depends on the actions chosen by all the agents. The situation where each agent only observes a private signal is studied by Sekiguchi (1997), Compte (1998), Kandori and Mat-

3In Ellison (1994) and Cho (2007b), a public randomization is used as a coordination device to resume cooperation. Global communication is discussed in Ben-Porath and Kahneman (1996), Kandori and Mat- sushima (1998), Compte (1998), and Obara (2007).

3 sushima (1998), Bhaskar and Obara (2002), Ely and Valimaki (2002), Hörnerand Olszewski (2006), and Obara (2007). Most of these studies show that, if signals convey sufficient information on the actions chosen by the other agents, a folk theorem holds. We note that the local interaction model this paper deals with is a specific case of imperfect private monitoring, where each agent’s private signals are his neighbor’s actions. However, the results of the folk theorem in the previous studies cannot be applied to our environment, because an agent’s signal does not say anything about the actions chosen by other agents who are not directly connected to him.4 There are earlier papers which share an aspect with our paper, in that agents are located in a network and each agent only observes the actions chosen by his neighbors. Ben- Porath and Kahneman (1996) allow agents to send messages about their observations to all the agents, and show that, if each agent has at least two neighbors, an efficient outcome can be obtained as a sequential equilibrium. Renault and Tomala (1998) do not allow communication and show that a folk theorem holds if a network satisfies a property, called 2-connectedness.5 Haag and Lagunoff (2006) consider local trigger strategy equilibrium, in which each agent triggers defection conditioning on the defection of a group of neighbors, as a solution concept and try to find an optimal network structure when individual discount factors are randomly determined. Compared with the research studying the relationship between efficiency and equilibrium, little attention has been paid to stability of equilibria in pervious studies. Kandori (1992b) and Ellison (1994) construct stable sequential equilibria supporting efficient outcome in anonymous random matching models. For stability, partial information processing for partners’ past actions, in Kandori (1992b), or public randomization, in Ellison (1994), is required. It is not known yet whether stability is possible in anonymous random matching models without partial information processing or public randomization. Stability in local interaction model was first studied by Xue (2004), where he constructs a sequential equilibrium which is stable and supports cooperation without public randomization and (local or global) communication. Although he does not allow public randomization or communication, he considers only line-shaped networks and his equilibrium strategy is complicated and difficult to implement. By introducing local communication, we can consider more general networks including line-shaped networks and can also construct a simpler

4We also note that Compte (1998), Kandori and Matsushima (1998), and Obara (2007) introduce global communication in their models by allowing agents to send a message to all other agents. As public randomization does, global communication provides agents with commonly shared information. By considering strategies depending only on common information, they could deal with the difficulty from non-recursive structures of repeated games with private information. 5A directed network G is 2-connected if, for any i, j, k ∈ N, there is a directed path from i to j in the graph obtained from G by removing agent i. Since Renault and Tomala (1998) consider average payoffs without discounting, agents can use a block of finite periods to communicate with their neighbors without any cost.

4 equilibrium.6 Cho (2007b) also considers a repeated prisoner’s dilemma game with local interaction. Here, we do not allow local communication but allow agents to observe a public randomization, and construct a sequential equilibrium which supports cooperation and in which cooperation eventually resumes after any history.

The remainder of the paper is organized as follows. In Section 1.2, we formulate the repeated prisoner’s dilemma game with local interaction and explain the solution concept. In Section 1.3, we construct a strategy σ∗ in which cooperation resumes in ﬁnite periods after any history. In Section 1.4, we build a belief system which is consistent with σ∗. In Section 1.5, we show that the strategy σ∗ is a sequential equilibrium with the belief system. Some discussions follow in Section 1.6, and we conclude in Section 1.7.

1.2 The Model

There are n ≥ 3 agents and the set of agents is N = {1, . . . , n}. Agents are located in an undirected network G, which is a collection of links ij ≡ {i, j} ⊂ N. We assume that G is minimally connected. That is, for any distinct i, j ∈ N with i 6= j, there is a unique subset

{i1, i2, . . . , iL} of N satisfying i1 = i, iL = j, and ilil+1 ∈ G for l = 1,...,L − 1. We call 7 such a subset {i1, i2, . . . , iL} a chain between i and j. For each i, we deﬁne a distance of j from i, denoted d(j; i), by the number of links which consist of agents in the chain between i and j. If ij ∈ G, agent j (resp. agent i) is said to be a neighbor of i (resp. j).8 For each i, let

Gi = {j ∈ N : ij ∈ G} be the set of agent i’s neighbors. Let Gi = Gi ∪ {i}. Since G is undirected, ij ∈ G is equivalent to ji ∈ G, and j ∈ Gi if and only if i ∈ Gj. Agent i is an end agent, if he has only one neighbor. Since G is minimally connected, there are at most n − 1 end agents (in a star-shaped network), and at least two end agents (in a line-shaped network). In each period t, each agent plays a prisoner’s dilemma game with communication against his neighbors. That is, in each period t, each agent i chooses an action between C (cooperation) and D (defection). In addition, he can communicate with his neighbors by sending a message between 0 and 1, which does not affect the payoffs directly. We let t t t ai ∈ {C0,C1,D0,D1} be agent i’s action in period t, where ai = Cm (resp. ai = Dm) means 6For example, punishments for unexpected defection and unexpected cooperation have different severities in Xue (2004), while they have the same severity in this paper. Furthermore, in the equilibrium Xue (2004) constructs, it is possible that an agent expects his neighbor to play cooperation for some finite periods, defection for the following finite periods, and cooperation thereafter, which cannot happen in our equilibrium. 7In some papers such as Jackson and Wolinsky (1996) and Bala and Goyal (2000), a subset {i1i2, i2i3, . . . , iL−1iL} of G where {i1, i2, . . . , iL} is a chain between i and j is called a path in G connecting i and j. We avoid this definition to escape from the confusion with σ-path we will define later. 8In some papers such as Hojman and Szeidl (2006), a neighbor refers to an indirectly connected agent as well as a directly connected agent.

5 i j C0,C1 D0,D1

C0,C1 1, 1 −l, 1 + g

D0,D1 1 + g, −l 0, 0

Table 1.1: Payoﬀs in prisoner’s dilemma game with communication

that agent i plays C (resp. plays D) and sends a message m ∈ {0, 1} in period t. The payoﬀs of prisoner’s dilemma game with communication between i and j are given as in Table 1.1, where g > 0 and l > 0. We assume that g and l are small enough to satisfy that, for all i ∈ N, 9 g(|Gi| − 1) < 1 and l(|Gi| − 1) < 1. (1.1)

Note that (1.1) implies g − l < 1 which guarantees that all agents playing C is the efficient t ∞ t t outcome. We simplify the notation by letting ai = (ai)t=1, aK = (aj)j∈K , and aK = t ∞ (aK )t=1 for K ⊂ N. Let w(a, a0) be agent i’s payoff in prisoner’s dilemma game with communication against 0 j ∈ Gi when i plays a and j plays a . That is, w(C,C) = 1, w(C,D) = −l, w(D,C) = 1+g, and w(D,D) = 0 where C ∈ {C0,C1} and D ∈ {D0,D1}. The stage game payoff of agent i t in period t, when aN is played, is the sum of his payoffs in prisoner’s dilemma games with communication against his neighbors:10 X t t t ui(aN ) = w(ai, aj). j∈Gi

The payoﬀ of agent i in the repeated prisoner’s dilemma game with communication, when aN is played, is the average of discounted stage game payoﬀs:

X∞ X∞ X t−1 t t−1 t t Ui(aN ) = (1 − δ) δ ui(aN ) = (1 − δ) δ w(ai, aj), t=1 t=1 j∈Gi where δ ∈ (0, 1) is a common discount factor. Note that the payoff of agent i depends only on his own actions and his neighbors’ actions. A history ht in period t is a profile of actions chosen by all agents before period t. That 1 t s t−1 t t is, h = ∅ and h = (aN )s=1 for t ≥ 2. Let H denote the set of all histories h in period 9For a set A, |A| denotes the number of elements in A. 10We may want to normalize agent i’s stage game payoff by letting X t 1 t t ui(aN ) = w(ai, aj ). |Gi| j∈Gi This normalization does not affect the result.

6 t. In our model, each agent observes only the actions chosen by himself and his neighbors. t s t−1 ˆt s t−1 Thus, for t ≥ 2, histories h = (aN )s=1 and h = (âN )s=1 are in the same information set of agent i if and only if (as )t−1 = (âs )t−1 .11 With a slight abuse of notation, we write Gi s=1 Gi s=1 o1 = ∅ and ot = (as )t−1 for t ≥ 2 for an information set of agent i in period t. Since i i Gi s=1 agents are finite and actions in stage game are finite, histories in each information set are t t t finite. We write oi(h ) for the information set of agent i to which history h belongs. Let t Oi denote the set of agent i’s information sets in period t. We restrict our attention to pure strategies. A strategy of agent i is a function σi : S∞ t t=1 Oi → {C0,C1,D0,D1}. Let σ = (σ1, . . . , σn). Let Σi be the set of all strategies t of agent i and Σ = ×i∈N Σi. Given a strategy σ, a σ-path conditioning on h , denoted t s t ∞ αN (σ; h ) = ((αi (σ; h ))s=1)i∈N , is the string of actions which agents actually play under t t the strategy σ, given that h is reached. Formally, αN (σ; h ) is defined as follows. Let t s t−1 t a strategy σ be given. Let h = (aN )s=1 for t ≥ 2 (or, h = ∅ for t = 1) be a history. s t s t t t t For each i, let αi (σ; h ) = ai for s ≤ t − 1 and αi(σ; h ) = σi(oi(h )) for s = t. For s t s t s t t t s ≥ t + 1, αi (σ; h ) is determined iteratively as αi (σ; h ) = σi(oi ) where oi = oi(h ) and os = (os−1, (αs−1(σ; ht)) ). i i j j∈Gi In the paper, we are interested in a sequential equilibrium. A belief system µ is a function which assigns each information set to a probability distribution on the histories in t t t the information set. We denote a distribution on oi which µ assigns to oi by µ(·; oi), and t t t t t t a probability of h ∈ oi given oi being reached by µ(h ; oi). Note that, since oi has finite t t t t 12 elements, h ∈ supp(µ(·; oi)) if and only if µ(h ; oi) > 0. A belief system µ is consistent with σ, if it is the limit of a sequence of belief systems which are generated by Bayesian updating of fully mixed behavioral strategies converging to σ.13 A strategy σ is a sequential equilibrium if, for some belief system µ which is consistent t with σ, it satisfies that, for each i and for each oi, X X t t t t t 0 t 0 µ(h ; oi)Ui(αN (σ; h )) ≥ µ(h ; oi)Ui(αN (σi, σ−i; h )) for all σi ∈ Σi. (1.2) t t t t h ∈oi h ∈oi

If a strategy σ satisﬁes (1.2) for some µ, it is said to be sequentially rational under µ.14

11In Kandori and Matsushima (1998) and Xue (2004), a (joint or global) history refers to a history ht and t a private history of agent i refers to an information set oi. 12A pair of a belief system µ and a strategy σ is called an assessment. 13 ∞ t t t t t t t A sequence {µk}k=1 of belief systems converges to µ if, for each oi, µk(h ; oi) → µ(h ; oi) for all h ∈ oi. t A fully mixed behavioral strategy βi of agent i is a function which assigns each information set oi to a t t distribution βi(·; oi) on {C0,C1,D0,D1} where βi(a; oi) > 0 for each a ∈ {C0,C1,D0,D1}. A sequence ∞ t t t {βik}k=1 of fully mixed behavioral strategies of agent i converges to βi, if for each oi, βik(a; oi) → βi(a; oi) t t t t for all a ∈ {C0,C1,D0,D1}. If, for each oi, there is some ai ∈ {C0,C1,D0,D1} such that βi(ai; oi) = 1, t t t then the behavioral strategy βi is equivalent to the pure strategy σi such that σi(oi) = ai for each oi. 14Although Kreps and Wilson (1982) define a sequential equilibrium for finite extensive form games, we can extend their definition to infinite extensive games without any conceptual innovation. Many previous studies, such as Kandori (1992b), Sekiguchi (1997), and Xue (2004), adopt sequential equilibrium as a

7 t Given an information set oi and a strategy σ, we deﬁne a continuation payoﬀ CUi of t agent i at oi by   X X∞ X t t t  τ−t τ t τ t  CUi(σ; oi) = µ(h ; oi) (1 − δ) δ w(αi (σ; h ), αj (σ; h )) . t t τ=t j∈G h ∈oi i

Let ot = (as )t−1 . Since, for all ht ∈ ot and for all σ ∈ Σ, i Gi s=1 i X t t t µ(h ; oi)Ui(αN (σ; h )) ht∈ot i   X Xt−1 X t t  τ−1 τ τ  t−1 t = µ(h ; oi) (1 − δ) δ w(ai , aj ) + δ CUi(σ; h ), t t τ=1 j∈G h ∈oi i

t 0 t 0 (1.2) holds if and only if CUi(σ; h ) ≥ CUi(σi, σ−i; h ) for all σi ∈ Σi.

1.3 Strategy σ∗

Since an agent in our environment cannot figure out whether his neighbor’s defection is a deviation or a consequence of punishment, it is not possible to construct an equilibrium in which he starts a punishment after observing a deviation. In this section, we first define the t phase for each information set oi, which represents agent i’s expectations on his neighbors’ actions and his neighbors’ expectations on i’s action in period t. After this, we construct a ∗ ∗ t t strategy σ such that σi (oi) depends on the phase of oi. t A phase of information set oi is represented as

t t t P (oi) = (λki, λik)k∈Gi ,

t t ∗ where λki, λik ∈ {0, 1,..., Λ}. We will determine Λ in Section 1.5 for which the strategy σ is a sequential equilibrium. Indeed, Λ is the length of periods when an agent plays defection to punish a deviator, and so it determines the severity of punishment for deviation. t t To define P (oi) for each oi, we first define an expectation function E : {0,..., Λ} → {{C0,C1}, {D0}, {D1}} by    {C ,C } if λ = 0  0 1 E(λ) = {D0} if λ = 1,..., Λ − 1    {D1} if λ = Λ solution concept for infinitely repeated games.

8 t t t t t t For information set oi with P (oi) = (λki, λik)k∈Gi , we can interpret E(λki)(resp. E(λik)) as agent k’s (resp. agent i’s) expectation on agent i’s action (resp. agent k’s action) in t period t. For example, if E(λki) = {C0,C1} for k ∈ Gi, agent k expects agent i to play t t C0 or C1 in period t. Furthermore, λki (resp. λik) can be interpreted as agent i’s (resp. agent k’s) expectation on how long agent k (resp. agent i) keeps playing D (D1 or D0) after t t period t (including period t). If λik 6= 0 in P (oi), agent k is said to be in defection phase t t under i’s expectation. If λik = 0 in P (oi), agent k is said to be in cooperation phase under i’s expectation. 1 For each i ∈ N, P (oi ) satisﬁes

1 1 (λki, λik) = (0, 0) for all k ∈ Gi.

For each t ≥ 2, let ot = (as )t−1 . Then, P (ot) is determined iteratively as follows. Let i Gi s=1 i ot−1 = (as )t−2 be the information set of agent i in period t − 1 which coincides with ot, i Gi s=1 i t−1 t−1 t−1 and let P (oi ) = (λki , λik )k∈Gi .

t−1 t−1 t−1 t−1 (P 1) In a case that ai ∈ E(λki ) and ak ∈ E(λik ),

t t−1 t t−1 λki = max{λki − 1, 0} and λik = max{λik − 1, 0}.

t−1 t−1 t−1 t−1 (P 2) In a case that ai ∈/ E(λki ) and ak ∈ E(λik ),

t−1 t t if ai 6= D1, then (λki, λik) = (Λ, Λ) t−1 t t if ai = D1, then (λki, λik) = (Λ − 1, Λ).

t−1 t−1 t−1 t−1 (P 3) In a case that ai ∈ E(λki ) and ak ∈/ E(λik ),

t−1 t t if ak 6= D1, then (λki, λik) = (Λ, Λ) t−1 t t if ak = D1, then (λki, λik) = (Λ, Λ − 1).

t−1 t−1 t−1 t−1 (P 4) In a case that ai ∈/ E(λki ) and ak ∈/ E(λik ),

t t (λki, λik) = (Λ, Λ).

t s t−1 s s s Given a history h = (aN )s=1, if ak ∈/ E(λik), we say that ak is a surprise to agent i by agent k, or that agent k surprises agent i in period s. In (P 1), there is no surprise between i and k in period t − 1. In this case, suppose t that agents i and k do not surprise each other after period t. If λik 6= 0, which means k is t supposed to play D in period t, then agent i expects that k plays D for λik periods and C t thereafter. If λik = 0 which means agent k is supposed to play C in period t, then agent i expects agent k to play C forever.

9 In (P 2), agent i surprises agent k but k does not surprise i in period t − 1. In this case, if there is no other surprise between i and k in the future, then agent i expects agent k to play D for Λ periods (D1 in period t and D0 for following Λ − 1 periods). Furthermore, if agent i played D1 in period t − 1, then agent k expects agent i to be in defection phase for

Λ − 1 periods. If agent i played D0 in period t − 1, then agent k expects that agent i starts a defection phase in period t. In (P 3), we just change the roles of agents i and k in (P 2). In (P 4), agents i and k surprise each other in period t − 1. In this case, if there is no other surprise between i and k in the future, each expects that the other agent plays D for

Λ periods (D1 in period t and D0 for following Λ − 1 periods) and C thereafter. t t Note that (λki, λki) depends only on the past actions chosen by agents i and k. Since, for t t s t−1 t each i and for each period t,(λ , λ )k∈Gi depends only on (a ) , P (o ) is well defined ki ki Gi s=1 i t for each oi. t t t t From the construction of P , it is not difficult to see that, for any oi, P (oi) = (λki, λik)k∈Gi satisfies that, for each k ∈ Gi,

t t λik − λki ∈ {−1, 0, 1}. (1.3)

Lemma 1.1 provides another property of P which is used in constructing σ∗.

t t t t t Lemma 1.1. Let t ≥ 2. For information set oi, let P (oi) = (λki, λik)k∈Gi . If λki 6= Λ and t 0 t t λk0i 6= Λ for k, k ∈ Gi, then λki = λk0i.

t s t s s s s s Proof. Let o = (a ) with t ≥ 2. For s < t, let P (o ) = (λ , λ )k∈Gi where o is the i Gi s=1 i i ki ik i t t t 0 information which is consistent with oi. Let λki 6= Λ and λk0i 6= Λ for k, k ∈ Gi and let j ∈ {k, k0}. s s s Suppose that {s : ai = D1, s < t} = ∅. Suppose in addition that aj ∈/ E(λij) for some s+1 s+1 s+1 s < t. By construction, λji = Λ holds. Since ai ∈/ {D1} = E(Λ) = E(λji ), we have s+2 s+2 s+2 s+3 λji = Λ. Since ai ∈/ {D1} = E(Λ) = E(λji ), we have λji = Λ. Continuing this t s s procedure until s + τ = t, we have λji = Λ which is a contradiction. Thus, aj ∈ E(λij) for s s s s s all s < t. Suppose that ai ∈/ E(λji) for some s < t. Since ai 6= D1 and ai ∈/ E(λji), we s+1 s+1 s+1 s+2 have λji = Λ. Since ai ∈/ {D1} = E(Λ) = E(λji ), we have λji = Λ. Continuing this t s s procedure until s + τ = t leads us to λji = Λ which is a contradiction. Thus, ai ∈ E(λji) s s s s for all s < t. Since ai ∈ E(λji) and aj ∈ E(λij) for all s < t, by the construction of P , we t 0 have λji = 0 for j ∈ {k, k }. s s Suppose that {s : ai = D1, s < t} 6= ∅. Lets ¯ = max{s : ai = D1, s < t}. Similarly as s s s s s¯ s¯ before, we have ai ∈ E(λji) and aj ∈ E(λij) for all s satisfyings ¯ < s < t. Since aj ∈ E(λij) s¯ s¯+1 t and ai = D1 imply λji = Λ−1, by the construction of P , we have λji = max{Λ−(t−s¯), 0} for j ∈ {k, k0}.

∗ t Now, we are ready to deﬁne the strategy proﬁle σ . Consider an information set oi with

10 t t t t P (oi) = (λki, λik)k∈Gi . Lemma 1.1 implies that P (oi) satisﬁes one of the followings:

t (S1) for some k ∈ Gi, λki = Λ.

t (S2) for all k ∈ Gi, λki = λ for some λ ∈ {0, 1,..., Λ − 1}.

That means, in each period t, agent i faces with only two situations: situation (S1) where at least one of his neighbors expects that agent i plays D1 in period t or situation (S2) where all his neighbors have the same expectation on agent i’s action in period t. ∗ t The strategy σi of agent i is deﬁned as follows: for each oi,

t ∗ t • when P (oi) satisﬁes (S1), agent i plays D1. That is, σ (oi) = D1.

t • when P (oi) satisﬁes (S2), agent i plays D0 if E(λ) = {D0}, and C0 if E(λ) = {C0,C1}. ∗ t ∗ t That is, σ (oi) = D0 if E(λ) = {D0}, and σ (oi) = C0 if E(λ) = {C0,C1}.

∗ In other words, agent i employing σi plays D1 if there is a neighbor who expects him to play D1, and follows his neighbors’ expectation if they have the same expectation about his action. Figure 1.1 provides examples of σ∗-path conditioning on history h2 under a line-shaped network. In Figure 1.1 (a), agent 3 surprises agents 2 and 4 by playing D1 in period 1, so 2 2 2 2 (λ23, λ32) = (Λ − 1, Λ) and (λ43, λ34) = (Λ − 1, Λ). Thus, agent 3 plays D0, and 2 and 4 play D1 in period 2. In Figure 1.1 (b), agent 3 surprises 2 and 4 by playing D0 in period 1, 02 02 02 02 so (λ23, λ32) = (Λ, Λ) and (λ43, λ34) = (Λ, Λ). Thus, agents 2, 3, and 4 play D1 in period 2, D0 for following Λ − 1 periods, and C0 thereafter. ∗ t t t Note that, if agent i is an end agent with Gi = {k}, σi (oi) = E(λki) for any oi, which t means he will not surprise his neighbor in any history. Moreover, for any history h , C1 is never played in period s ≥ t along the σ∗-path conditioning on ht.15 ∗ Lemma 1.2 states that, under σ , C0 is recovered in ﬁnite periods from any history. To prove Lemma 1.2, we need to deﬁne h∗s for each s ≥ 1 which is the history in period s when ∗ t t τ t−1 ∗ σ is played given that h is reached. Given a history h = (aN )τ=1 and the strategy σ , let

∗1 ∗s τ ∗ t s−1 h = ∅, and h = (αN (σ ; h ))τ=1 for s ≥ 2. (1.4)

∗s τ s−1 ∗s ∗ Since h = (aN )τ=1 for each s with 2 ≤ s ≤ t, h for s ≤ t does not depend on σ .

Lemma 1.2. Given the strategy σ∗, there exists τ¯ ≥ 0 such that, for any ht and for each t+τ ∗ t i ∈ N, αi (σ , h ) = C0 for all τ ≥ τ¯.

15 Indeed, three actions (C, D0, and D1) are enough to construct a sequential equilibrium which supports cooperation and in which cooperation is recovered in ﬁnite periods from any history. In this paper, the reason for considering four actions (C0, C1, D0, and D1) is that we want to separate message space from action space in a prisoner’s dilemma game.

11 t 1 2 3 4 5 6 1 2 3 4 5 6

2 1 C0 C0 D1 C0 C0 C0 C0 C0 D0 C0 C0 C0 h 2 C0 D1 D0 D1 C0 C0 C0 D1 D1 D1 C0 C0 3 D1 D0 D0 D0 D1 C0 D1 D0 D0 D0 D1 C0 4 D0 D0 D0 D0 D0 D1 D0 D0 D0 D0 D0 D1 5 D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 ...... Λ D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 D0 Λ + 1 D0 D0 C0 D0 D0 D0 D0 D0 D0 D0 D0 D0 Λ + 2 D0 C0 C0 C0 D0 D0 D0 C0 C0 C0 D0 D0 Λ + 3 C0 C0 C0 C0 C0 D0 C0 C0 C0 C0 C0 D0 ......

(a) (b)

Figure 1.1: σ∗-path conditioning on history h2

t s t−1 t s ∗s s s Proof. Fix a history h = (aN )s=1 ∈ oi. For each s ≥ 1, let P (oi (h )) = (λki, λik)k∈Gi , ∗s where h is deﬁned as in (1.4). For each i, let κi(0) = {i} and κi(x) = {k ∈ N\{i} : d(k; i) = x} for x = 1,..., M¯ where M¯ = max{d(k; i): i ∈ N and k ∈ N\{i}}. Thus,

κi(x) is the set of agents who have distance x from i. Since G is minimally connected, N is partitioned into κi(0), . . . , κi(M¯ ). Consider agents i and k with ik ∈ G. We ﬁrst show that there is no surprise between i and k after t + M¯ . Since n ≥ 3, M¯ ≥ 2 holds. Suppose that there is a surprise to ¯ t+M¯ +τ t+M¯ +τ i by k in period t + M + τ for some τ ≥ 0. That is, ak ∈/ E(λik ). Since t+M¯ +τ ∗ t+M¯ +τ t+M¯ +τ t+M¯ +τ t+M¯ +τ ak = σk(oi (h )), ak = D1 and λik 6= Λ. Thus, there is an agent ¯ k ∈ G such that k 6= i and λt+M+τ = Λ. Notice that k ∈ κ (2). Then, in period 2 k 2 k2k 2 i ¯ t+M¯ +τ−1 t+M +τ −1, we have either (i) agent k surprises k2 by playing ak 6= D1, or (ii) agent ∗ t+M¯ +τ−1 t+M¯ +τ−1 k2 surprises k. Since (i) implies the contradiction that σk(oi (h )) 6= D1 and ¯ ¯ σ∗(ot+M+τ−1(ht+M¯ +τ−1)) ∈/ E(λt+M+τ−1), (i) cannot be the case. Thus, agent k surprises k i k2k 2 ¯ k in period t+M +τ −1 and, from the same argument as before, there is an agent k3 ∈ Gk2 with k3 ∈ κi(3) who surprises k2 in period t + M¯ + τ − 2. Continuing this procedure, we have that km ∈ κi(m) is an end agent so there is no agent km+1 ∈ Gkm with km+1 6= km−1 ¯ who surprises k in period t + M¯ + τ − m − 1. Thus, k does not play σ∗ (ot+M+τ−m) in m m km km period t + M¯ + τ − m, which is a contradiction. Therefore, there is no surprise to i by k in period t + M¯ + τ for some τ ≥ 0. Similarly, we can show that there is no surprise to k by i in period t + M¯ + τ for some τ ≥ 0. Since ik ∈ G is arbitrary, there is no surprise between i and his neighbors after t + M¯ .

12 ∗ Then, by the construction of σ , agent i plays C0 after period t+M¯ +Λ. Lettingτ ¯ = M¯ +Λ, we complete the proof.

Kandori (1992b) introduces global stability as a desirable property for equilibria in infinitely repeated games. An equilibrium is globally stable if, for any finite history ht, the continuation expected payoffs of agents eventually return to the payoffs the equilibrium sustains. In our notion, an equilibrium strategy σ is globally stable if, for any ht,

s ∗s 1 1 lim CUi(σ; o (h )) = CUi(σ; o (h )) for all i ∈ N, s→∞ i i P where h∗s is constructed as in (1.4) for σ and ht. Since CU (σ∗; o1(h1)) = w(C,C), i j∈Gi Lemma 1.2 obviously implies that the strategy σ∗ is a globally stable equilibrium if σ∗ is an equilibrium.

1.4 Belief System µ

In this section, we construct a belief system µ which is consistent with σ∗ and provide a t τ t τ ∗ τ ∗τ ∗τ property of µ. In history h , an action ak ∈ h is a mistake if ak 6= σk(ok(h )) where h is ∗ t s t−1 deﬁned as in (1.4). Fix the strategy σ . The number of mistakes in history h = (aN )s=1 is denoted by ρ(ht). That is,

t s t s ∗ s ∗s ρ(h ) = |{ak ∈ h : ak 6= σk(ok(h )), k ∈ N}|.

Let µε be a belief system which is generated by Bayesian updating from behavioral ∗ t strategy which assigns (1 − 3ε) to σi (oi) and ε to each of other actions at each information t ∗ set oi. Let µ be the limit of µε when ε → 0. Trivially, µ is consistent with σ . t t t For each information set oi and h ∈ oi, we have

ρ(ht) |ht|−ρ(ht) t t ε (1 − 3ε) µε(h ; o ) = P . i ˆt t ˆt t µε(h ; o ) h ∈oi i

t t t ˆt t ˆt t Given an information set oi, if h ∈ oi and h ∈ oi satisfy ρ(h ) < ρ(h ), then

µ (ht; ot) ερ(ht)(1 − 3ε)|ht|−ρ(ht) µ(ht; ot) ε i = → 0 = i as ε → 0, ˆt t ρ(hˆt) |hˆt|−ρ(hˆt) ˆ t µε(h ; oi) ε (1 − 3ε) µ(h; oi)

t t t t which implies µ(h ; oi) = 0. Therefore, to conclude that a history h ∈ oi does not belong t t ˆt t to the support of µ(·; oi), denoted supp(µ(·; oi)), it is enough to ﬁnd another history h ∈ oi which has the smaller number of mistakes than ht. This argument will prove Lemma 1.3.

13 i k1 k2 k3 k4 i k1 k2 k3 k4 G λt D λt A t ik1 =6 Λ 1 k2k1 = Λ t−1 t−1 t−1 t−1 t at−1 at−1 at−1 at−1 at−1 at−1 a aˆ aˆ aˆ − 1 i k1 k2 k3 k4 i k1 k2 k3 k4 t−2 t−2 t−2 t−2 t at−2 at−2 at−2 at−2 at−2 at−2 a aˆ aˆ aˆ − 2 i k1 k2 k3 k4 i k1 k2 k3 k4 t−3 t−3 t−3 t−3 t at−3 at−3 at−3 at−3 at−3 at−3 a aˆ aˆ aˆ − 3 i k1 k2 k3 k4 i k1 k2 k3 k4 t−4 t−4 t−4 t−4 t at−4 at−4 at−4 at−4 at−4 at−4 a aˆ aˆ aˆ − 4 i k1 k2 k3 k4 i k1 k2 k3 k4 ...... B t (a) history h (b) history hˆt

Figure 1.2: histories ht and hˆt

Lemma 1.3. Consider an information set ot = (as )t−1 and a history ht = (as )t−1 ∈ ot. i Gi s=1 N s=1 i For each s ≥ 1, let s ∗s s s P (oi (h )) = (λki, λik)k∈Gi ,

∗s t t where h is deﬁned as in (1.4). Suppose that h ∈ supp(µ(·; oi)). Then, for each τ ≥ 0, ∗ t+τ ∗t+τ t+τ σk(ok (h )) ∈ E(λik ) for all k ∈ Gi.

The formal proof of Lemma 1.3 is found in Appendix A. Here, we provide a sketch of proof.

Sketch of Proof. Suppose that, for some τ ≥ 0, σ∗ (ot+τ (h∗t+τ )) ∈/ E(λt+τ ) for some k ∈ k1 k1 ik1 1 G . In Steps 1 and 2, we show that if there is a surprise as to i by k ∈ G in period i k1 1 i s ≥ 1, then there is a mistake as0 where k is an agent who has distance m from i and km m s0 = s − m + 1 or s0 = s − m. For example, consider an agent i and a history ht described in Figure 1.2 (a). Suppose that at is a surprise to agent i. Since it is not a mistake, k1 at = σ∗(ot(ht)) = D and λt 6= Λ. Thus, there is an agent k ∈ G such that k 6= i and k1 k i 1 ik1 2 k1 2 λt = Λ. By the construction of P , there is a surprise between k and k in period t − 1. k2k1 2 1 Suppose that k surprises k in period t−1. Since λt = Λ, we have at−1 6= D . Since at−1 1 2 k2k1 k1 1 k1 is a surprise to agent i and at−1 6= D , by the construction of σ∗, at−1 is a mistake. Suppose k1 1 k1 that k surprises k in period t − 1. Then, at−1 is a mistake, or there is an agent k ∈ G 2 1 k2 3 k2 such that k3 6= k1 and λk3k2 = Λ. Continuing this procedure, it ends when t − m = 1 or k is an end agent. Therefore, for a surprise at to agent i by k , there is a mistake which m k1 1 induces at in shaded area A. In this case, we say that a surprise at to agent i is induced k1 k1 by the mistake which we ﬁnd. Similarly, if at−3 is a surprise to i by k , there is a mistake k1 i which induces at−3 in shaded area B. k1 In Step 3, we show that a mistake can induce at most one surprise to agent i. Since

14 there is no mistake in period s ≥ t, mistakes in the history ht are more than the actions which are agent i’s mistakes, surprises to agent i by k, or C1 played by k ∈ Gi. ˆt s t−1 s t−1 s t−1 In Step 4, we construct a history h = (âN )s=1 in which (âk)s=1 = (ak)s=1 for k ∈ Gi s t−1 ∗ s ˆ∗s t−1 and (âk)s=1 = (σk(ok(h )))s=1 for k∈ / Gi, which is described in Figure 1.2 (b). Trivially, t t hˆ is in the same information set as h . Furthermore, surprises to agent i by k ∈ Gi and s ˆt s aˆk ∈ h witha ˆk = C1 for k ∈ Gi are mistakes. We also show that there is no other mistake in hˆt. Then, since an action of agent i in ht is a mistake if and only if it is a mistake in hˆt, the number of mistakes in hˆt is equal to the number of actions which are agent i’s mistakes, t surprises to i by k ∈ Gi, or C1 played by k ∈ Gi. Therefore, the number of mistakes in hˆ t t t are smaller than that in h , and the argument before Lemma 1.3 implies µ(h ; oi) = 0.

Lemma 1.3 means that, at any information set, agent i believes that none of his neighbors will surprise him in the future under σ∗ and µ. Since an agent forms his expectation about his neighbors’ future actions without randomness, his neighbors’ future actions depend on his actions and he considers his neighbors’ future actions as deterministic. This makes it possible to calculate the continuation payoﬀs for each agent explicitly.

1.5 Sequential Equilibrium

In this section, we will show that the strategy σ∗ is a sequential equilibrium with the t t t t 0 t 0 belief system µ. Given oi with P (oi) = (λki, λik)k∈Gi , deﬁne K(λ, λ ; oi) for each λ, λ ∈ 0 t t t 0 0 t {0, 1,..., Λ} by K(λ, λ ; oi) = {k ∈ Gi : λki = λ, λik = λ }. That is, K(λ, λ ; oi) is the set t of agents k ∈ Gi such that k expects i to play ai ∈ E(λ) and agent i expects k to play t 0 t ak ∈ E(λ ). Because of (1.3) and Lemma 1.1, each oi satisﬁes one of the following seven cases.

t t t t Case A. P (oi) = (λki, λik)k∈Gi with λki = Λ for some k ∈ Gi. t t t t Case 1. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ−1; oi),K(0, 1; oi), and K(0, 0; oi). t t t t Case 2. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ−1; oi),K(λ, λ+1; oi),K(λ, λ; oi), t and K(λ, λ − 1; oi), where λ = 3,..., Λ − 1. t t t t Case 3. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ−1; oi),K(2, 3; oi),K(2, 2; oi), and t K(2, 1; oi). t t t t Case 4. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ−1; oi),K(1, 2; oi),K(1, 1; oi), and t K(1, 0; oi). t t t t Case B. P (oi) = (λki, λik)k∈Gi with λki = λ 6= Λ for all k ∈ Gi. t t Case 5. Gi is partitioned into K(0, 1; oi), and K(0, 0; oi). t t t Case 6. Gi is partitioned into K(λ, λ + 1; oi),K(λ, λ; oi), and K(λ, λ − 1; oi), where λ ∈ {2,..., Λ − 1}.

15 t t t Case 7. Gi is partitioned into K(1, 2; oi),K(1, 1; oi), and K(1, 0; oi).

t t Note that, in Cases 1 to 4, K(Λ, Λ; oi) ∪ K(Λ, Λ − 1; oi) 6= ∅. In addition, it is possible t t t t t that K(0, 1; oi) ∪ K(0, 0; oi) = ∅ in Case 1, K(λ, λ + 1; oi) ∪ K(λ, λ; oi) ∪ K(λ, λ − 1; oi) = ∅ t t t t t in Case 2, K(2, 3; oi) ∪ K(2, 2; oi) ∪ K(2, 1; oi) = ∅ in Case 3, and K(1, 2; oi) ∪ K(1, 1; oi) ∪ t t t t t K(1, 0; oi) = ∅ in Case 4. In Cases 5 to 7, K(0, 1; oi), K(0, 0; oi), K(λ, λ+1; oi), K(λ, λ; oi), t t t t K(λ, λ − 1; oi), K(1, 2; oi), K(1, 1; oi), and K(1, 0; oi) can be empty. Recall Lemma 1.3 saying that, under σ∗ and µ, agent i believes that his neighbors will not surprise him in the future. This means, agent i considers his neighbors’ future actions as deterministic. For instance, consider agent i in Case 1 in period t. If he chooses ∗ t t D1 following σi , it surprises the agents in K(0, 1; oi) ∪ K(0, 0; oi) but not the agents in t t K(Λ, Λ; oi) ∪ K(Λ, Λ − 1; oi). Since he believes that there is no surprise from his neighbors, t+1 t+1 t+1 t+1 t+1 he believes that, in the next period, P (oi ) = (λki , λik )k∈Gi satisfies (λki , λik ) = t t t+1 t+1 t (Λ − 1, Λ) for k ∈ K(0, 1; oi) ∪ K(0, 0; oi), (λki , λik ) = (Λ − 1, Λ − 1) for k ∈ K(Λ, Λ; oi), t+1 t+1 t and (λki , λik ) = (Λ − 1, Λ − 2) for k ∈ K(Λ, Λ − 1; oi). Thus, agent i believes that all his ∗ neighbors expect him to play D0 in period t + 1. Under σ , agent i will play D0 in period ∗ t + 1 not to surprise his neighbors. If agent i sticks on σi , he believes that there is no more surprise between i and k ∈ Gi after period t + 1, so he is sure of the actions which will be chosen by his neighbors in the future. On the other hand, if he chooses C0 deviating from ∗ t t σi in period t, it surprises the agents in K(Λ, Λ; oi) ∪ K(Λ, Λ − 1; oi) but not the agents t t t+1 t+1 t+1 in K(0, 1; oi) ∪ K(0, 0; oi). Then, he believes that P (oi ) satisfies (λki , λik ) = (0, 0) for t t t+1 t+1 t t k ∈ K(0, 1; oi) ∪ K(0, 0; oi) and (λki , λik ) = (Λ, Λ) for k ∈ K(Λ, Λ; oi) ∪ K(Λ, Λ − 1; oi) in ∗ period t+1. After then, if he follows σi by choosing D1 in period t+1, agent i believes that t+2 t+2 t+2 t+2 t+2 t t P (oi ) = (λki , λik )k∈Gi satisfies (λki , λik ) = (Λ − 1, Λ) for k ∈ K(0, 1; oi) ∪ K(0, 0; oi) t+1 t+1 t t and (λki , λik ) = (Λ − 1, Λ − 1) for k ∈ K(Λ, Λ; oi) ∪ K(Λ, Λ − 1; oi) in period t + 2. In this manner, we can determine the actions which, agent i believes, will be chosen by his t t ∗ ∗ oi neighbors and himself in the future. For oi and σi , we denote by σi |a a strategy of agent t ∗ oi t ∗ i such that σi |a (oi) = a and it agrees with σi at any other information set. For each case, t ∗ ∗ ∗ oi ∗ the actions in periods s ≥ t under (σi , σ−i) and (σi |a , σ−i) are described in Appendix B. Proposition 1.1 is the main result of this paper.

Proposition 1.1. There is δ∗ ∈ (0, 1) such that, for any δ ∈ (δ∗, 1), there is a sequential equilibrium which supports cooperation and in which cooperation resumes in ﬁnite periods after any history.

Proof. See Appendix C.

The proof is found in Appendix C. In the proof, it is shown that, for any Λ ≥ 2, there exists δ∗ ∈ (0, 1) such that, for any δ ∈ (δ∗, 1), the strategy σ∗ with Λ is a sequential equilibrium. More precisely, we ﬁx Λ ≥ 2 and, for each case, show that, for suﬃciently

16 t ∗ t ∗ oi ∗ t large δ, CUi(σ ; oi) ≥ CUi(σi |a , σ−i; oi) for any a ∈ {C0,C1,D0,D1}. Since each agent i considers the future actions of his neighbors as deterministic, we can calculate and compare t ∗ t ∗ oi ∗ t CUi(σ ; oi) and CUi(σi |a , σ−i; oi) for a ∈ {C0,C1,D0,D1} explicitly. Then, the one deviation property of sequential equilibrium implies that σ∗ is a sequential equilibrium. Trivially, under σ∗, all agents play cooperation along the equilibrium path and, as see in Lemma 1.2, cooperation resumes in finite periods after any history. Note that the strategy σ∗ with 2-period punishment can be a sequential equilibrium for sufficiently patient agents. This relies on the assumption that g is small enough. Since the current gain from defection is smaller than two-period benefit from cooperation, two periods is enough for punishment to enforce agents to stick on playing cooperation if agents are sufficiently patient. We also note that, although a strategy σ∗ with arbitrary large Λ ≥ 2 can be a sequential equilibrium for sufficiently patient agents, σ∗ with Λ = ∞ cannot be a sequential equilibrium. The strategy σ∗ with Λ = ∞ is a trigger strategy that a surprise causes a permanent punishment. As mentioned earlier, this trigger strategy cannot be a sequential equilibrium since an agent may not want to punish a defector for fear of being punished by other neighbors. Finally, since a strategy σ∗ with fixed Λ ≥ 2 is a sequential equilibrium for any sufficiently high δ, σ∗ with Λ can be interpreted as a uniform equilibrium for sufficiently high δ in the sense that σ∗ does not depend on δ as long as δ is high enough. Uniformity seems to be another desirable property of equilibria in repeated games. If an equilibrium is uniform for sufficiently large discount factor, we can remove the restriction that agents share a common discount factor as long as all agents are sufficiently patient.

1.6 Discussion

In the previous section, we show that σ∗ is a sequential equilibrium in which cooperation resumes after any history. The role of local communication in σ∗ is to enable agents to inform their neighbors when they start a new defection phase. Consider a repeated prisoner’s dilemma game without local communication. That is, each agent has only two actions {C,D} in each period. The payoﬀs of prisoner’s dilemma game between two linked agents are given as in Table 1.1. Suppose, in addition, that each i employs a trigger strategyσ ¯i such that he plays D if and only if he observed D played by himself or his neighbors in the past history.16 In an environment without communication, one can easily show that the trigger strategyσ ¯ is a sequential equilibrium if δ ∈ (0, 1) satisﬁes that g/(1 + g) ≤ δ ≤

16Xue (2004) discusses a trigger strategy for an environment where agents are located in a line-shaped network, while agents in our model are located in a minimally connected network. The argument in Xue (2004) can be applied to an environment with general networks.

17 t 1 2 3

1 C ← D → C 2 C → D D h3 3 DD D 4 DD D . . . . Λ+3 . . . . h Λ D D D Λ + 1 D DD→ σˆ Λ + 2 DDD Λ + 3 CDD← Λ + 4 DDD

Figure 1.3: σˆ-path conditioning on h3

g/(1 + g) + l/((|Gi| − 1)(1 + g)) for all i. Trivially,σ ¯ supports cooperation. If l is small enough and g/(1 + g) + l/((|Gi| − 1)(1 + g)) < δ < 1 for some i,σ ¯ cannot be a sequential equilibrium. The intuitive reason is that agent i, when he observes one of his neighbors playing defection, may have an incentive to play C for fear of being punished by other neighbors. Althoughσ ¯ cannot be a sequential equilibrium for small l and high δ, Ellison (1994) provides an idea to construct a sequential equilibrium supporting cooperation for sufficiently high δ. That is, agents divide the game into T replica games where tth replica game is played in periods t, T + t, 2T + t, . . ., and they play the trigger strategyσ ¯ in each replica game and ignore observations in other replica games. Here, T is chosen to satisfy g/(1 + g) ≤ T δ ≤ g/(1 + g) + l/((|Gi| − 1)(1 + g)) for all i. Under this modification ofσ ¯, if an agent deviates from cooperation by mistake in a replica game, cooperation never resumes in that replica game. Thus, it is not stable though it can be a sequential equilibrium which supports cooperation. One may be tempted to find an equilibrium in which cooperation resumes after any history by considering a strategy with finite-period punishment. For instance, consider a strategyσ ˆ such that each agent plays C when he did not observe D in his past history. If he is surprised by his neighbor in period t, he plays D for following Λ periods and C thereafter. If he surprises his neighbor by playing D, he plays D for following Λ − 1 periods and C thereafter. If he surprises his neighbor by playing C, he plays D for following Λ periods and C thereafter.17 However, we can find a history after which cooperation never resumes

17We can defineσ ˆ formally in a similar way to the way we define the strategy σ∗. That is, we first define t t t t ∗ the phase P (oi) = (λki, λik)k∈Gi of each information set oi in the same way as for σ with an expectation function Eˆ : {0, 1,..., Λ} → {C,D} given by Eˆ(0) = C and Eˆ(λ) = D for λ 6= 0. Notice that Lemma 1.1 ˆ t does not hold in this case. The strategyσ î is as follows: if E(λki) = D for some k ∈ Gi, he plays D, and if ˆ t E(λki) = C for all k ∈ Gi, he plays C.

18 t 1 2 3 4 1 2 3 4 5 1 C0 C0 C0 C0 C0 ← D1 → C0 C0 C0 C0 h3 h˜3 2 C0 ← D1 → C0 C0 C0 D0 D1 → C0 C0 ← D1 3 ← D1 D0 D1 → C0 C0 D0 D0 D1 ↔ D1 D0 4 D0 D0 D0 D1 ↔ D1 D0 D0 ← D1 D1 → D0 5 D0 D0 D0 ←D1 D1 → D0 ←D1 D0 D0 D1 → ...... ∗ 3 ∗ 3 (a) αN (σ ; h ) (b) αN (σ ; h˜ )

∗ 3 ∗ 3 Figure 1.4: αN (σ ; h ) and αN (σ ; h˜ )

underσ ˆ. Moreover,σ ˆ is not a sequential equilibrium. For example, consider a network andσ ˆ-path conditioning on h3 described in Figure 1.3. A right-arrow (resp. a left-arrow) represents that the agent in the left side (resp. the right side) surprises the agent in the right side (resp. the left side). Since agent 2 surprises 3 by playing D in period 1 and there is no surprise between 2 and 3 until period Λ, agent 3 expects that agent 2 plays D until period Λ and C in period Λ + 1. However, since agent 1 surprises 2 by playing C in period 2, agent 2 will play D in period Λ + 1 to satisfy agent 1’s expectation. This makes agent 3 surprised again and so he starts a defection phase in period Λ + 2 again. Since agents 1 and 3 cannot agree on the period when agent 2 ends his defection phase, cooperation never resumes after history h3. To check the sequential rationality ofσ ˆ, notice that there is only 1 2 Λ+3 Λ+3 two mistakes a2 and a1 in h . We also can see that h is the only history which can Λ+3 Λ+3 survive in supp(ˆµ(·; o1 (h ))) for any belief systemµ ˆ consistent withσ ˆ. Then, playing C in period Λ + 3 cannot be the best response for agent 1 to the strategyσ ˆ−1, because agent 2 never plays C after period 3. In the paper, G is assumed to be minimally connected. If we drop this assumption, the future actions may not be uniquely determined among the histories in the support of t ∗ µ(·; oi). For instance, consider a network G = {12, 23, 34, 45, 51} and σ -paths conditioning on h3 and h˜3 in Figure 1.4. Notice that histories h3 and h˜3 are in the same information set 3 3 o3 of agent 3 and each of them has only one mistake. Since o3 cannot be reached without 3 ˜3 3 mistake, h and h may be in the support of µ(·, o3). However, the future actions of agent 3’s neighbors after period 2 are different, which makes it difficult to calculate the continuation payoffs. Moreover, if the network is not minimally connected, cooperation may not be recovered from some history under σ∗. Figure 1.5 provides an example where a network is not minimally connected and cooperation never resumes after h2 under σ∗. Here, the pattern of actions in periods 3 to 6 are infinitely repeated along the σ∗-path conditioning on h2.

19 1 2 3 4 t ← 1 D0 → C0 C0 ← D0 → h2

2 D1 D1 ↔ D1 D1

3 D0 ←D1 D1 → D0

4 ← D1 D0 D0 D1 → ∗ 5 D1 → D0 D0 ← D1 σ

6 D0 D1 ↔ D1 D0

7 D0 ←D1 D1 → D0 ......

∗ 2 Figure 1.5: αN (σ ; h )

We also assume that g and l are sufficiently small to satisfy (1.1). If this assumption ∗ is not satisfied, we can find a history at which σi is not optimal for agent i. For example, consider a line-shaped network and history h2 described in Figure 1.6. Agent 2 employing ∗ σ2 is supposed to play D1 in period 2, which will cause a loss from agent 3’s defection in period Λ + 2. If he plays D0 instead of D1 in period 2, he will not suffer from this loss although the benefit from cooperation with his neighbor 1 in period Λ + 2 will be gone.

Thus, if the loss from being cheated is large enough, agent 2 has an incentive to play D0 in period 2. For the assumption of small g, consider a line-shaped network with 3 agents and t t t t t a history h where λ12 = λ21 = Λ, λ32 = 1, and λ23 = 0. Agent 2 is supposed to play D1 ∗ in period t under σ . If agent 2 plays D0 instead of D1 in period t, he will enjoy the gain from his neighbor 3 in period t + 1 but the resumption of cooperation is delayed. If g is large enough, the gain in period t + 1 is greater than the loss from delaying resumption of ∗ cooperation. Thus, following σ2 is not optimal for agent 2 in period t. Figure 1.7 describes the path for each case.

1.7 Conclusion

In the paper, we consider repeated prisoner’s dilemma games with local interaction and local communication and construct a sequential equilibrium which supports cooperation and in which cooperation resumes after any history. Construction of the sequential equilibrium heavily relies on the assumptions that a network is minimally connected and that gains and losses in a prisoner’s dilemma game between neighboring agents are suﬃciently small. In Cho (2007b), we show that, if there is global information processing such as public randomization, we can drop these assumptions in constructing a stable sequential equilibrium. It is not known yet whether or not these assumptions are necessary for stability of sequential

20 1 2 3 1 2 3 t 2 1 D0 C0 C0 D0 C0 C0 h 2 D1 D1 C0 D1 D0 C0 3 D0 D0 D1 D1 D1 D1 4 D0 D0 D0 D0 D0 D0 ...... Λ + 1 D0 D0 D0 D0 D0 D0 Λ + 2 C0 C0 D0 D0 D0 D0 Λ + 3 C0 C0 C0 C0 C0 C0 ...... ∗ ∗ 2 0 ∗ 2 αN (σ2 , σ−2; h ) αN (σ2, σ−2; h )

Figure 1.6: Assumption of small l

1 2 3 1 2 3 t ...... t − 1 ...... ht t D1 D1 C0 D1 D0 C0 t + 1 D0 D0 D1 D1 D1 C0 t D0 D0 D0 D0 D0 D1 +. 2 ...... Λ + t − 1 D0 D0 D0 D0 D0 D0 Λ + t C0 C0 D0 D0 D0 D0 Λ + t + 1 C0 C0 C0 C0 C0 D0 t C0 C0 C0 C0 C0 C0 Λ + . + 2 ...... ∗ ∗ t 0 ∗ t αN (σ2 , σ−2; h ) αN (σ2, σ−2; h )

Figure 1.7: Assumption of small g

equilibria without global information processing. In the equilibrium we construct, an agent can resolve the discrepancy of his neighbors’ beliefs on his future action, which helps us to construct a stable sequential equilibrium. Recall that the sequential equilibrium in Xue (2004) works only with line-shaped networks, though it does not allow local communication. In addition, as in Renault and Tomala (1998) and Hörnerand Olszewski (2006), we may want to consider a strategy where agents communicate with each other by choosing actions (cooperation or defection) in block of finite periods without choosing extra actions for communication. If agents are sufficiently patient, the effect of the actions in a communication block on the payoffs is negligible and so they can communicate with their neighbors with very little cost. In this strategy, agents may be able to reach an agreement on the agent who should be punished and the length of punishment periods. However, it is not trivial to check the sequential rationality for such

21 strategies to be sequential equilibria for our environment. Note that, in Renault and Tomala (1998), sequential rationality is not required for their solution concept and, in H¨ornerand Olszewski (2006), almost perfect monitoring is required for the folk theorem. Since local communication is not plausible in many situations, it seems worthwhile to study stable sequential equilibria without local communication or other information processing. We also note that the role of communication in our model is to create two types of defection: defection with one message and defection with the other message. Hence, if each agent has two types of defections in the stage game, the same stability result can be obtained without local communication.18 Communication has been discussed in many previous studies, such as Kandori and Matsushima (1998) and Compte (1998), as a tool to reveal private information, but there are few papers in which agents can generate types of actions by communicating with each other. It also seems interesting to study communication as a device generating types of actions. Finally, the equilibrium we construct has the property that each agent’s strategy does not depend on the structure of networks he cannot observe. For example, his strategy does not depend on the number of links his neighbors have (given that the network is minimally connected). We can interpret this property of equilibria as robustness to irrelevant network structures, which has not yet been studied. If an equilibrium is robust to irrelevant network structures, it is also an equilibrium for the game with incomplete information on the network structure.

Appendix A: Proof of Lemma 1.3

Fix an information set ot = (as )t−1 and a history ht = (as )t−1 ∈ ot. For convenience, let i Gi s=1 N s=1 i ∗∞ s ∗ t ∞ s ∞ s ∗s s s 19 h = (αN (σ ; h ))s=1 = (aN )s=1 and P (oi (h )) = (λki, λik)k∈Gi for each s ≥ 1. Suppose that, for some τ ≥ 0, σ∗ (ot+τ (h∗t+τ )) ∈/ E(λt+τ ) for some k ∈ G . If we ﬁnd hˆt ∈ ot such that k1 k1 ik1 1 i i ˆt t t t ρ(h ) < ρ(h ), then the argument before Lemma 1.3 implies that h ∈/ supp(µ(·; oi)), which will complete the proof. We deﬁne κi(x) for x = 1 ..., M¯ as in the proof of Lemma 1.2. In addition, for distinct i, j ∈ N, a chain between i and j is denoted by i ↔ j.

Step 1. Let km−1 ∈ κi(m − 1), km ∈ κi(m), and km−1km ∈ G where 2 ≤ m + 1 ≤ M¯ . Suppose that as ∈/ E(λs ). If there is no agent k ∈ G such that k ∈ κ (m + 1) and km km−1km m+1 km m+1 i as−1 ∈/ E(λs−1 ), then either km+1 kmkm+1

s ∗ s ∗s akm 6= σkm (okm (h )), or (1.5) as−1 6= σ∗ (os−1(h∗s−1)) and as−1 ∈ E(λs−1 ). (1.6) km km km km km−1km

Proof. Suppose that, for some m with 2 ≤ m + 1 ≤ M,¯ as ∈/ E(λs ) and there is no agent km km−1km k ∈ G such that k ∈ κ (m + 1) and as−1 ∈/ E(λs−1 ). In addition, suppose that m+1 km m+1 i km+1 kmkm+1 18I thank an anonymous referee for pointing out this aspect. 19 ∗ t ∗∞ t By the deﬁnition of αN (σ ; h ), h agrees with h for the actions in periods s ≤ t − 1.

22 as = σ∗ (os (h∗s)). Since σ∗ (os (h∗s)) = as ∈/ E(λs ), by the construction of σ∗, km km km km km km km−1km σ∗ (os (h∗s)) = D and λs 6= Λ. Thus, there is an agent k ∈ G such that k 6= k km km 1 km−1km m+1 km m+1 m−1 and λs = Λ. Then, only two cases are possible in period s − 1: km+1km

as−1 ∈/ E(λs−1 ) (1.7) km+1 kmkm+1 as−1 ∈/ E(λs−1 ) and as−1 6= D (1.8) km km+1km km 1

Since (1.7) contradicts our assumption, we have as−1 ∈/ E(λs−1 ) and as−1 6= D . By the km kmkm+1 km 1 construction of σ∗, as−1 ∈/ E(λs−1 ) and as−1 6= D imply that as−1 6= σ∗ (os−1(h∗s−1)). km kmkm+1 km 1 km km km Furthermore, since as−1 ∈/ E(λs−1 ) and as−1 6= D imply a contradiction that λs = Λ, we km km−1km km 1 km−1km have as−1 6= σ∗ (os−1(h∗s−1)) and as−1 ∈ E(λs−1 ). km km km km km−1km

Given an agent i, for each k ∈ Gi, we denote γi(k) as the set of agents j such that the chain between i and j contains k. That is, j ∈ γi(k) if and only if k ∈ i ↔ j. Since G is minimally connected, N is partitioned into {i} and γi(k) for k ∈ Gi. Step 2. Let k ∈ G . If as ∈/ E(λs ) then, for some m with 1 ≤ m ≤ M¯ , there is an agent 1 i k1 ik1 km ∈ κi(m) ∩ γi(k1) such that

as−m+1 6= σ∗ (os−m+1(h∗s−m+1)) and as−m+1 ∈/ E(λs−m+1 ), or (1.9) km km km km km−1km as−m 6= σ∗ (os−m(h∗s−m)) and as−m ∈ E(λs−m ). (1.10) km km km km km−1km

Proof. Let as ∈/ E(λs ) for some k ∈ G . If (1.9) and (1.10) do not hold for m = 1, then Step 1 k1 ik1 1 i implies that there is an agent k ∈ G such that k ∈ κ (2) and as−1 ∈/ E(λs−1 ). Then, if (1.9) and 2 k1 2 i k2 k1k2 (1.10) do not hold for m = 2, there is an agent k ∈ G such that k ∈ κ (3) and as−2 ∈/ E(λs−2 ). 3 k2 3 i k3 k2k3 Continuing this procedure, we eventually have a contradiction that there is no agent km ∈ κi(m + 1) such that as−m+1 ∈/ E(λs−m+1 ) because s − m + 1 = 0 or m = M¯ . This proves Step 2. km+1 kmkm+1

Let as ∈/ E(λs ). From Step 2, we know that there is a mistake as−m+1 such that as−m+1 6= k1 ik1 km km σ∗ (os−m+1(h∗s−m+1)) and as−m+1 ∈/ E(λs−m+1 ), or as−m such that as−m 6= σ∗ (os−m(h∗s−m)) km km km km−1km km km km km and as−m ∈ E(λs−m ), where k ∈ κ (m) ∩ γ (k ). In this case, we say that a surprise as is km km−1km m i i 1 k1 induced by mistake as−m+1 or as−m, respectively. km km 0 s ∗∞ s0 ∗∞ s s Step 3. Let k ∈ Gi and k ∈ Gi, and let ak ∈ h and ak0 ∈ h satisfy that ak ∈/ E(λik) and s0 s0 s sˆ s0 sˆ0 0 0 0 a 0 ∈/ E(λ 0 ). Let a be induced by a and a 0 be induced by a . If s 6= s or k 6= k , then sˆ 6=s ˆ or k ik k kˆ k kˆ0 kˆ 6= kˆ0. This means that a mistake can induce at most one surprise to agent i.

0 ˆ ˆ0 0 0 ˆ ˆ0 Proof. Suppose that k 6= k , then k ∈ γi(k) and k ∈ γi(k ). Since γi(k)∩γi(k ) = ∅, we have k 6= k . Suppose that k = k0 and s 6= s0. Without loss of generality, let s > s0. In addition, suppose that 0 kˆ = kˆ0 ands ˆ =s ˆ0, so asˆ = asˆ . Since G is minimally connected, there is a unique chain k ↔ kˆ = k0 ↔ kˆ kˆ0 ˆ0 0 ˆ ˆ0 k = {k1, . . . , km} such that k1 = k = k , km = k = k , and klkl+1 ∈ G for all l = 1, . . . , m−1. Then, 0 0 as−m = asˆ = asˆ = as −m+1. From Step 2, we have asˆ = as−m ∈ E(λs−m ) = E(λsˆ ) km km km km km km km−1km km−1km 0 0 and asˆ = as −m+1 ∈/ E(λs −m+1 ) = E(λsˆ ), which is a contradiction. km km km−1km km−1km

s ∗∞ From Steps 1 to 3, we know that for each surprise ak ∈ h to agent i by k ∈ Gi, there is a s ∗ s t mistake which induces only ak. Furthermore, since C1 is never played under σ , ak ∈ h for k ∈ Gi

23 s satisfying ak = C1 is a mistake. Therefore, we have

t s t s ∗ s ∗s s ∗∞ s s ρ(h ) ≥ |{ai ∈ h : ai 6= σi (oi (h ))}| + |{ak ∈ h : ak ∈/ E(λik), k ∈ Gi}| s t s s +|{ak ∈ h : ak = C1 ∈ E(λik), k ∈ Gi}| s t s ∗ s ∗s s t s s > |{ai ∈ h : ai 6= σi (oi (h ))}| + |{ak ∈ h : ak ∈/ E(λik), k ∈ Gi}| s t s s +|{ak ∈ h : ak = C1 ∈ E(λik), k ∈ Gi}|.

ˆt s t−1 ˆt t ˆt t Step 4. There is a history h = (ˆaN )s=1 such that h ∈ oi and ρ(h ) < ρ(h ).

ˆs τ s−1 Proof. We construct a history h = (ˆaN )τ=1 for each s with 1 ≤ s ≤ t iteratively as follows: Let hˆ1 = ∅ and, for each s satisfying 2 ≤ s ≤ t,

ˆs ˆs−1 s−1 h = (h , aˆN ) where s−1 s−1 aˆk = ak if k ∈ Gi, s−1 ∗ s−1 ˆs−1 aˆk = σk(ok (h )) if k∈ / Gi.

ˆt ˆt t t s ˆs ˆs ˆs By the construction of h , we have h ∈ oi(h ). For each s, let P (oi (h )) = (λki, λik)k∈Gi . s−1 ˆt s−1 t Notice thata ˆk ∈ h is a surprise to agent i if and only if ak ∈ h is a surprise to i. s−1 ˆt Furthermore, since there is no mistake for agent k∈ / Gi, any surprisea ˆk ∈ h to agent i is a s−1 ˆs−1 s−1 ∗ s−1 ˆs−1 mistake. That is,a ˆk ∈/ E(λik ) impliesa ˆk 6= σk(ok (h )). Furthermore, since C1 is never ∗ s ˆt s ˆs s played under σ , an actiona ˆk ∈ h satisfyinga ˆk ∈ E(λik) anda ˆk = C1 for k ∈ Gi is a mistake. s ˆt s ˆs s Now, we want to show that, ifa ˆk ∈ h for k ∈ Gi is a mistake, thena ˆk ∈/ E(λik) ora ˆk = ˆs s ˆt ˆs ˆs C1 ∈ E(λik). Suppose thata ˆk ∈ h for k ∈ Gi is a mistake where λik = λk0k = λ ∈ {0,..., Λ − 1} 0 s ˆs s ˆs 0 for all k ∈ Gk. Suppose in addition thata ˆk ∈ E(λik). Sincea ˆk ∈ E(λk0k) for all k ∈ Gk and s ∗ s ˆs s ˆs s ˆt aˆk 6= σk(ok(h )), we havea ˆk = C1 ∈ E(λki). Suppose that there is a mistakea ˆk ∈ h for k ∈ Gi s ˆs ˆs 0 such thata ˆk ∈ E(λik) where λk0k = Λ for some k ∈ Gk. Lets ¯ denote the earliest period when s¯ such a mistake exists. Let k1 ∈ Gi be an agent who makes the mistakea ˆk in periods ¯ and k2 ∈ Gk1 be an agent with λˆs¯ = Λ. Note thats ¯ > 1 since λˆ1 = 0. Sincea ˆs¯ 6= σ∗ (os¯ (hˆs¯)) = D and k2k1 k2k1 k1 k1 k1 1 aˆs¯ ∈ E(λˆs¯ ), we have λˆs¯ 6= Λ, i 6= k . Since λˆs¯ = Λ, by the construction of P , we have either k1 ik1 ik1 2 k2k1 (i)a ˆs¯−1 ∈/ E(λˆs¯−1 ) or (ii)a ˆs¯−1 ∈/ E(λˆs¯−1 ) and as¯−1 6= D . If (i) is the case, Step 2 implies that k2 k1k2 k1 k2k1 k1 1 ˆt ˆt there is a mistake by some agent k∈ / Gi in h , which contradicts the construction of h . If (ii) is the case, thena ˆs¯−1 is the mistake anda ˆs¯−1 ∈ E(λˆs¯−1) since λˆs¯ 6= Λ. Then, by the definition ofs ¯, k1 k1 ik1 ik1 ˆs¯−1 ˆs¯−1 0 s¯−1 ˆs¯−1 we should have λ = λ 0 = λ ∈ {0,..., Λ − 1} for all k ∈ G , which impliesa ˆ ∈ E(λ 0 ) ik1 k k1 k1 k1 k k1 0 s ˆt for all k ∈ Gk1 . However, this contradicts that (ii) is true. Therefore, ifa ˆk ∈ h for k ∈ Gi is a s ˆs mistake, then it is a surprise to agent i or it satisfiesa ˆk = C1 ∈ E(λik). s ∗s s ˆs s t s Furthermore, since oi (h ) = oi (h ) for all τ ≤ t,a î is a mistake in h if and only if ai is a

24 t s ˆs s s mistake in h , and for k ∈ Gi and for s ≤ t − 1,a ˆk ∈/ E(λik) if and only if ak ∈/ E(λik). Therefore,

ˆt s ˆt s ∗ s ˆs s ˆt s ∗ s ˆs ρ(h ) = |{aˆi ∈ h : ai 6= σi (oi (h ))}| + |{aˆk ∈ h :a ˆk 6= σk(ok(h )), k ∈ Gi}| s ˆt s ∗ s ˆs s ˆt s ˆs = |{aˆi ∈ h : ai 6= σi (oi (h ))}| + |{aˆk ∈ h :a ˆk ∈/ E(λik), k ∈ Gi}| s ˆt s ˆs +|{aˆk ∈ h :a ˆk = C1 ∈ E(λik), k ∈ Gi}| s t s ∗ s ∗s s t s s = |{ai ∈ h : ai 6= σi (oi (h ))}| + |{ak ∈ h : ak ∈/ E(λik), k ∈ Gi}| s t s ˆs +|{ak ∈ h : ak = C1 ∈ E(λik), k ∈ Gi}| < ρ(ht).

This completes the proof.

Appendix B: Actions in Periods s ≥ t

t t t t Case A. P (oi) = (λki, λik)k∈Gi with λki = Λ for some k ∈ Gi.

t t t t Case 1. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ − 1; oi),K(0, 1; oi), and K(0, 0; oi).

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· ∗ ∗ Λ (σi , σ−i) KΛ D1 D0 D0 ··· D0 C0 C0 C0 ··· Λ−1 KΛ D0 D0 D0 ··· C0 C0 C0 C0 ··· 1 K0 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K0 C0 D1 D0 ··· D0 D0 C0 C0 ··· {i} D0 D1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σ | , σ ) K D1 D1 D0 ··· D0 D0 C0 C0 ··· i D0 −i Λ Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K0 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K0 C0 D1 D0 ··· D0 D0 C0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σi |C , σ−i) KΛ D1 D1 D0 ··· D0 D0 C0 C0 ··· Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K0 D0 C0 D1 ··· D0 D0 D0 C0 ··· 0 K0 C0 C0 D1 ··· D0 D0 D0 C0 ···

C ∈ {C0,C1}

25 t t t t Case 2. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ−1; oi),K(λ, λ+1; oi),K(λ, λ; oi), and K(λ, λ− t 1; oi), where λ = 3,..., Λ − 1.

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· ∗ ∗ Λ (σi , σ−i) KΛ D1 D0 D0 ··· D0 C0 C0 C0 ··· Λ−1 KΛ D0 D0 D0 ··· C0 C0 C0 C0 ··· λ+1 Kλ DD1 D0 ··· D0 D0 C0 C0 ··· λ Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ−1 Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· {i} D0 D1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σ | , σ ) K D1 D1 D0 ··· D0 D0 C0 C0 ··· i D0 −i Λ Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ+1 Kλ DD0 D1 ··· D0 D0 D0 C0 ··· λ Kλ D0 D0 D1 ··· D0 D0 D0 C0 ··· λ−1 Kλ D0 D0 D1 ··· D0 D0 D0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σi |C , σ−i) KΛ D1 D1 D0 ··· D0 D0 C0 C0 ··· Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ+1 Kλ DD1 D0 ··· D0 D0 C0 C0 ··· λ Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ−1 Kλ D0 D1 D0 ··· D0 D0 C0 C0 ···

C ∈ {C0,C1}

D = D1 if λ = Λ − 1 and D = D0 otherwise.

t t t t t Case 3. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ − 1; oi),K(2, 3; oi),K(2, 2; oi), and K(2, 1; oi).

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· ∗ ∗ Λ (σi , σ−i) KΛ D1 D0 D0 ··· D0 C0 C0 C0 ··· Λ−1 KΛ D0 D0 D0 ··· C0 C0 C0 C0 ··· 3 K2 D0 D1 D0 ··· D0 D0 C0 C0 ··· 2 K2 D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K2 D0 D1 D0 ··· D0 D0 C0 C0 ··· {i} D0 D1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σ | , σ ) K D1 D1 D0 ··· D0 D0 C0 C0 ··· i D0 −i Λ Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 3 K2 D0 D0 D1 ··· D0 D0 D0 C0 ··· 2 K2 D0 D0 D1 ··· D0 D0 D0 C0 ··· 1 K2 D0 C0 D1 ··· D0 D0 D0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σi |C , σ−i) KΛ D1 D1 D0 ··· D0 D0 C0 C0 ··· Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 3 K2 D0 D1 D0 ··· D0 D0 C0 C0 ··· 2 K2 D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K2 D0 D1 D0 ··· D0 D0 C0 C0 ···

C ∈ {C0,C1}

26 t t t t t Case 4. Gi is partitioned into K(Λ, Λ; oi), K(Λ, Λ − 1; oi),K(1, 2; oi),K(1, 1; oi), and K(1, 0; oi).

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· ∗ ∗ Λ (σi , σ−i) KΛ D1 D0 D0 ··· D0 C0 C0 C0 ··· Λ−1 KΛ D0 D0 D0 ··· C0 C0 C0 C0 ··· 2 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K1 C0 D1 D0 ··· D0 D0 C0 C0 ··· {i} D0 D1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σ | , σ ) K D1 D1 D0 ··· D0 D0 C0 C0 ··· i D0 −i Λ Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 2 K1 D0 D0 D1 ··· D0 D0 D0 C0 ··· 1 K1 D0 C0 D1 ··· D0 D0 D0 C0 ··· 0 K1 C0 C0 D1 ··· D0 D0 D0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ Λ (σi |C , σ−i) KΛ D1 D1 D0 ··· D0 D0 C0 C0 ··· Λ−1 KΛ D0 D1 D0 ··· D0 D0 C0 C0 ··· 2 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K1 C0 D1 D0 ··· D0 D0 C0 C0 ···

C ∈ {C0,C1}

t t t t Case B. P (oi) = (λki, λik)k∈Gi with λki = λ 6= Λ for all k ∈ Gi.

t t Case 5. Gi is partitioned into K(0, 1; oi), and K(0, 0; oi).

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· ∗ ∗ (σi , σ−i), {i} CC0 C0 ··· C0 C0 C0 C0 ··· t ∗ oi ∗ 1 (σ | , σ ) K D0 C0 C0 ··· C0 C0 C0 C0 ··· i C1 −i 0 0 K0 C0 C0 C0 ··· C0 C0 C0 C0 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· t ∗ oi ∗ 1 (σ | , σ ) K D0 D1 D0 ··· D0 D0 C0 C0 ··· i D1 −i 0 0 K0 C0 D1 D0 ··· D0 D0 C0 C0 ··· {i} D0 D1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ 1 (σ | , σ ) K D0 D1 D0 ··· D0 D0 C0 C0 ··· i D0 −i 0 0 K0 C0 D1 D0 ··· D0 D0 C0 C0 ··· ot C = C for (σ∗, σ∗ ) and C = C for (σ∗| i , σ∗ ) 0 i −i 1 i C1 −i

27 t t t Case 6. Gi is partitioned into K(λ, λ+1; oi),K(λ, λ; oi), and K(λ, λ−1; oi), where λ ∈ {2,..., Λ− 1}.

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + λ − 2 t + λ − 1 t + λ t + λ + 1 ··· {i} D0 D0 D0 ··· D0 D0 C0 C0 ··· ∗ ∗ λ+1 (σi , σ−i) Kλ DD0 D0 ··· D0 D0 D0 C0 ··· λ Kλ D0 D0 D0 ··· D0 D0 C0 C0 ··· λ−1 Kλ D0 D0 D0 ··· D0 C0 C0 C0 ··· strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· t ∗ oi ∗ λ+1 (σ | , σ ) K DD1 D0 ··· D0 D0 C0 C0 ··· i D1 −i λ λ Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ−1 Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ λ+1 (σi |C , σ−i) Kλ DD1 D0 ··· D0 D0 C0 C0 ··· λ Kλ D0 D1 D0 ··· D0 D0 C0 C0 ··· λ−1 Kλ D0 D1 D0 ··· D0 D0 C0 C0 ···

C ∈ {C0,C1}

D = D1 if λ = Λ − 1 and D = D0 otherwise.

t t t Case 7. Gi is partitioned into K(1, 2; oi),K(1, 1; oi), and K(1, 0; oi).

s ak, s ≥ t strategy k ∈ t t + 1 t + 2 ··· t + Λ − 1 t + Λ t + Λ + 1 t + Λ + 2 ··· {i} D0 C0 C0 ··· C0 C0 C0 C0 ··· ∗ ∗ 2 (σi , σ−i) K1 D0 D0 C0 ··· C0 C0 C0 C0 ··· 1 K1 D0 C0 C0 ··· C0 C0 C0 C0 ··· 0 K1 C0 C0 C0 ··· C0 C0 C0 C0 ··· {i} D1 D0 D0 ··· D0 C0 C0 C0 ··· t ∗ oi ∗ 2 (σ | , σ ) K D0 D1 D0 ··· D0 D0 C0 C0 ··· i D1 −i 1 1 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K1 C0 D1 D0 ··· D0 D0 C0 C0 ··· {i} CD1 D0 ··· D0 D0 C0 C0 ··· t ∗ oi ∗ 2 (σi |C , σ−i) K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 1 K1 D0 D1 D0 ··· D0 D0 C0 C0 ··· 0 K1 C0 D1 D0 ··· D0 D0 C0 C0 ···

C ∈ {C0,C1} Appendix C: Proof of Proposition 1.1

Let Λ ≥ 2 be ﬁxed. For each case, we will show that, for suﬃciently large δ,

t ∗ t ∗ oi ∗ t CUi(σ ; oi) ≥ CUi(σi |a , σ−i; oi) for any a ∈ {C0,C1,D0,D1}. (1.11)

∗ Then, since N is ﬁnite and, for each case, the set of partitions of Gi is ﬁnite, there exists δ ∈ (0, 1) ∗ t ∗ such that, for any δ ∈ (δ , 1), (1.11) holds for any oi. The result that σ is a sequential equilibrium for δ ∈ (δ∗, 1) follows from the one deviation property of sequential equilibria. Recall that, under σ∗, all agents play cooperation along the equilibrium path and cooperation resumes after any history. λ0 0 t For notational convenience, we let Kλ = K(λ, λ ; oi) if there is no confusion.

28 Λ Λ−1 1 0 Case 1. Gi is partitioned into KΛ , KΛ , K0 , and K0 . ∗ Given σ−i, the continuation payoﬀs for each strategy of agent i is as follows:

∗ t 0 Λ−1 Λ−1 CUi(σ ; oi) = (1 − δ) |K0 | (1 + g) + (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ 1 0 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K0 | + |K0 | l + δ |Gi|,

t ∗ oi ∗ t 0 Λ+1 CUi(σ | , σ ; o ) = (1 − δ) |K |(1 + g) + δ |Gi|, i D0 −i i 0 (1.12) t ¡ ¢ ∗ oi ∗ t 0 Λ Λ−1 1 CUi(σi |C , σ−i; oi) = (1 − δ) |K0 | − (|KΛ | + |KΛ | + |K0 |)l ¡ 1 0 ¢ + (1 − δ) δ |K0 | + |K0 | (1 + g) Λ+1 ¡ Λ Λ−1 ¡ 1 0 ¢ ¢ Λ+2 + (1 − δ) δ |KΛ | + |KΛ | − |K0 | + |K0 | l + δ |Gi|. ¡ ¢ Λ Λ−1 1 0 where C ∈ {C0,C1}. Since |KΛ | + |KΛ | − |K0 | + |K0 | l ≥ 1 − (|Gi| − 1) l ≥ 0, we have

ot CU (σ∗; ot) − CU (σ∗| i , σ∗ ; ot) i i i i D0 −i i ¡ Λ−1 Λ−1 Λ ¡ Λ Λ−1 ¡ 1 0 ¢ ¢¢ = (1 − δ) δ |KΛ | (1 + g) + δ |KΛ | + |KΛ | − |K0 | + |K0 | l > 0.

From (1.12), we have

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |C , σ−i; oi) ¡ ¡ ¢ ¢ = (1 − δ) |K0|g + |KΛ| + |KΛ−1| + |K1| l 0 Λ Λ 0 (1.13) ¡ 1 0 ¢ Λ−1 Λ − (1 − δ) δ |K0 | + |K0 | (1 + g) + (1 − δ) δ |KΛ | (1 + g) 2 Λ ¡ Λ Λ−1 ¡ 1 0 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K0 | + |K0 | l + (1 − δ) δ |Gi|.

Dividing (1.13) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

Λ ¡ 1 0 ¡ Λ Λ−1 1 ¢ ¢ |Gi| + |KΛ | (1 + g) − |K0 | (1 + g) + |K0 | − |KΛ | + |KΛ | + |K0 | l 0 1 ¡ 1 0 ¢ ≥ 1 + |K0 | + |K0 | − |K0 | (1 + g) + |K0 | 0 1 ¡ 1 0 ¢ > (|Gi| − 1) g + |K0 | + |K0 | − |K0 | (1 + g) + |K0 | ¡ 1 ¢ = |Gi| − 1 − |K0 | g ≥ 0.

t ∗ t ∗ oi ∗ t This implies that, for suﬃciently large δ, CUi(σ ; oi) ≥ CUi(σi |C , σ−i; oi) holds.

Λ Λ−1 λ+1 λ λ−1 Case 2. Gi is partitioned into KΛ , KΛ , Kλ , Kλ , and Kλ , where λ = 3,..., Λ − 1.

29 In Case 2, agent i’s continuation payoﬀ for each strategy is as follows:

∗ t Λ−1 Λ−1 CUi(σ ; oi) = (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ λ+1 λ λ−1 ¢ ¢ + (1 − δ) δ |KΛ | + |KΛ | − |Kλ | + |Kλ | + |Kλ | l Λ+1 + δ |Gi|, ot ¡ ¡ ¢ ¢ (1.14) CU (σ∗| i , σ∗ ; ot) = (1 − δ) δΛ+1 |KΛ| + |KΛ−1| − |Kλ+1| + |Kλ| + |Kλ−1| l i i D0 −i i Λ Λ λ λ λ Λ+2 + δ |Gi|, t ∗ oi ∗ t Λ+1 CUi(σi |C , σ−i; oi) = − (1 − δ) |Gi|l + δ |Gi|. where C ∈ {C0,C1}. From (1.14), we have

ot CU (σ∗; ot) − CU (σ∗| i ; ot) i i i i D0 i Λ−1 Λ−1 (1.15) = (1 − δ) δ |KΛ | (1 + g) 2 Λ ¡ Λ Λ−1 ¡ λ+1 λ λ−1 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |Kλ | + |Kλ | + |Kλ | l + (1 − δ) δ |Gi|.

Λ−1 Dividing (1.15) by 1−δ and evaluating the right-hand-side at δ = 1, we have |KΛ | (1 + g)+|Gi| > 0. ot Thus, for suﬃciently large δ, CU (σ∗; ot) ≥ CU (σ∗| i ; ot) holds. From (1.14), we also have i i i i D0 i

t ∗ t ∗ oi t CUi(σ ; oi) − CUi(σi |C ; oi) Λ−1 Λ−1 (1.16) = (1 − δ) |Gi|l + (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ λ+1 λ λ−1 ¢ ¢ + (1 − δ) δ |KΛ | + |KΛ | − |Kλ | + |Kλ | + |Kλ | l .

Dividing (1.16) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

Λ−1 ¡ Λ Λ−1 ¡ λ+1 λ λ−1 ¢ ¢ |KΛ | (1 + g) + |Gi| + |KΛ | + |KΛ | − |Kλ | + |Kλ | + |Kλ | l > 0.

t ∗ t ∗ oi ∗ t This implies that, for suﬃciently large δ, we have CUi(σ ; oi) ≥ CUi(σi |C , σ−i; oi).

Λ Λ−1 3 2 1 Case 3. Gi is partitioned into KΛ , KΛ , K2 , K2 , and K2 . In Case 3, agent i’s continuation payoﬀ for each strategy is as follows:

∗ t Λ−1 Λ−1 CUi(σ ; oi) = (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ 3 2 1 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K2 | + |K2 | + |K2 | l + δ |Gi|,

ot CU (σ∗| i , σ∗ ; ot) = (1 − δ) δ|K1| (1 + g) i i D0 −i i 2 Λ+1 ¡ Λ Λ−1 ¡ 3 2 1 ¢ ¢ Λ+2 + (1 − δ) δ |KΛ | + |KΛ | − |K2 | + |K2 | + |K2 | l + δ |Gi|,

t ∗ oi ∗ t Λ+1 CUi(σi |C , σ−i; oi) = − (1 − δ) |Gi|l + δ |Gi|. (1.17)

30 where C ∈ {C0,C1}. From (1.17), we have

ot CU (σ∗; ot) − CU (σ∗| i , σ∗ ; ot) i i i i D0 −i i 1 Λ−1 Λ−1 (1.18) = − (1 − δ) δ|K2 | (1 + g) + (1 − δ) δ |KΛ | (1 + g) 2 Λ ¡ Λ Λ−1 ¡ 3 2 1 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K2 | + |K2 | + |K2 | l + (1 − δ) δ |Gi|.

Dividing (1.18) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

1 Λ−1 1 1 −|K2 | (1 + g) + |KΛ | (1 + g) + |Gi| ≥ − |K2 | − (|Gi| − 1) g + |K2 | + 1 > 0

ot This implies that, for suﬃciently large δ, CU (σ∗; ot) ≥ CU (σ∗| i , σ∗ ; ot) holds. From (1.17), we i i i i D0 −i i also have

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |C , σ−i; oi) Λ−1 Λ−1 (1.19) = (1 − δ) |Gi|l + (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ 3 2 1 ¢ ¢ + (1 − δ) δ |KΛ | + |KΛ | − |K2 | + |K2 | + |K2 | l .

Dividing (1.19) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

Λ−1 ¡ Λ Λ−1 ¡ 3 2 1 ¢ ¢ |Gi|l + |KΛ | (1 + g) + |KΛ | + |KΛ | − |K2 | + |K2 | + |K2 | l > 0.

t ∗ t ∗ oi ∗ t Thus, for suﬃciently large δ, CUi(σ ; oi) − CUi(σi |C , σ−i; oi) ≥ 0 holds.

Λ Λ−1 2 1 0 Case 4. Gi is partitioned into KΛ , KΛ , K1 , K1 , and K1 . In Case 4, agent i’s continuation payoﬀ for each strategy is as follows:

∗ t 0 Λ−1 Λ−1 CUi(σ ; oi) = (1 − δ) |K1 | (1 + g) + (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ 2 1 0 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K1 | + |K1 | + |K1 | l + δ |Gi|

ot ¡ ¢ CU (σ∗| i , σ∗ ; ot) = (1 − δ) |K0| (1 + g) + (1 − δ) δ |K1| + |K0| (1 + g) (1.20) i i D0 −i i 1 1 1 Λ+1 ¡ Λ Λ−1 ¡ 2 1 0 ¢ ¢ Λ+2 + (1 − δ) δ |KΛ | + |KΛ | − |K1 | + |K1 | + |K1 | l + δ |Gi| t ¡ ¡ ¢ ¢ ∗ oi ∗ t 0 Λ Λ−1 2 1 Λ+1 CUi(σi |C , σ−i; oi) = (1 − δ) |K1 | − |KΛ | + |KΛ | + |K1 | + |K1 | l + δ |Gi| where C ∈ {C0,C1}. From (1.20), we have

ot CU (σ∗; ot) − CU (σ∗| i , σ∗ ; ot) i i i i D0 −i i ¡ ¢ 1 0 Λ−1 Λ−1 (1.21) = − (1 − δ) δ |K1 | + |K1 | (1 + g) + (1 − δ) δ |KΛ | (1 + g) 2 Λ ¡ Λ Λ−1 ¡ 2 1 0 ¢ ¢ Λ+1 + (1 − δ) δ |KΛ | + |KΛ | − |K1 | + |K1 | + |K1 | l + (1 − δ) δ |Gi|.

Dividing (1.21) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

¡ Λ−1 1 0 ¢ |KΛ | − |K1 | − |K1 | (1 + g) + |Gi| ≥ − (|Gi| − 1) (1 + g) + |Gi| > 0.

31 ot Thus, for suﬃciently large δ, CU (σ∗; ot) ≥ CU (σ∗| i , σ∗ ; ot). From (1.20), we also have i i i i D0 −i i

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |C , σ−i; oi) ¡ ¡ ¢ ¢ 0 Λ Λ−1 2 1 Λ−1 Λ−1 (1.22) = (1 − δ) |K1 |g + |KΛ | + |KΛ | + |K1 | + |K1 | l + (1 − δ) δ |KΛ | (1 + g) Λ ¡ Λ Λ−1 ¡ 2 1 0 ¢ ¢ + (1 − δ) δ |KΛ | + |KΛ | − |K1 | + |K1 | + |K1 | l .

Dividing (1.22) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

0 ¡ Λ Λ−1 ¢ Λ−1 Λ Λ−1 0 |K1 |g + |KΛ | + |KΛ | l + |KΛ | (1 + g) + |KΛ | + |KΛ | − |K1 |l > 0

t ∗ t ∗ oi ∗ t Hence, for suﬃciently large δ, CUi(σ ; oi) ≥ CUi(σi |C , σ−i; oi) holds.

1 0 Case 5. Gi is partitioned into K0 , and K0 . In Case 5, the payoﬀ for each strategy is given as follows:

ot CU (σ∗; ot) = CU (σ∗| i , σ∗ ; ot) i i i i C1 −i i ¡ 0 1 ¢ = (1 − δ) |K0 | − |K0 |l + δ|Gi| (1.23) ot CU (σ∗| i , σ∗ ; ot) = (1 − δ) |K0| (1 + g) − (1 − δ) δΛ|G |l + δΛ+1|G | i i D1 −i i 0 i i ot CU (σ∗| i , σ∗ ; ot) = (1 − δ) |K0| (1 + g) + δΛ+1|G | i i D0 −i i 0 i

From (1.23), we have

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |D , σ−i; oi) 0 (1.24) ¡ 0 1 0 ¢ ¡ 2 Λ¢ = (1 − δ) |K0 | − |K0 |l − |K0 | (1 + g) + (1 − δ) δ + δ + ... + δ |Gi|.

Dividing (1.24) by 1 − δ and evaluating the right-hand-side at δ = 1, since Λ ≥ 2 > max{g, l}, we have 1 0 Λ|Gi| − |K0 |l − |K0 |g > 0. ot Thus, for suﬃciently large δ, we have CU (σ∗; ot) ≥ CU (σ∗| i , σ∗ ; ot). In addition, from (1.23), i i i i D0 −i i ∗ ∗ t ∗ ∗ t ∗ t CU (σ | t , σ ; o ) ≥ CU (σ | t , σ ; o ) holds. So, for suﬃciently large δ, CU (σ ; o ) ≥ i i oi,D0 −i i i i oi,D1 −i i i i ot CU (σ∗| i , σ∗ ; ot). i i D1 −i i

λ+1 λ λ−1 Case 6. Gi is partitioned into Kλ , Kλ , and Kλ , where λ ∈ {2,..., Λ − 1}. In Case 6, the payoﬀ for each strategy is as follows:

∗ t λ−1 λ−1 CUi(σ ; oi) = (1 − δ) δ |Kλ | (1 + g) λ ¡ λ−1 λ λ+1 ¢ λ+1 + (1 − δ) δ |Kλ | + |Kλ | − |Kλ |l + δ |Gi|

ot (1.25) CU (σ∗| i , σ∗ ; ot) = (1 − δ) δΛ|G | (−l) + δΛ+1|G | i i D1 −i i i i

t ∗ oi ∗ t Λ+1 CUi(σi |C , σ−i; oi) = (1 − δ) |Gi| (−l) + δ |Gi|

32 where C ∈ {C0,C1}. From (1.25), we have

ot CU (σ∗; ot) − CU (σ∗| i , σ∗ ; ot) i i i i D1 −i i ¡ ¢ λ−1 λ−1 λ λ−1 λ λ+1 (1.26) = (1 − δ) δ |Kλ | (1 + g) + (1 − δ) δ |Kλ | + |Kλ | − |Kλ |l Λ ¡ λ+1 Λ¢ + (1 − δ) δ |Gi|l + (1 − δ) δ + ... + δ |Gi|.

Dividing (1.26) by 1 − δ and evaluating the right-hand-side at δ = 1, we have

λ−1 λ−1 λ ¡ λ+1 ¢ |Kλ | (1 + g) + |Kλ | + |Kλ | + |Gi| − |Kλ | l + (Λ − λ) |Gi| > 0.

ot ot In addition, since CU (σ∗| i , σ∗ ; ot) ≥ CU (σ∗| i , σ∗ ; ot), we have that, for suﬃciently large δ, i i D1 −i i i i C −i i ot ot CU (σ∗; ot) ≥ CU (σ∗| i , σ∗ ; ot) ≥ CU (σ∗| i , σ∗ ; ot). i i i i D1 −i i i i C −i i

2 1 0 Case 7. Gi is partitioned into K1 , K1 , and K1 . In Case 7, the payoﬀ for each strategy is as follows:

∗ t 0 ¡ 1 0 2 ¢ 2 CUi(σ ; oi) = (1 − δ) |K1 | (1 + g) + (1 − δ) δ |K1 | + |K1 | − |K1 |l + δ |Gi|

ot CU (σ∗| i , σ∗ ; ot) = (1 − δ) |K0| (1 + g) − (1 − δ) δΛ|G |l + δΛ+1|G | (1.27) i i D1 −i i 1 i i t ¡ ¡ ¢ ¢ ∗ oi ∗ t 0 2 1 Λ+1 CUi(σi |C , σ−i; oi) = (1 − δ) |K1 | − |K1 | + |K1 | l + δ |Gi|

From (1.27), we have

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |D , σ−i; oi) 1 (1.28) ¡ 1 0 2 ¢ Λ ¡ 2 Λ¢ = (1 − δ) δ |K1 | + |K1 | − |K1 |l + (1 − δ) δ |Gi|l + (1 − δ) δ + ... + δ |Gi|.

Dividing (1.28) by 1 − δ and evaluating the right-hand-side at δ = 1, since Λ ≥ 2, we have

1 0 ¡ 2 ¢ |K1 | + |K1 | + |Gi| − |K1 | l + (Λ − 1) |Gi| > 0.

ot Thus, for suﬃciently large δ, we have CU (σ∗; ot) ≥ CU (σ∗| i , σ∗ ; ot). From (1.27), we also have i i i i D1 −i i

t ∗ t ∗ oi ∗ t CUi(σ ; oi) − CUi(σi |C , σ−i; oi) ¡ ¡ ¢ ¢ ¡ ¢ 0 2 1 1 0 2 (1.29) = (1 − δ) |K1 |g + |K1 | + |K1 | l + (1 − δ) δ |K1 | + |K1 | − |K1 |l ¡ 2 Λ¢ + (1 − δ) δ + ... + δ |Gi|.

Dividing (1.29) by 1 − δ and evaluating the right-hand-side at δ = 1, since Λ ≥ 2, we have

0 1 1 0 |K1 |g + |K1 |l + |K1 | + |K1 | + (Λ − 1) |Gi| > 0.

t ∗ t ∗ oi ∗ t Thus, for suﬃciently large δ, we have CUi(σ ; oi) ≥ CUi(σi |C , σ−i; oi).

33 Chapter 2

Public Randomization in the Repeated Prisoner’s Dilemma Game with Local Interaction

2.1 Introduction

A result in the long-term relationships is that people succeed in cooperating with others even when it is possible to gain in the short-term by deceiving others. The intuitive reason behind this result is that an agent behaves honestly because he is afraid of the collapse of cooperation which will ruin his future gains. The earliest work on this issue is done by Friedman (1971) and Fudenberg and Maskin (1986), in which each agent is assumed to monitor the actions of the others and agents can punish the deviator immediately. However, there are some situations in which perfect monitoring or immediate punishment is not possible. The first example is the random matching model, explored by Kandori (1992b) and Ellison (1994), where agents are randomly matched in each period and each agent does not know the actions his partner has played in the past. The other example is the overlapping generation model which is explored in Kandori (1992a) and Bhaskar (1998). In the overlapping generation model, agents cannot punish the defection of old generations who will die in the next period. In this paper, we are interested in the situation where agents play an infinitely repeated game and interactions and monitoring are local. That is, agents are directly or indirectly connected through a social or geometric network, and each agent interacts with his neighbors who are directly connected to him and observes only the actions which his neighbors played in the past. An example is monopolistic competition of firms suggested by Salop (1979). In this example, firms compete locally with their neighboring firms by setting a price. Each firm can cooperate with his neighbors by setting a high price or defect by setting a low

34 price. Thus, each firm plays a prisoner’s dilemma game against his neighboring firms but he cannot choose different actions against his different neighbors. The profit of a firm does not depend on the prices of other firms who are not directly connected to him. Thus, if there is a cost to observe the others’ prices, then a firm may not want to pay the cost to see the prices which are irrelevant to his profit.1 This makes the assumption of local observability reasonable when there is a cost to observe the prices offered by other firms. In this paper, we consider the prisoner’s dilemma game as a way of local interaction between two adjacent agents. The first question for this situation is: can cooperation be supported as an equilibrium? The answer for this question is ‘yes’ if agents are sufficiently patient. For reasoning, consider a trigger strategy such that each agent chooses defection if and only if he observes defection in his past history. One can show that the trigger strategy can be a sequential equilibrium which supports cooperation if agents are sufficiently patient and the loss from being cheated is large enough.2 If the loss from being cheated is small enough, then, due to Ellison (1994), we can construct a sequential equilibrium supporting cooperation for sufficiently patient agents. The basic idea to construct such an equilibrium is that agents divide the game into T -replica games where tth replica game is played in periods t, T + t, 2T + t, . . ., and each agent plays the trigger strategy for each replica game and ignores the observations in other replica games. Although the trigger strategy or the modified trigger strategy described above is a sequential equilibrium which supports cooperation, depending on the amount of loss from being cheated, cooperation is not stable to mistakes in the sense that a defection played by a mistake results in a complete breakdown of cooperation and cooperation is never recovered. This may not be a desirable property of equilibrium. Indeed, as we will see in this paper, if there is a small possibility of mistakes, stable equilibrium can bring a more efficient outcome than the trigger strategy equilibrium, though both equilibria give the same payoff in the limit as the possibility of mistakes decreases to zero. The purpose of this paper is to find a sequential equilibrium which supports cooperation and in which cooperation is eventually recovered from any history. We may attempt to construct a sequential equilibrium which is stable to mistakes and supports cooperation by having an agent punish his neighbors in finite periods when he observes deviation. However, the local observability may cause a discrepancy between the beliefs of agents, say i and j, on the period when their neighbor, say k, ends a defection phase that is a span of periods when he plays defection. If there is such a discrepancy in

1Ben-Porath and Kahneman (2003) and Miyagawa et al. (2008) explore the situation where there is a monitoring cost. 2Since a single defection causes the spread of defection over the network, one may call the equilibrium with this strategy a contagious equilibrium following Kandori (1992b).

35 beliefs at some history, then agent k may not be able to follow the beliefs of his neighbors i and j in the future. Then, since agent k will surprise i or j repeatedly, the defection phases will repeat infinitely and so cooperation is never recovered.3 To resolve this discrepancy in beliefs, we introduce a public randomization, an idea from Ellison (1994). In the real world, people in a society share some information through newspaper or television. We can interpret the public randomization as information shared by society through mass media. In the equilibrium we will construct, all agents in a society can restore cooperation in the period when a specific event happens, though they have to have a consensus about what the event is. Here, the distribution of public randomization does not depend on agents’ action, so an agent cannot infer the unobserved actions.4 The literature which shares the environment with this paper are Xue (2004) and Cho (2007a). Xue (2004) considers a society in which agents are located in a line-shaped network and construct a sequential equilibrium which is stable and supports cooperation. Though Xue (2004) does not require the device to restore cooperation in the same period, the equilibrium has an undesirable feature, that is, the strategy is complicated and difficult to implement. Cho (2007a) considers a society with a minimally connected network and allows agents to communicate locally with their neighbors. Cho (2007a) also assumes that the gain from cheating and the loss from being cheated are small enough and construct a sequential equilibrium in which agents recover cooperation in finite periods. This paper is organized as follows. Section 2.2 describes the environment and the equilibrium. In Section 2.3, we provides a strategy and show that it is a sequential equilibrium. Section 2.4 discusses efficiency of stable equilibrium and robustness, and 2.5 concludes.

2.2 The Model

The basic structure of the environment is described as follows. There is a set N = {1, . . . , n} of agents who live in inﬁnite periods. We assume that n ≥ 3. A network G through which each agent interacts with others is a collection of subsets of N containing only two agents. We denote ij, called a link, as an element {i, j} in G. If ij ∈ G, agents i and j are said to be linked or directly connected. In addition, we say that agent i (resp. agent j) is a neighbor of j (resp. neighbor of i). For each i, let Gi = {j ∈ N : ij ∈ G} denote the set of i’s neighbors

3If monitoring is perfect and agents are sufficiently patient, then the trigger strategy with the punishment of finite but sufficiently long periods can be a sequential equilibrium which supports cooperation. In this equilibrium, the perfect monitoring makes it possible for all agents to start and finish a defection phase in the same period. 4The Folk Theorem with imperfect monitoring and public randomization is explored in many papers, such as Fudenberg et al. (1994), Mailath and Morris (2002), and so on. In these papers, the distribution of public randomization depends on agents’ action, so a deviation can be stochastically detected by other agents.

36 i j C D C 1, 1 −l, 1 + g D 1 + g, −l 0, 0

Table 2.1: Payoﬀ of prisoner’s dilemma game

5 and let Gi = Gi ∪ {i}. Without loss of generality, we assume that G is connected. That 0 0 is, for all i, i ∈ N, there is a subset {i1, i2, . . . , iL} of N such that i1 = i, iL = i , and ilil+1 ∈ G for all l = 1,...,L − 1. At the beginning of each period t, a public randomization Qt is realized. We assume that Qt is drawn independently from a uniform distribution on [0, 1], That is,

Qt ∼ i.i.d.U[0, 1].

We denote qt as a realization of Qt. The sequence of the game is as follows. In each period t, qt is realized and all agents t observe q . Next, each agent i plays a prisoner’s dilemma game against his neighbors j ∈ Gi. The payoff of a prisoner’s dilemma game between i and j is given in Table 2.1 where g > 0 and l > 0. Let w(a, a0) denote the payoff of agent i in a prisoner’s dilemma game with 0 j ∈ Gi when i plays a and j plays a . In each period t, each agent has to choose the same action in prisoner’s dilemma games against different neighbors. t We denote ai ∈ {C,D} as the action which agent i chooses in period t. For a subset t t t ∞ t ∞ K ⊂ N, let aK = (ai)i∈K . We also let ai = (ai)t=1 and aK = ((ai)i∈K )t=1 for convenience. The payoff of agent i in a stage game in period t is given by X t t t ui(aN ) = w(ai, aj), j∈Gi

t when agents play aN in period t. Notice that the actions of other agents who are not directly connected to agent i does not affect the agent i’s payoff. We refer to a stage game described above as a prisoner’s dilemma game with local interaction. The payoff of agent i in a repeated prisoner’s dilemma game with local interaction, when aN is played, is the average of discounted sum of stage game payoffs:

X∞ X∞ X t−1 t t−1 t t Ui(aN ) = (1 − δ) δ ui(aN ) = (1 − δ) δ w(ai, aj). t=1 t=1 j∈Gi

t t s s+1 t−1 A history h in period t, represented as h = (aN , q )s=1, is the set of actions and

5 Since G is an undirected network, ij ∈ G is equivalent to ji ∈ G and j ∈ Gi if and only if i ∈ Gj .

37 the realization of public randomization until the period t.6 We denote Ht as the set of all possible histories in period t.7 In the environment, we assume that each agent cannot observe the action of other agents who are not directly connected to him. Thus, in each period t, agent i’s information set t oi is the set of histories where the actions of agents k ∈ Gi and the realizations of public t s s+1 t−1 0t 0s 0s+1 t−1 randomization are same. This means, h = (aN , q )s=1 and h = (aN , q )s=1 are in the same information set of agent i if and only if (as , qs+1)t−1 = (a0s , q0s+1)t−1 . With a Gi s=1 Gi s=1 slight abuse of notation, we write an information set of agent i as ot = (as , qs+1)t−1 .8We i Gi s=1 t t t t also write oi(h ) as the information set of agent i which h belongs to. Let Oi denote the t set of all information sets of agent i in period t. Note that, for each i, Oi is a partition of Ht for each period t. In this paper, we restrict our attention to pure strategies. A strategy of agent i is a function [∞ t σi : Oi → {C,D}. t=1 t t For example, σi(oi) = C means that, agent i chooses C when he is at an information set oi. Let Σi be the set of strategies of agent i.A strategy σ is a profile (σi)i∈N of each agent’s strategy. t ∞ Given a strategy σ, we denote (αN (σ))t=1 as a σ-path, which is defined iteratively as 1 1 1 1 t t−1 t−1 t follows: Let Oi = (∅,Q ) and αi (σ) = σi(Oi ). For each t ≥ 2, Oi = (O , α (σ),Q ) and i Gi t t t αi(σ) = σi(Oi). Here, αi(σ) is the action agent i chooses in period t under the strategy σ. Since agent i’s action in each period depends on the public randomization as well as the past t actions of agents in Gi, αi(σ) for t ≥ 1 is a probabilistic choice on {C,D} whose distribution 1 t t s−1 s t−1 t depends on Q ,...,Q . Furthermore, given a history h = (aN , q )s=1 ∈ H and strategy t s t ∞ t t t σ, we define a σ-path from h , denoted (αN (σ; h ))s=t, as follows. Let Oi = oi(h ) and t t t τ τ−1 τ−1 t τ τ t t α (σ; h ) = σi(O ). For τ ≥ t + 1, O = (O , α (σ; h ),Q ) and α (σ; h ) = σi(O ). i i i i Gi i i τ t Note that, for τ ≥ t + 1, αi (σ; h ) is a probabilistic choice on {C,D} whose distribution depends on Qt+1,...,Qτ . In this paper, we are interested in sequential equilibria. A belief system µ is a function t t which assigns each information set oi to a probability distribution µ(·; oi) on the set of t t t t t t t histories in oi. For an information set oi, µ(h ; oi) is the probability of h ∈ oi given that oi t t t t t is reached. Since oi is finite, h ∈ supp(µ(·; oi)) if and only if µ(h ; oi) > 0. A belief system is consistent with σ, if it is the limit of a sequence of belief systems which are generated by Beysian updating of fully mixed behavioral strategies converging to σ.9

6A history h1 in period 1 is represented as h1 = (∅, q1). 7In Kandori and Matsushima (1998) and Xue (2004), a (joint or global) history refers to a history ht and t a private history of agent i refers to an information set oi. 8 1 1 1 An information set oi of agent i in period 1 is represented as oi = (∅, q ). 9 ∞ t t t t t t t A sequence {µn}n=1 of belief systems converges to µ if, for each oi, µn(h ; oi) → µ(h ; oi) for all h ∈ oi.

38 t Let a strategy σ and a belief system µ be given. For each information set oi = (as , qs+1)t−1 , an expected payoﬀ of agent i conditioning on ot is deﬁned by Gi s=1 i

t EUi(σ; µ; oi) Xt−1 X τ−1 τ τ = (1 − δ) δ w(ai , aj ) τ=1 j∈G i   X X t  t−1 t t t t  + (1 − δ) µ(ht; oi) δ w(αi(σ; h ), αj(σ; h )) t ht∈o j∈Gi i    X X∞ X t  τ−1  τ t τ t  + (1 − δ) µ(ht; oi) δ EQt+1,...,Qτ w(αi (σ; h ), αj (σ; h )) . t t τ=t+1 j∈G h ∈oi i

A strategy σ is a sequential equilibrium if, for some belief system µ which is consistent with t σ, σ satisﬁes that, for each i and for each oi,

t 0 t 0 10 EUi(σ; µ; oi) ≥ EUi(σi, σ−i; µ; oi) for all σi ∈ Σi. (2.1)

If a strategy σ satisfies (2.1) for some µ, then σ is said to be sequentially rational under µ. Given a strategy σ and a belief system µ, we define a continuation expected payoff of t agent i from oi as

t CEUi(σ; µ; oi)   X X t  t t t t  = (1 − δ) µ(ht; oi) w(αi(σ; h ), αj(σ; h )) ht∈ot j∈Gi i    X X∞ X t  τ−t  τ t τ t  + (1 − δ) µ(ht; oi) δ EQt+1,...,Qτ w(αi (σ; h ), αj (σ; h )) . t t τ=t+1 j∈G h ∈oi i

t A fully mixed behavioral strategy βi of agent i is a function which assigns each information set oi to a t t ∞ distribution βi(·; oi) on {C,D} satisfying βi(a; oi) > 0 for all a ∈ {C,D}. A sequence {βin}n=1 of fully t t t mixed behavioral strategies of agent i converges to βi, if for each oi, βi(a; oi) → βi(a; oi) for all a ∈ {C,D}. t t If, for each oi, there is a ∈ {C,D} such that βi(a; oi) = 1, then the behavioral strategy βi is equivalent to t the pure strategy σi such that σi(oi) = a. 10Note that Kreps and Wilson (1982) define a sequential equilibrium for finite extensive form games. Although we consider an infinite extensive form game, we can apply their definition to our model without any conceptual innovation. Some previous studies, such as Kandori (1992b) and Xue (2004), use sequential equilibrium as a solution concept for infinitely repeated games.

39 Since

Xt−1 X t τ−1 τ τ t−1 t EUi(σ; µ; oi) = (1 − δ) δ w(ai , aj ) + δ CEUi(σ; µ; oi), τ=1 j∈Gi

(2.1) holds if and only if

t 0 t 0 CEUi(σ; µ; oi) ≥ CEUi(σi, σ−i; µ; oi) for all σi ∈ Σi.

2.3 Sequential Equilibrium

This section is devoted to the construction of a sequential equilibrium for sufficiently high δ. We first define a strategy which will show itself to be a sequential equilibrium. t t t For each information set oi, the phase of oi, denoted P (oi), is determined iteratively as 1 1 1 follows. For period 1, P (oi ) = PI for all oi ∈ Oi . For period t ≥ 2, consider an information set ot = (aτ , qτ+1)t−1 , and let ot−1 be the information set in period t − 1 where all past i Gi τ=1 i t t−1 τ τ+1 t−2 actions of agent k ∈ Gi coincide with o . That is, o = (a , q ) . Then, the phase i i Gi τ=1 t t P (oi) of oi is determined as follows: for some p ∈ [0, 1],

t−1 t−1 t (P1) If P (oi ) = PI and ak = C for all k ∈ Gi, then P (oi) = PI . t−1 t−1 t t If P (oi ) = PI , ak = D for some k ∈ Gi, and q > p, then P (oi) = PI . t−1 t−1 t t If P (oi ) = PI , ak = D for some k ∈ Gi, and q ≤ p, then P (oi) = PII .

t−1 t t (P2) If P (oi ) = PII and q > p, then P (oi) = PI . t−1 t t If P (oi ) = PII and q ≤ p, then P (oi) = PII .

t t t If agent i is at information set oi with P (oi) = PI (resp. P (oi) = PII ), then we say that agent i is in phase I (resp. in phase II ). Note that if agent i is in phase II in period t then he will stay in phase II until period s > t when qs > p is realized. Furthermore, in period s when qs > p is realized, all agents are in phase I. We will determine p later so that the p t strategy σ we will deﬁne is a sequential equilibrium. Notice that if P (oi) = PII for some t i ∈ N, then there is an agent k ∈ Gi such that P (ok) = PII . Indeed, p determines the severity of punishment for deviation. p The strategy σi of agent i is the strategy that agent i plays C if he is in phase I, and p t t p t t D if he is in phase II. Formally, σi (oi) = C if P (oi) = PI , and σi (oi) = D if P (oi) = PII . The basic idea of the strategy σp is that each agent follows the trigger strategy with small possibility of forgiveness. If an agent observes defection in his past history, then he plays t t s s+1 t−1 D until the period t when q > p is realized. Consider a history h = (aN , q )s=1 where qt > p. Since the realization of qt is publicly known to all agents, under σp, all agents are

40 in phase I in period t and play C thereafter. Note that if p = 1, then agent i will never play C after a history that he observes D in the past. Thus, the strategy σp with p = 1 is the trigger strategy where there is no possibility of forgiveness for a defection. In Proposition 1, we show that, if agents are suﬃciently patient, σp can be a sequential equilibrium with some p ∈ (0, 1).

Proposition 2.1. In the repeated prisoner’s dilemma game with local interaction and public randomization, there is δ ∈ (0, 1) such that, for all δ ∈ (δ, 1), there is a sequential equilibrium in which all agents play cooperation along the equilibrium path and cooperation is eventually recovered from any history.11

The proof of Proposition 2.1 involves tedious calculations, so we leave it in the Appendix. Indeed, we show that the strategy σp is a sequential equilibrium if p is properly determined. Here, we provide the sketch of proof.

Sketch of Proof. Let g δ = , (2.2) 1 + g and δ ∈ (δ, 1). Consider the strategy σp with p ∈ (0, 1) satisfying

g g l ≤ p ≤ + , (2.3) δ(1 + g) δ(1 + g) δ(|GM | − 1)(1 + g) where M is the agent such that |GM | ≥ |Gi| for all i ∈ N. Note that δ ∈ (δ, 1) and (2.2) imply that there exists p ∈ (0, 1) satisfying (2.3). Let µ be a belief system which is consistent with σp.12 In the proof, we show that σp with p ∈ (0, 1) satisfying (2.3) is a sequential equilibrium. For the strategy σp to be a sequential equilibrium, the severity of the punishment, which is determined by p, should be large enough to prevent an agent in phase I from playing D and small enough to prevent an agent in phase II from playing C. The intuitive reason for this is easy to understand. An agent in phase I can get the short-term benefit by playing D and will get some loss in continuation expected payoff from punishment by his neighbors. If the punishment is not severe enough, then the short-term gain from playing D may be greater than the loss in continuation payoff. Thus, σp cannot be a sequential equilibrium. Consider an agent in phase II who observed defection by one of his neighbors in his past history. If he starts to punish his neighbor who played defection, he gets the short-term gain and will sustain loss in the continuation expected payoff from other neighbors because

11 S∞ If P( τ=1{ω happens in period τ}) = 1, then we say that ω happens eventually. 12For ε > 0, let µε be a belief system which is obtained by Bayesian updating of a fully mixed behavioral ε t t p t strategy (βi )i∈N such that, in each information set oi, βi(·; oi) assigns 1 − ε to σ (oi) and ε to the other action. Then, the limit of µε as ε → 0 is the belief system which is consistent with σp. Thus, there exists a belief system which is consistent with σp.

41 they will punish him. Thus, if the punishment is too severe, an agent in phase II may not want to follow the strategy σp. The condition (2.3) provides the range of the severity of punishment so that σp is a sequential equilibrium. We first consider an agent i in Phase I. To check the sequential rationality, we compare i i p p ot p ot t the continuation payoff for σi and that for the strategy σi |D such that σi |D(oi) = D and p p it agrees with σi at all other information sets, given the other agents’ strategy σ−i. Since the belief system µ is consistent with σp, an agent in phase I believes that all agents are in phase I. Then, since the future actions of all agents depend on the realization of public randomization, we can calculate and compare his continuation expected payoffs in which randomness is caused only by public randomization. The first inequality in (2.3) guarantees i p p ot that σi gives the higher continuation expected payoff than σi |D. Indeed, for an information t t t t set oi with P (oi) = PI , h ∈ supp(µ(·; oi)) implies that   X  t p t t p t  (1 − δ) w(αi(σ ; h ), αj(σ ; h )) j∈G  i   X∞ X  τ−t  τ p t τ p t  + (1 − δ) δ EQt+1,...,Qτ w(αi (σ ; h ), αj (σ ; h )) (2.4) τ=t+1 j∈Gi

= |Gi|.

t t t t For an agent at oi with P (oi) = PII , we consider an arbitrary history h ∈ supp(µ(·; oi)). t t Since he is in phase II, at least one of his neighbors plays D in period t. Given history h ∈ oi, i i p p p ot p p ot we consider the two strategies, (σi , σ−i) and (σi |C , σ−i) where σi |C is the strategy such that i p ot t p t σi |C (oi) = C and it agrees with σi at all other information sets. Given that h is reached, since the only source of randomness in continuation expected payoff for each strategy is public randomization, we can calculate and compare the continuation expected payoffs. p The second inequality in (2.3) guarantees that σi gives the higher continuation expected i p ot t t payoff to agent i than σi |C . Since h is an arbitrary history in the support of µ(·; oi), taking t an expectation of continuation payoffs on µ(·; oi) does not change the strategy which gives t t the higher continuation expected payoff. Indeed, for an information set oi with P (oi) = PII , t t h ∈ supp(µ(·; oi)) implies that   X  t p t t p t  (1 − δ) w(αi(σ ; h ), αj(σ ; h )) j∈G  i   X∞ X  τ−t  τ p t τ p t  + (1 − δ) δ EQt+1,...,Qτ w(αi (σ ; h ), αj (σ ; h )) (2.5) τ=t+1 j∈Gi

42 µ ¶ δ(1 − p) = (1 − δ)|Ct(σp; ht)|(1 + g) + |G | , i i 1 − δp

t p t p where Ci (σ ; h ) is the set of agent i’s neighbors who play C in period t under σ given that ht is reached. Trivially, all agents play cooperation under the strategy σp, and since 0 < p < 1, cooperation is eventually recovered from any history.

Consider the strategy σp in Proposition 1 and a history ht where all agents are in phase p I. If an agent deviates from σi , then all of his neighbors start to punish him by playing D in the next period and continue punishing him until period s > t when qs > p is realized. If qt+1 > p is realized, then there is no punishment in period t + 1. If qt+1 ≤ p and qt+2 > p are realized, then the length of punishment is one period. Since the average length of punishment is X∞ p pτ−1(1 − p)(τ − 1) = , (2.6) 1 − p τ=1 the length of punishment under σp with p increases as p increases. This is the reason why we said that p determines the severity of the punishment. A desirable property for equilibrium is global stability which is suggested by Kandori (1992b). An equilibrium is globally stable if from any finite history ht, the continuation expected payoff of each agent eventually return to the payoffs the equilibrium sustains. In Proposition 2.1, we show that, under σp, cooperation is eventually recovered after any finite history. Therefore, the continuation expected payoffs eventually return to the equilibrium payoffs, which means that σp is globally stable. In Proposition 2.1, we show that, for each δ ∈ (δ, 1), the strategy σp is a sequential equilibrium when p satisfies the condition (2.3). Since the range of p differs for different δ, the sequential equilibrium σp may depend on δ. Proposition 2.2 states that we can find a strategy σ∗ which is a sequential equilibrium supporting cooperation for all δ which is sufficiently high.

Proposition 2.2. In the repeated prisoner’s dilemma game with local interaction and public randomization, there exist a strategy σ∗ and δ∗ ∈ (δ, 1) such that, for all δ ∈ [δ∗, 1), σ∗ is a sequential equilibrium in which all agents play cooperation along the equilibrium path and cooperation is eventually recovered from any history.

Proof. Suppose that l < |GM | − 1. Let

(|G | − 1)g δ∗ = M (2.7) (|GM | − 1)g + l

43 and σ∗ be a strategy having the same structure as σp with

g p∗ = . (2.8) δ∗(1 + g)

∗ Note that (2.2), (2.7), and l < |GM | − 1 imply δ < δ . Furthermore, we have

∗ g g l p = ∗ = + < 1. δ (1 + g) 1 + g (|GM | − 1)(1 + g) and, for all δ ∈ [δ∗, 1), g g ≤ = p∗ δ(1 + g) δ∗(1 + g) g l g l = + < + , 1 + g (|GM | − 1)(1 + g) δ(1 + g) δ(|GM | − 1)(1 + g) so condition (2.3) holds. ∗ ∗ If l ≥ |GM | − 1, let δ be arbitrary in (δ, 1) and let σ be a strategy having the same structure as σp with p∗ in (2.8). Then, for all δ ∈ [δ∗, 1), g g g ≤ = p∗ < = 1 δ(1 + g) δ∗(1 + g) δ(1 + g) g l g l ≤ + ≤ + , (1 + g) (|GM | − 1)(1 + g) δ(1 + g) δ(|GM | − 1)(1 + g) so condition (2.3) holds. Then, the result follows from Proposition 2.1

Note that, δ∗ in Proposition 2.2 is strictly greater than δ in Proposition 2.1. This means that more patience is required for the existence of the strategy which is a sequential equilibrium for all sufficiently high δ than for the existence of a sequential equilibrium. The other desirable property for an equilibrium is that efficiency is obtained at any history in the limit as δ converges to one. For this property, consider the trigger strategy that each agent plays C if and only if he does not observe D in the past. The public randomization does not play any role in this strategy. Suppose that l ≥ |GM | − 1. Then, it is not difficult to show that, for any δ ∈ (δ, 1) where δ is in (2.2), the trigger strategy is a sequential equilibrium. However, consider a history h2 where all agents play D in period 1. Trivially, for any δ ∈ (δ, 1), all agents play D after period 2, so the continuation expected payoff of each agent from his information set is zero, which is not efficient. Proposition 2.3 states that, under the sequential equilibrium we constructed, efficiency is obtained in the limit from any history as δ converges to one.

Proposition 2.3. For each δ ∈ [δ∗, 1) where δ∗ is in (2.7), consider the sequential equilibrium σ∗ deﬁned in Proposition 2.2. For any history hˆt, the continuation expected payoﬀ of

44 each agent i from his information set converges to |Gi| as δ converges to one. That is, for any history hˆt, ∗ t ˆt lim CEUi(σ ; µ; oi(h )) = |Gi|. δ→1 Proof. From the proof in Proposition 2.2, we know that p∗ satisfies (2.3). Suppose that a ˆt t ˆt ∗ t t ˆt history h satisfies P (oi(h )) = PI . Since (2.4) with σ holds for all h ∈ supp(µ(·; oi(h ))) ∗ ∗ t ˆt ˆt and for all δ ∈ [δ , 1), we have limδ→1 CEUi(σ ; µ; oi(h )) = |Gi|. Suppose that a history h t ˆt t t ˆt satisfies P (oi(h )) = PII . Since (2.5) holds for all h ∈ supp(µ(·; oi(h ))) and

· µ ∗ ¶¸ t ∗ t δ(1 − p ) lim (1 − δ)|Ci (σ ; h )|(1 + g) + |Gi| = |Gi|, δ→1 1 − δp∗ we have

∗ t ˆt lim CEUi(σ ; µ; oi(h )) δ→1 · µ ¶¸ X ∗ t t ˆt t ∗ t δ(1 − p ) = lim µ(h ; oi(h )) (1 − δ)|Ci (σ ; h )|(1 + g) + |Gi| δ→1 1 − δp∗ t t ht∈o (hˆ ) i · µ ¶¸ X ∗ t t ˆt t ∗ t δ(1 − p ) = µ(h ; oi(h )) lim (1 − δ)|Ci (σ ; h )|(1 + g) + |Gi| δ→1 1 − δp∗ t ˆt ht∈oi(h )

= |Gi|.

2.4 Stability and Robustness of σp

In the previous section, we construct a sequential equilibrium with small possibility of forgiveness. In this equilibrium, cooperation is played along the equilibrium path and defection is never played in the repeated game. Note that the trigger strategy that observing a defection causes a permanent punishment can be a equilibrium which supports cooperation when the loss from being cheated is large enough. When the loss is small, we can also construct a sequential equilibrium supporting cooperation by modifying the trigger strategy, following Ellison (1994). Since defection is never played along the equilibrium path both in the trigger strategy and in the strategy σp we construct with p < 1, it seems unclear why the stability is a desirable property of equilibrium. The purpose of this section is to discuss that the strategy with small possibility of forgiveness can give higher payoﬀs to players than the trigger strategy if there is a small possibility of mistakes. To make the analysis tractable, we assume that the agents are located on a complete network, so that the monitoring is perfect and |Gi| = n − 1 for all

45 i ∈ N. Because the monitoring is perfect, the trigger strategy is a sequential equilibrium even when the loss from being cheated is small enough. The reason is that, if an agent observes a deviation, he then believes that every other agent observes the deviation and will play D in the future periods, so playing D in the future periods is the best response for him even when the loss is small enough. With the perfect monitoring, given that the other agents employ the trigger strategy, an agent cannot enjoy the benefit of cooperation by blocking the spread of defection when he observes a defection. Note that the strategy σp constructed in Section 2.3 is the trigger strategy for p = 1. 1 1 So, we denote σ = (σi )i∈N as the trigger strategy. Since the monitoring is perfect, all agents are in the same phase in each period. For the strategy σp, we introduce the noise by assuming that each agent has to play each action in each information set with the probability p p at least ε > 0. Let σ (ε) = (σi (ε))i∈N be the strategy profile such that each agent i chooses p the action suggested by σi with probability 1 − ε and the other action with probability ε in each information set. That is, if an agent i is in Phase I (resp. Phase II), then he chooses C (resp. D) with probability 1 − ε and D (resp. C) with probability ε. For convenience, we let σp(0) = σp. Since all agents are in the same phase in each period, we can calculate the probability that each action profile is played for each phase. For example, if agents are in Phase I, then the probability that all agents choose C is (1 − ε)n and the probability that all agents choose D is εn. The probability that only m agents play C in Phase I is m n−m nCm(1 − ε) ε .

Given a strategy profile σ, we denote VI (σ) as the continuation payoff for agent i from a history of Phase I and VII (σ) as the continuation payoff for agent i from a history of

Phase II. Since all agents are identical, VI (σ) and VII (σ) do not depend on the label of agents. Since all agents are in Phase I in period 1 and cooperation is played along σp-path, p p the payoff of agent i under σ is VI (σ ) = n − 1. Similarly, the payoff of agent i under the 1 1 trigger strategy σ is VI (σ ) = n − 1. However, if we introduce the noise for each strategy, p 1 then we are able to show that VI (σ (ε)) > VI (σ (ε)) for sufficiently small ε > 0. This means that σp(ε) gives higher payoffs than σ1(ε) if there is a small possibility of mistakes. This gives the partial answer for stability as a desirable property of equilibrium. p 0 Given the strategy σ (ε), let v(ε) = (vI (ε), vII (ε)) be the expected payoff vector for agent i in each period, where vI (ε) is the payoff in a stage game in Phase I and vII (ε) is the payoff in a stage game in Phase II. Note that

nX−1 n−1−m m vI (ε) = (1 − ε) n−1Cm (1 − ε) ε (n − 1 − m − ml) m=0 nX−1 n−1−m m +ε n−1Cm (1 − ε) ε (n − 1 − m) (1 + g) (2.9) m=0

46 = (1 − ε) (1 − (1 + l) ε)(n − 1) + ε (1 − ε) (1 + g)(n − 1)

nX−1 n−1−m m vII (ε) = (1 − ε) n−1Cm (1 − ε) ε m (1 + g) m=0 nX−1 n−1−m m +ε n−1Cm (1 − ε) ε (m − (n − 1 − m) l) (2.10) m=0 = (1 − ε) ε (1 + g)(n − 1) − ε (l − (1 + l) ε)(n − 1) and so

vI (ε) − vII (ε) = (1 − 2ε)(n − 1) > 0 for all ε ∈ (0, 1/2). (2.11)

p p To derive VI (σ (ε)) and VII (σ (ε)), notice that if agents are in Phase I in some period then they are in Phase I with probability (1 − ε)n + (1 − p) (1 − (1 − ε)n) and in Phase II with probability p (1 − (1 − ε)n) in the next period under σp(ε). If agents are in Phase I in period t, then they are in Phase I with probability 1 − p and in Phase II with probability p in period t + 1. Thus, we have the following relation:

p VI (σ (ε)) = (1 − δ) vI (ε) n p n p +δ [(p (1 − ε) + (1 − p)) VI (σ (ε)) + p (1 − (1 − ε) ) VII (σ (ε))]

p p p VII (σ (ε)) = (1 − δ) vII (ε) + δ [(1 − p) VI (σ (ε)) + pVII (σ (ε))] or equivalently, Ã ! Ã ! p VI (σ (ε)) vI (ε) (I − δA (p, ε)) p = (1 − δ) (2.12) VII (σ (ε)) vII (ε) where Ã ! p (1 − ε)n + (1 − p) p − p (1 − ε)n A (p, ε) = . 1 − p p

Solving the system in (2.12), we get

1 V (σp(ε)) = ((1 − pδ) v (ε) + pδ(1 − (1 − ε)n)v (ε)) (2.13) I 1 − pδ (1 − ε)n I II 1 V (σp(ε)) = (δ (1 − p) v (ε) + ((1 − δ) + pδ (1 − (1 − ε)n)) v (ε)) (2.14) II 1 − pδ (1 − ε)n I II

Plugging (2.9) and (2.10) in (2.13) and (2.14) and ε = 0, we have

δ(1 − p) V (σp(0)) = n − 1 and V (σp(0)) = (n − 1) . (2.15) I II 1 − pδ

47 p p Note that VI (σ (0)) does not depend on p, which coincide with that the strategy σ with p < 1 and the trigger strategy give the same payoff. Since all agents are in Phase I in period 1, given the strategy σp(ε), the payoff of each p agents in the repeated prisoner’s dilemma game is VI (σ (ε)). Taking a differentiation in (2.13) with respect to p,(2.11) implies that, for all ε ∈ (0, 1/2),

p n dVI (σ (ε)) δ (1 − (1 − ε) ) = − (vI (ε) − vII (ε)) < 0. (2.16) dp (1 − pδ (1 − ε)n)2

p 1 Therefore, we have VI (σ (ε)) > VI (σ (ε)) for all ε ∈ (0, 1/2). These observations give us the following result.

Proposition 2.4. Consider the repeated prisoner’s game with local interaction and public randomization. Let G be a complete network. For ε > 0, deﬁne the strategies σp(ε) for p ∈ (0, 1) and σ1(ε) as before. Then there exists ε¯ > 0 such that for all ε ∈ (0, ε¯), σp(ε) results in a more eﬃcient outcome than σ1(ε), even though both strategies give the same outcome in the limit as ε goes to 0. Here, ε¯ does not depend on δ.

Proof. The result follows the argument before Proposition 2.4.

Another desirable property for equilibrium is robustness to mistakes. A strategy σp = p p (σi )i∈N is robust to mistakes if, for each i, the action suggested by σi in each information set p is the optimal choice, given that he follows σi (ε) in all other information sets and the other p p players follow σ−i(ε) for sufficiently small ε > 0. In other words, σ is robust to mistakes, t p t if for each i and for each oi, σi (oi) is the best response for agent i in the information set t oi in the agent normal form game of the original game, given that each agent in all other information sets (including himself) follows σp(ε) for sufficiently small ε > 0.13 Let δ = g/(1 + g) and δ ∈ (δ, 1). Furthermore, let σp be a strategy profile with p satisfying g < p < 1 (2.17) δ(1 + g) The existence of p satisfying (2.17) is guaranteed by δ < δ. First, consider an agent i in t t p p oi with P (oi) = PI . Suppose that the other agents play σ−i(ε) and he follows σi (ε) in all p t other information sets. To see that σi (oi) is the best response to agent i in information t t set oi, it is enough to look at the continuation payoffs from oi. Given his actions in the p other information sets and the others’ strategies, his continuation payoffs VI (a; σ (ε)) from

13This concept is closely related with the trembling hand perfect equilibrium in ﬁnite extensive form games. It is well known that, in a ﬁnite extensive form game with perfect recall, any trembling hand perfect equilibrium is a sequential equilibrium with some belief system. However, the converse is not true in general.

48 t t oi, when he plays a ∈ {C,D} in oi, are

p VI (C; σ (ε)) nX−1 n−1−m m = (1 − δ) n−1Cm (1 − ε) ε (n − 1 − m − ml) (2.18) ³³ m=0 ´ ³ ´ ´ n−1 p n−1 p +δ p (1 − ε) + (1 − p) VI (σ (ε)) + p − p (1 − ε) VII (σ (ε)

p VI (D; σ (ε)) nX−1 n−1−m m = (1 − δ) n−1Cm (1 − ε) ε (n − 1 − m) (1 + g) (2.19) m=0 p p +δ ((1 − p) VI (σ (ε)) + pVII (σ (ε)))

Plugging (2.15) in (2.18) and (2.19), (2.17) guarantees that

V (C; σp(0)) = n − 1 I µ ¶ δ2p(1 − p) > (1 − δ) (1 + g) + δ (1 − p) + (n − 1) (1 − pδ) p = VI (D; σ (0)),

p p for all δ ∈ (δ, 1). Then, the continuity of VI (C; σ (ε)) and VI (D; σ (ε)) in ε implies that p p there existsε ˜ > 0 such that for all ε ∈ (0, ε˜), VI (C; σ (ε)) > VI (D; σ (ε)). t t Next, consider an an agent i in oi with P (oi) = PII . Given his actions in the other p t information sets and the others’ strategies, his continuation payoﬀs VII (a; σ (ε)) from oi for each action a ∈ {C,D} are

nX−1 p n−1−m m VII (C; σ (ε)) = (1 − δ) n−1Cm (1 − ε) ε (m − (n − 1 − m) l) m=0 p p +δ ((1 − p) VI (σ (ε)) + pVII (σ (ε)) p p = (1 − δ)(−l + (1 + l) ε)(n − 1) + δ ((1 − p) VI (σ (ε)) + pVII (σ (ε))

nX−1 p n−1−m m VII (D; σ (ε)) = (1 − δ) n−1Cm (1 − ε) ε m (1 + g) m=0 p p +δ ((1 − p) VI (σ (ε)) + pVII (σ (ε)) p p = (1 − δ) (1 + g)(n − 1) ε + δ ((1 − p) VI (σ (ε)) + pVII (σ (ε))

Since p p VII (C; σ (ε)) − VII (D; σ (ε)) = − (1 − δ)(n − 1) (gε + (1 − ε) l) > 0

49 p p p p for all ε > 0, we have VI (C; σ (ε)) > VI (D; σ (ε)) and VII (C; σ (ε) < VII (D; σ (ε)) for all ε ∈ (0, ε˜). This means, the strategy σp with p satisfying (2.17) is robust to mistakes. Proposition 2.5 summarizes the result.

Proposition 2.5. Consider the repeated prisoner’s game with local interaction and public randomization. Let G be a complete network. Then, there exists δ ∈ (0, 1) such that for all δ ∈ (δ, 1), the strategy σp with p satisfying (2.17) a sequential equilibrium which is robust to mistakes.

Proof. See the argument preceding Proposition 2.5.

Let δ∗ and the strategy σ∗ be be deﬁned as in Proposition 2.2. Since δ < δ∗ and g/(δ(1 + g)) < p∗ < 1 for all δ ∈ (δ∗, 1), σ∗ is robust to mistakes. Thus, Proposition 2.2 and 2.5 implies Corollary 2.1.

Corollary 2.1. Consider the repeated prisoner’s game with local interaction and public randomization. Let G be a complete network. Then, there exist δ∗ ∈ (0, 1) and a strategy σ∗ such that, for all δ ∈ (δ∗, 1), σ∗ is a sequential equilibrium which is robust to mistakes and in which all agents play cooperation along the equilibrium path and cooperation is eventually recovered from any history.

In this section, we restrict our attention to the complete network so that monitoring is perfect and all agents are identical. Since monitoring is perfect under the complete network, all agents are in the same phase. Thus, all finite histories can be classified into two cases: one that all agents are in Phase I and the other that all agents are in Phase II. Since the strategy σp (or σp(ε)) generates the same outcome from the histories in the same case, we can derive the payoff in each period by calculating the stage game payoff for each case and the probability of each case being reached in that period. Because, in each period t, all the histories belong to two cases and all histories in each case generate the same outcome from period t, determining the distribution on each case is enough to derive the continuation payoff. This is represented by a 2 × 2 transition matrix A(p, ε). If network is not complete, then the dimension of the transition matrix depends of the number of agents and networks. For example, suppose that three agents, 1, 2, and 3, are on a line-shaped network G = {12, 23} and they play σp(ε). Then, in each information set, histories belong to four cases: PI,I,I , PII,II,I , PI,II,II , and PII,II,II . Here, PI,I,I is the set of histories in which all agents are in Phase I, PII,II,I is the set of histories in which agents 1 and 2 are in Phase

II and agent 3 is in Phase I, and so on. Note that the cases such as PII,I,II and PI,II,I are not possible, because an agent cannot be in Phase II without at least one neighbor being in Phase II. Thus, with three agents on line-shaped network, we may need a 4 × 4 transition matrix A(p, ε) in calculating the continuation payoﬀs for each case. The dimension of the

50 transition matrix A(p, ε) sensitively depends on the number of agents and the network, which make it difficult to derive continuation payoffs. Furthermore, in the example with three agents on the line-shaped network, agent 1’s stage game payoff is not equal to agent 2’s stage game payoff, Thus, v(ε) corresponding to (2.9) and (2.10) depends on the label of agents, so we cannot not consider a representative agent to obtain the results. This may cause another difficulty in analyzing the model without the assumption of complete network.

2.5 Concluding Remarks

In this paper, we assume that there is a public randomization which is observed by all agents and construct a sequential equilibrium which supports cooperation and in which cooperation is eventually recovered from any history. Here, the role of public randomization is to enable agents to recover cooperation in the same period and the possibility that cooperation is recovered determines the severity of punishment for deviation. Notice that in the equilibrium we construct, we need that the distribution of public randomization is known to all agents and that there is an agreement of the whole society on p or on what the event is. The trigger strategy causes a complete breakdown of cooperation after a single deviation and cooperation is never recovered, even though it can be a sequential equilibrium when the loss from being cheated is large enough. Furthermore, since the local observability may cause a discrepancy on the beliefs about when an agent starts a punishment, the strategy such that each agent punishes his neighbors in finite periods when he observes defection cannot be a sequential equilibrium in which cooperation is recovered. To resolve the discrepancy in the beliefs, Cho (2007a) allows the agent to communicate with his neighbors and construct a sequential equilibrium which supports cooperation and in which cooperation is recovered in finite periods. Although the equilibrium in Cho (2007a) does not require the common knowledge of distribution of public randomization, it requires that agents are on a minimally connected network and the gain from cheating and the loss from being cheated are sufficiently small. Note that these assumptions are not required in this paper. In the paper, we assume that all agents are identical except in their number of neighbors. This means that all agents have a common discount factor and the prisoner’s dilemma games between any two agents are identical. For the assumption of common discount factor, notice ∗ that in Proposition 2.2, verifying the sequential rationality of σi for each i does not depend on the others’ discounter factors. Thus, we can replace the assumption that agents have a ∗ common discount factor with the weaker assumption that δi ∈ [δ , 1) for all i where δi is agent i’s discount factor. It is worthwhile to mention the difference between the local interaction model and the

51 random matching model explored by Kandori (1992b) and Ellison (1994). In the random matching model, since agents are not labeled and their actions are not recorded, there is no information processing. However, in the local interaction model, the neighbors of an agent do not change over the periods and each agent observes and remembers what his neighbors play. This means that there is a local information processing. Furthermore, Kandori (1992b) and Ellison (1994) consider a uniform random matching model so that there is no heterogeneity in agents. This makes it possible to check the sequential rationality of the strategy in interest by considering a representative agent. But, in the local interaction model, agents may be heterogeneous because the numbers of their neighbors can differ. Fortunately, considering an agent with the largest number of neighbors is enough to check the sequential rationality of the strategy we construct. Finally, we may want to consider a two person symmetric game which is more general than the prisoner’s dilemma game. Let A be a finite action space for each agent and consider a two person symmetric game G : A2 → R2 instead of the prisoner’s dilemma game. Let 2 g(ai, aj) denote agent i’s payoff when agent i plays ai and j plays aj. Let (m, m) ∈ A be a strict Nash equilibrium for G with g(m, m) = 0 and let (¯a, a¯) ∈ A2 satisfy g(¯a, a¯) > 0. Furthermore, suppose that g(m, a¯) ≥ g(a, a¯) for all a ∈ A.14 Then, one can show that, if agents are sufficiently patient, there is a sequential equilibrium with public randomization which supports (¯a, a¯) and in which (¯a, a¯) is eventually recovered from any finite history. The arguments are similar to those with prisoner’s dilemma game but contain tedious calculations. Moreover, it is interesting to investigate the relationship between efficiency and equilibrium under the local interaction model with more general two person games. We leave this for future research.

Appendix: Proof of Proposition 2.1

Proof of Proposition 2.1. Deﬁne δ as in (2.2) and let δ ∈ (δ, 1). Consider the strategy σp with p ∈ (0, 1) satisfying (2.3). Since δ ∈ (δ, 1), there exists p ∈ (0, 1) satisfying (2.3). Let µ be a belief system which is consistent with σp. To prove Proposition 2.1, we will show that, under the strategy σp with p satisfying (2.3) and the belief system µ, playing D in phase I and playing C in phase II are not beneﬁcial. For an t t t p oi p oi t information set oi, we denote σi |a for a ∈ {C,D} as the strategy of agent i such that σi |a (oi) = a p and it agrees with σi at any other information set. For an information set ot ∈ (as , qs+1)t−1 with t ≥ 2, let t¯ = max{τ : qτ > p, τ ≤ t − 1} if i Gi s=1 {τ : qτ > p, τ ≤ t − 1} 6= ∅ and let t¯ = 1 if {τ : qτ > p, τ ≤ t − 1} = ∅. Then, in period t,¯ all agents are in phase I.

14If (m, m) is a dominant strategy equilibrium, then it satisﬁes this condition.

52 t s s+1 t−1 t Phase I. Consider an information set o ∈ (a , q ) such that P (o ) = PI . If t = 1, then all i Gi s=1 i t t s agents are in phase I. If t ≥ 2, then oi satisﬁes that q > p or that ak = C for all k ∈ Gi and all s ≥ t¯. Furthermore, since µ is consistent with σp, agent i believes that no one plays D after period ¯ t s s+1 t−1 t t s t. Thus, for all h = (aN , q )s=1 ∈ supp(µ(·; oi)), either q > p or ak = C for all k ∈ N and for all ¯ t t s ≥ t is satisﬁed. That means, all agents are in phase I in period t in any history h ∈ supp(µ(·; oi)). p t t τ p t If agents play σ after a history h ∈ supp(µ(·; oi)), then αk(σ ; h ) = C for all k ∈ N and for all τ ≥ t whatever qτ is realized. Therefore,   X X∞ X p t  τ−t  CEUi(σ ; µ; oi) = (1 − δ) w(C,C) + (1 − δ) δ w(C,C)

j∈Gi τ=t+1 j∈Gi

= |Gi|.

t p oi p t t Next, suppose that agents play (σi |D , σ−i) after history h ∈ supp(µ(·; oi)). Since all agents are t t t p oi p t t p oi p t in phase I in period t, αi(σi |D , σ−i; h ) = D and αk(σi |D , σ−i; h ) = C for all k ∈ Gi. Furthermore, 0 for each τ ≥ t + 1, if qτ ≤ p for all τ 0 with t + 1 ≤ τ 0 ≤ τ which happens with probability t t 0 τ τ p oi p t τ p oi p t τ p , then αi (σi |D , σ−i; h ) = D and αk(σi |D , σ−i; h ) = D for all k ∈ Gi. If q > p for some t 0 0 τ τ p oi p t τ with t + 1 ≤ τ ≤ τ which happens with probability 1 − p , then αi (σi |D , σ−i; h ) = C and t τ p oi p t αk(σi |D , σ−i; h ) = C for all k ∈ Gi. Therefore,

t p oi p t CEUi(σ | , σ ; µ; o ) i D −i i    X X∞ X X = (1 − δ) w(D,C) + (1 − δ)  δτ−t pτ−t w(D,D) + (1 − pτ−t) w(C,C)

j∈Gi τ=t+1 j∈Gi j∈Gi Ã ! X∞ τ−t τ−t = (1 − δ)|Gi|(1 + g) + (1 − δ) δ (1 − p )|Gi| τ=t+1 µ ¶ δ(1 − p) = |G | (1 − δ)(1 + g) + . i 1 − δp

Then, (2.3) implies that

t s s+1 t−1 t Phase II. Consider an information set o ∈ (a , q ) such that P (o ) = PII . By construction i Gi s=1 i t t τ ¯ of P , oi satisﬁes that q < p and ak = D for some k ∈ Gi and for some τ with t ≤ τ ≤ t − 1. Given a t t t t t history h and a strategy σ, let Di (σ; h )(resp. Ci (σ; h )) be the set of agent i’s neighbors who play t τ t t+1 τ D (resp. play C) in period t along the σ-path from h . For each τ ≥ t+1, let Di (σ; h ; Q ,...,Q ) τ t t+1 τ (resp. Ci (σ; h ; Q ,...,Q )) be the set of i’s neighbors who play D (resp. play C) in period τ t 15 τ t t+1 τ along the σ-path from h . Note that, for each τ ≥ t + 1, Di (σ; h ; Q ,...,Q ) is realized t+1 τ τ t t+1 τ depending on the realization of (Q ,...,Q ). That means, Di (σ; h ; Q ,...,Q ) is random

15Formally, t t t t Di (σ; h ) = {j ∈ Gi : αj (σ; h ) = D},

53 given that ht is reached. t t t t t Let h be an arbitrary history such that h ∈ supp(µ(·; oi)). Since h ∈ oi, there is an agent j ∈ Gi who played D or observed i playing D in some τ with t¯ ≤ τ ≤ t − 1. Then, since j plays p t p t t t D in period t under σ , Di (σ ; ht) is not empty. Since αi(oi(h )) = D, each agent j ∈ Gi will play D under σp until the period τ in which qτ > p is realized and plays C thereafter along the σp-path t p τ p t t+1 τ τ 0 from h . So, by the construction of σ , Di (σ ; h ; Q ,...,Q ) = Gi conditioning on that q ≤ p is realized for all τ 0 satisfying t + 1 ≤ τ 0 ≤ τ, and all agents play C in period τ conditioning on that 0 qτ > p is realized for some τ 0 satisfying t + 1 ≤ τ 0 ≤ τ. t t t t Thus, for each h ∈ oi satisfying h ∈ supp(µ(·; oi)), we have   X  t p t t p t  (1 − δ) w(αi(σ ; h ), αj(σ ; h ))

j∈Gi    X∞ X  τ−t  τ p t τ p t  + (1 − δ) δ EQt+1,...,Qτ w(αi (σ ; h ), αj (σ ; h ))

τ=t+1 j∈Gi ¡ t p t p ¢ = (1 − δ) |Di (σ ; ht)|w(D,D) + |Ci (σ ; ht)|w(D,C)] (2.21)    X∞ X X + (1 − δ)  δτ−t pτ−t w(D,D) + (1 − pτ−t) w(C,C)

t t p oi p t p oi p t Next, consider the strategy (σi |C , σ−i). Since αj(σi |C , σ−i; h ) for all j 6= i depends only on t t t t p oi p t t p t t p oi p t t p t h , we have Di (σi |C , σ−i; h ) = Di (σ ; h ) and Ci (σi |C , σ−i; h ) = Ci (σ ; h ). By the construction t 0 p oi p τ 0 of (σi |C , σ−i), for each τ ≥ t + 1, conditioning on that q ≤ p is realized for all τ with t + 1 ≤ t t 0 τ p oi p t+1 τ τ p oi p τ ≤ τ, Di (σi |C , σ−i; ht; Q ,...,Q ) is nonrandom. Thus, we can denote Di (σi |C , σ−i; ht) as t 0 τ p oi p t t+1 τ τ 0 0 Di (σi |C , σ−i; h ; Q ,...,Q ) conditioning on that q ≤ p is realized for all τ with t+1 ≤ τ ≤ τ. t p oi p t Along the (σi |C , σ−i)-path from h , if agent k ∈ N chooses D in period τ ≥ t, then he keeps 0 t τ τ p oi p playing D until q > p is realized and play C thereafter. Thus, for each τ ≥ t, Di (σi |C , σ−i; ht) ⊆ t t t τ+1 p oi p τ p oi p τ+1 p oi p τ+1 Di (σi |C , σ−i; ht) and Ci (σi |C , σ−i; ht) ⊇ Ci (σi |C , σ−i; ht) given that q ≤ p is realized. If 0 qτ ≤ p is realized for some τ 0 with t + 1 ≤ τ 0 ≤ τ, then all agents play C in period τ. Furthermore, t since agent i observes that an agent k ∈ Gi played D in the history h , i will play D in period t + 1 0 0 and continue it until qτ > p is realized. After qτ > p is realized, he will play C forever. for each τ ≥ t + 1, τ t t+1 τ τ t Di (σ; h ; Q ,...,Q ) = {j ∈ Gi : αj (σ; h ) = D}.

54 t t t Thus, for each h satisfying h ∈ supp(µ(·; oi)), we have  

³ t ´ t+1 p oi p + (1 − δ)δ p|Ci (σi |C , σ−i; ht)|(1 + g) + (1 − p)|Gi| Ã ! X∞ τ−t £ τ−t ¤ + (1 − δ) δ (1 − p )|Gi| τ=t+2 ¡ t p t p ¢ ≤ (1 − δ) |Ci (σ ; ht)| − |Di (σ ; ht)|l Ã ! X∞ ¡ t p ¢ τ−t £ τ−t ¤ + (1 − δ)δ p|Ci (σ ; ht)|(1 + g) + (1 − p)|Gi| + (1 − δ) δ (1 − p )|Gi| τ=t+2 µ ¶ δ(1 − p) = (1 − δ)(|Ct(σp; h )| − |Dt(σp; h )|l + δp(|Ct(σp; h )|)(1 + g)) + |G | . i t i t i t i 1 − δp

Then, since (2.3) implies

t p g |Di (σ ; ht)|l p ≤ + t p , δ(1 + g) δ|Ci (σ ; ht)|(1 + g) we get   X  t p t t p t  (1 − δ) w(αi(σ ; h ), αj(σ ; h ))

j∈Gi    X∞ X  τ−t  τ p t τ p t  + (1 − δ) δ EQt+1,...,Qτ w(αi (σ ; h ), αj (σ ; h ))

τ=t+1 j∈Gi µ ¶ δ(1 − p) = (1 − δ)|Ct(σp; h )|(1 + g) + |G | i t i 1 − δp µ ¶ δ(1 − p) ≥ (1 − δ)(|Ct(σp; h )| − |Dt(σp; h )|l + δp(|Ct(σp; h )|)(1 + g)) + |G | (2.22) i t i t i t i 1 − δp

55  

X ot ot  t p i p t t p i p t  ≥ (1 − δ) w(αi(σi |C , σ−i; h ), αj(σi |C , σ−i; h )) j∈Gi    ∞ X X ot ot  τ−t  τ p i p t τ p i p t  + (1 − δ) δ EQt+1,...,Qτ w(αi (σi |C , σ−i; h ), αj (σi |C , σ−i; h )) . τ=t+1 j∈Gi

t t Since (2.22) holds for all h ∈ supp(µ(·; oi)), we have

p CEUi(σ ; µ; oit)   X X t t  t p t t p t  = (1 − δ) µ(h ; oi) w(αi(σ ; h ), αj(σ ; h )) t ht∈o j∈Gi i    X X∞ X t t  τ−t  τ p t τ p t  + (1 − δ) µ(h ; oi) δ EQt+1,...,Qτ w(αi (σ ; h ), αj (σ ; h )) (2.23)

ht∈oit τ=t+1 j∈Gi  

X X ot ot t t  t p i p t t p i p t  ≥ (1 − δ) µ(h ; oi) w(αi(σi |C , σ−i; h ), αj(σi |C , σ−i; h )) t ht∈o j∈Gi i    ∞ X X X ot ot t t  τ−t  τ p i p t τ p i p t  + (1 − δ) µ(h ; oi) δ EQt+1,...,Qτ w(αi (σi |C , σ−i; h ), αj (σi |C , σ−i; h )) t τ=t+1 j∈G ht∈oi i t p oi p = CEUi(σi |C , σ−i; µ; oit).

By the one deviation property of sequential equilibrium, (2.20) and (2.23) imply that σp with p satisfying (2.3) is a sequential equilibrium. Under σp, cooperation is played along the equilibrium path and, since 0 < p < 1, cooperation is eventually recovered from history ht.

56 Chapter 3

Endogenous Formation of Networks for Local Public Goods

3.1 Introduction

In this paper, we study endogenous formation of networks in which agents make a decision both on the quantity of local public goods and on the structure of the networks through which they share the benefits of public goods. Since an individual’s benefit from a public good is smaller than its cost, self-interested agents will neither produce public goods nor form links with others if there is no commitment. So, it is clear that for such a society to be viable, the agents concerned must have a binding agreement to jointly maintain links and produce public goods. This paper considers the situation in which agents can make binding agreements both on whom they choose to link with and on the actions each will undertake in the production phase that follows once the network is in place. The underlying model has the following features. Each agent has a technology to produce a public good, which confers benefits onto himself and those who are (directly or indirectly) connected to the agent in question. Connections (or links) are undirected. Each must pay a small cost to maintain a direct link as well as to produce a public good. Utility is not generally directly pairwise transferable; an agent can increase or reduce his production, but this affects everyone connected with him, either directly or indirectly, including himself. Information which agents acquire and share is an example of this kind of public good.1 Throughout the paper, we will speak of information as the public good that is produced and shared. We note that this framework applies generally to local public goods.

1As discussed in Hippel (1988), U.S. steel minimill producers trade their know-how through informal social networks. He also argues that know-how is transferred through indirect connections as well as direct connections. Granovetter (1974) studies the flow of information within social networks in finding a new job and finds that social networks such as family and friends play a significant role in getting job information.

57 The paper analyzes and compares two different approaches to determining the nature of binding agreements. The first relies on coalition formation through sequential bargaining, in common with the growing literatures on extensive-form models of bargaining. The second uses the notion of the core, popular from the cooperative game theory, which we will discuss in the sequel. We can interpret resulting outcomes in these approaches as allocations which can be obtained by fully binding contracts. In the real world, it is not difficult to observe situations where firms or individuals make a fully binding contract through prospective enforceable penalties. Since a violation of the contract can make it invalid immediately, an agent cannot gain by deviation, given others’ actions as stipulated in the contract. If there are a few agents in society or if agents can monitor others’ actions completely, it seems reasonable that the contract will break down immediately after deviations. The first part of this paper is devoted to analyzing sequential bargaining games in which each agent suggests a contract to a coalition of agents in an exogenously determined order. If all agents in the coalition accept this contract, they leave the game with the allocation the contract assigns. If some agent rejects the suggested contract, the first rejecter becomes a new proposer after one period. In this situation, we are interested in sufficiently, but not fully, patient agents. If agents are fully patient, all agents receive the same payoff in equilibrium. On the other hand, if they are not patient at all, the initial proposer is a strong residual claimant. For the solution concept, we restrict our interest to symmetric stationary perfect equilibria, which makes the analysis tractable.2 Coalitional sequential bargaining games are studied in earlier papers such as Chatterjee et al. (1993) and Ray and Vohra (1999). In these studies, they consider transferable utility games, where agents in a coalition can make an agreement on the share of benefit. Chatterjee et al. (1993) assume that there is no externality across coalitions and show that there is a pure strategy stationary perfect equilibrium in a sequential bargaining game satisfying supperadditivity. Ray and Vohra (1999) assume that there is an externality across coalitions, and show that there is a stationary perfect equilibrium in mixed strategies. Ray and Vohra (1999) also argue that, without allowing mixed strategies, there is a sequential bargaining game which does not have a stationary perfect equilibrium. In analyzing sequential bargaining procedures, we consider two different models which are distinguished by the type of permissible proposals. In the first model, each proposer suggests a network and an effort profile for a coalition and each respondent decides whether or not to accept it, depending on his payoff. In this model, we find the result that there is no symmetric stationary perfect equilibrium, which seems similar to that in Ray and Vohra (1999). However, the negative result in this paper comes from the non-connectedness of

2As discussed in Gul (1989), a justiﬁcation for selecting stationary equilibria is computational simplicity. Chatterjee and Sabourian (2000) deﬁne the complexity of strategies for sequential bargaining games, with which a stationary strategy is minimally complex.

58 utility possibility frontier, while that in Ray and Vohra (1999) comes from the the externality across coalitions and the asymmetry of partition function. In the second model, we allow each proposer to suggest a distribution on networks and efforts, and each respondent decides whether to accept it or not, depending on his expected payoff. It will become clear that we distinguish between “mixed strategies” considered in Ray and Vohra (1999) and “mixed proposals” in this paper. The latter, as mentioned, consists of a distribution of networks and effort levels, and can be interpreted as tossing a coin to allocate an indivisible good.3 In the model with mixed proposals, agents can transfer their utilities by changing distribution on allocations, which makes the utility possibility frontier connected. Thus, by introducing mixed proposals, we have the positive results that there is a symmetric stationary perfect equilibrium and that it is efficient if agents are sufficiently patient. After investigating sequential bargaining games, we fully characterize the allocations in the core. A core allocation consists of a network and an effort profile for which there is no coalition of agents who can improve themselves by deviating from the allocation without cooperating with others. As a result, we find that an allocation in the core consists of a minimally connected network and an effort profile in which at most one agent does not makes a full effort to produce his public good. There are an increasing number of papers on networks. Jackson and Wolinsky (1996) study the pairwise stability and efficiency of network formation. In their study, mutual agreement is needed to form a link and there is a cost to sustain a link. Bala and Goyal (2000) investigate equilibria and efficiency using a noncooperative approach. They consider a model with one-way flow of benefits as well as a model with two-way flow. These studies assume that the value of a link is exogenously given and ignore the actions which determine the value of links. For example, investors decide how much to invest in a cooperative R&D venture with the decision of joint venture partners. Job seekers decide whether or not to acquire job information as well as with whom to share it. Jackson and Watts (2002) and Hojman and Szeidl (2006) provide network formation models with underlying games whose outcomes determine the value of links. Their models, however, leave out some aspects in analyzing the situation of producing and sharing information. In an information sharing network, for example, an agent may be able to access the information from other agents he does not know directly (e.g., accessing information of friends of friends). In addition, it seems reasonable to assume that linked agents share their information rather than to assume that only one agent extracts information from the other. In Jackson and Watts (2002), agents cannot enjoy benefits from indirectly connected agents. In Hojman and Szeidl (2006), an agent, say i, cannot reach information from the other, say j, if i does not make a link with j even though j makes a link with i. Galeotti

3We can also interpret a mixed proposal as time sharing of indivisible good as discussed in Bogomolnaia and Moulin (2004).

59 and Goyal (2008) study the strategic behavior in acquiring and sharing information through a network. In their model, the socially optimal level of information can be obtained from strategic behavior of agents without commitment, while in our model no production is the unique outcome if agents cannot make a binding agreement.4 The remainder of the paper is organized as follows. In Section 3.2, we explain the basic environment for acquiring and sharing information. Section 3.3 explains sequential bargaining games. Sequential bargaining games with pure proposals are discussed in Section 3.3.1 and games with mixed proposals are discussed in Section 3.3.2. In Section 3.4, we deﬁne and characterize core allocations. We conclude in Section 3.5.

3.2 Models

There are n ≥ 2 agents and the set of agents is N = {1, . . . , n}. Throughout the paper, we let t and s be the numbers of elements in T ⊂ N and S ⊂ N, respectively. Each agent i decides his effort xi ∈ [0, 1] to acquire information. Let x = (x1, . . . , xn) be an effort profile and X = [0, 1]n be the set of all effort profiles. Agent i’s effort determines the value and the cost of information he acquires. We denote v(xi) as the value of information agent i acquires and c(xi) as the cost of it when his effort is xi. We assume that v : [0, 1] → R and c : [0, 1] → R are continuous and satisfy v(0) = 0 and c(0) = 0. We also assume that, for each x, x0 ∈ [0, 1) and each ε > 0 with x + ε ∈ (0, 1] and x0 + ε ∈ (0, 1],

v(x) < v(x + ε), (3.1) c(x) < c(x + ε), (3.2) c(x) − v(x) < c(x + ε) − v(x + ε), (3.3) v(x) − c(x) + v(x0) < v(x + ε) − c(x + ε) + v(x0 + ε). (3.4)

Here, (3.1) and (3.2) mean that the value and the cost of information agent i acquires increase as his effort increases. (3.3) implies that, when an agent increases his effort, the benefit from his information is less than the cost he has to pay. This captures the idea that individual benefit from public good is smaller than its cost. Finally, (3.4) means that, 0 when agent i and j choose their efforts at xi = x and xj = x , increasing their efforts by the same amount and sharing their information are beneficial to agent i. This implies that social benefit from public good is greater than its cost.5 For distinct agents i and j, a link between them, denoted ij, is a subset of N containing

4Bramoull´eand Kranton (2007) also study the strategic behavior of producing public goods with an exogenously given network. In their model, each agent decides only the amount of a public good whose beneﬁt is shared with directly connected agents. 5Examples of v and c which satisfy these properties are linear functions, that is v(x) = vx and c(x) = cx where 0 < v < c < 2v.

60 only i and j. Note that ij = ji for all i, j ∈ N. Agents share their information by forming a network G, which is a set of links. A network G = {ij : i ∈ N and j ∈ N} is said to be a complete network, and G = ∅ is said to be an empty network. If ij ∈ G, we say that agents i and j are directly connected (or linked) in G. Agents i and j are indirectly connected in G if i 6= j and there exists a ﬁnite subset {i1, . . . , iL} of N with L ≥ 3 such that i1 = i, iL = j, and ilil+1 ∈ G for all l = 1,...,L − 1. We simplify notations by letting G + ij = G ∪ {ij} and G − ij = G\{ij}. Let G be the set of all networks.

For G ∈ G and S ⊂ N, we deﬁne a subnetwork G|S of G by

G|S = {ij ∈ G | i ∈ S and j ∈ S}.

A network G is connected in S if, for all i, j ∈ S, i and j are connected in G|S. G is simply said to be connected, if G is connected in N. A network G is minimally connected in S if

G|S is connected in S and, for each ij ∈ G|S, G|S −ij is not connected in S. G is minimally connected, if G is minimally connected in N. We also denote by G|S the set of all networks

G such that ij ∈ G implies i ∈ S and j ∈ S. G ∈ G|S can be interpreted as G can be achieved by agents in S without cooperating with others. Notice that, for any G ∈ G and

S ⊂ N, G|S ∈ G|S holds. For a network G, H(G) is the partition of N in which each element H ∈ H(G) is called a component of G. H ∈ H(G) if and only if, for all i, j ∈ H, i = j or they are connected in G and, for all i ∈ H and j 6∈ H, i and j are not connected in G.6 Let H(i, G) denote a component to which agent i belongs. Note that j ∈ H(i, G) implies i ∈ H(j, G) and

H(j, G) = H(i, G). In addition, for any H ∈ H(G), G|H is connected in H. Let L(i, G) be the set of links in which agent i is involved. Note that 0 ≤ |L(i, G)| ≤ n−1 and that, for each i ∈ N, L(i, G) = L(i, G|H(i,G)). Each agents has to pay a cost d > 0 for maintaining a link. We assume that the link cost is small enough to satisfy

2v(1) − c(1) − 2d > 0. (3.5)

Since (3.3) implies c(1) − v(1) > 0, (3.5) implies v(1) − 2d > 0 and c(1) − 2d > 0. An agent i can access the information acquired by himself and others who are (directly or indirectly) connected to him. We assume that the value of information acquired by different agents is separable, and that there is no decay of information from indirect connections. We call a pair (G, x) of network and effort profile an allocation. For an allocation (G, x),

6This definition of component is different from that found in Jackson and Wolinsky (1996). In Jackson and Wolinsky (1996), a component of G is a subnetwork G|H where H ∈ H(G) and |H| ≥ 2. Given a network, an agent who does not have a link can be a component under our definition, while it is not a component under their definition.

61 agent i’s payoﬀ is given by X Πi(G, x) = v(xj) − c(xi) − |L(i, G)|d. (3.6) j∈H(i,G)

Here, notice that there is no externality across components in the sense that agent i’s payoff does not depend on the efforts of others who are not connected to him. However, there is negative externality inside a component in the sense that increasing agent i’s effort decreases his payoff while it increases the payoff of others who are connected to him. Some previous studies such as Bramoulléand Kranton (2007) and Galeotti and Goyal

(2008) consider payoﬀ functions in which c(xi) is linear and the value of information is P P v( j∈H(i,G) xj) rather than j∈H(i,G) v(xj). We note that, in their environments, strategic behaviors without commitment may result in a socially optimal level of public goods (though it is not always), so Nash equilibrium may be a reasonable solution concept. However, in our environment, no production is the unique outcome without commitment and a binding agreement is necessary to achieve social eﬃciency.7

3.3 Sequential Bargaining Games

In this section, we describe sequential bargaining games for information sharing networks. A sequential bargaining game is characterized by two protocols, proposer protocol p and respondent protocol r. For the set T of remaining agents, p determines the initial proposer p(T ) ∈ T . When proposer p ∈ T makes a proposal to a coalition S with s ≥ 2, r determines the order of respondents in S\{p}. For example, when an agent p makes the proposal to a coalition S with j ∈ S\{p}, r(j; S\{p}) = l means that agent j is the lth respondent. We consider two types of sequential bargaining games which are distinguished by permissible proposals. In the game we consider first, each agent chooses a coalition S and an allocation (G, x) which is achievable by S when he has to make a proposal. We call such a proposal a pure proposal. In the second game, each proposal, called a mixed proposal, consists of a coalition S and a distribution on the set of allocations which are achievable by S. In the sequential bargaining game with mixed proposals, each agent chooses his action based on his expected payoff. A sequential bargaining game with pure proposals proceeds as follows. The initial proposer p = p(N) makes a proposal (S, G, x) which satisfies S ⊂ N, p ∈ S, G ∈ G|S,

7We can consider Nash equilibrium as a solution concept for the environment in which we are interested. A strategy si of agent i consists of his eﬀort xi and a set Gi of agents whom agent i wants to link with. Each agent i simultaneously announces his strategy si = (xi,Gi), and the outcome of the game is (G, x) where ij ∈ G if and only if j ∈ Gi and i ∈ Gj . The payoﬀ of each agent i is given by (3.6). In this game, the unique Nash equilibrium outcome is G = ∅ and xk = 0 for all k ∈ N.

62 8 and xk = 0 for k∈ / S. Then, the agents in S\{p}, if any, respond to the proposal by saying “Yes” or “No” in the order following the respondent protocol. If there is an agent who rejects the proposal, then the payoffs of all agents in the game are discounted with common discount factor δ ∈ (0, 1) and the first rejecter becomes a new proposer. If there is no respondent who rejects the proposal, then every agent i in S leaves the game with 9 payoff Πi(G, x) and a new game starts with the set N\S of remaining agents. We assume that there is no lapse in time before starting a new game. This procedure continues until all agents leave the game. Note that δ ∈ (0, 1) implies that the payoffs of agents who stay in the game forever are zero. A strategy of an agent is stationary if his action, when he has to make a proposal, depends only on the set of remaining agents and, when he has to respond to a proposal, depends only on the set of remaining agents and ongoing proposal. A stationary perfect equilibrium is a perfect equilibrium in which each agent employs a stationary strategy. ˜ P Consider a subgame where T with t ≥ 2 is the set of remaining agents. Let Πi (T ) be an equilibrium payoff to agent i when he is the initial proposer in this subgame. Let ˜ R ˜ P Πi (T ) = δΠi (T ). Suppose that the initial proposer p in this subgame proposes (S, G, x) in a stationary perfect equilibrium. Then, the last respondent j will accept the proposal if ˜ R ˜ R 10 Πj(G, x) ≥ Πj (T ), and reject it if Πj(G, x) < Πj (T ). Given the last respondent’s action, 0 ˜ R the second last respondent j will accept the proposal if Πk(G, x) ≥ Πk (T ) for all k such that 0 ˜ R ˜ R r(j ; S\{p}) ≤ r(k; S\{p}), and reject it if Πj0 (G, x) < Πj0 (T ) and Πk(G, x) ≥ Πk (T ) for all k such that r(j0; S\{p}) < r(k; S\{p}). Continuing these arguments, in an equilibrium, ˜ R agent i accepts a proposal (S, G, x) proposed by agent p if Πk(G, x) ≥ Πk (T ) for all k such ˜ R ˜ R that r(i; S\{p}) ≤ r(k; S\{p}), and rejects it if Πi(G, x) < Πi (T ) and Πk(G, x) ≥ Πk (T ) for all k such that r(i; S\{p}) < r(k; S\{p}). Given the reactions of respondents in an

8 Since the network in a proposal (S, G, x) belongs to G|S , the payoff of each agent in S does not depend on xk for k∈ / S. The constraint that xk = 0 for k∈ / S is added in order to describe the proposal completely. 9Given a strategy, let S1,...,SM be coalitions which are eventually formed and leave the game and let m m m (G , x ) be an allocation with which coalition S leavesP the game. The resulting allocation in a sequential ˜ ˜ M m M m ˜ m m bargaining game is (G, x˜) where G = ∪m=1G andx ˜ = m=1 x . Since Πk(G, x˜) = Πk(G , x ) for each m m m m k ∈ S , we can consider Πk(G , x ) as the payoff of agent k ∈ S instead of Πk(G,˜ x˜). Note that we allow S to be a singleton, that is, S = {p}. If S = {p} in a proposal (S, G, x), then G = ∅ and x = (0,..., 0, xp, 0,..., 0). Thus, agent p leaves the game with payoff Πi(∅, x). 10 ˜ R To ensure the equilibrium, agent j should accept the proposal if Πj (G, x) = Πj (T ). This holds true for all other respondents.

63 equilibrium, a proposer p can guarantee the payoﬀ

max Πp(G, x) subject to (S,G,x) p ∈ S S ⊂ T (P ) xj = 0 for all j∈ / S

G ∈ G|S ˜ R if S\{p} 6= ∅,Πj(G, x) ≥ Πj (T ) for all j ∈ S\{p}

˜ P by making an accepted proposal. Therefore, in any equilibrium, for any p ∈ T , Πp (T ) is greater than or equal to the maximum value of Problem (P ). If, in a stationary perfect equilibrium, every agent employs the same strategy in the ˜ R ¯ R sense that, for each i, Πi (T ) = Π (T ) for all T ⊂ N, we call such an equilibrium a symmetric stationary perfect equilibrium. In this paper, we restrict our attention to symmetric stationary perfect equilibria in order to make the analysis tractable. An equilibrium satisﬁes the no-delay property, if every agent makes an accepted proposal when he has to make a proposal.

Lemma 3.1. Every symmetric stationary perfect equilibrium satisﬁes the no-delay property.

Proof. Consider a symmetric stationary perfect equilibrium. Suppose that a proposer p makes a rejected proposal in a subgame in which T with t ≥ 2 is the set of remaining agents. Since the strategies are symmetric and stationary, every successive proposer makes a rejected proposal. So, the payoff for each agent k ∈ T is zero in this subgame. However, p can get a positive payoff by offering an accepted proposal (S, G, x) such that S = T , G is minimally connected in T , and xj = 1 for all j ∈ T . This means that making a rejected proposal in the subgame is not agent p’s best response to the strategy of others.

Consider a symmetric stationary equilibrium and a subgame where T is the set of remaining agents and p ∈ T has to make a proposal. Lemma 3.1 implies that p makes an accepted proposal, and so his payoﬀ Π¯ P (T ) in this subgame is equal to the value of Problem (P ) where ˜ R ¯ R ¯ P Πj (T ) = Π (T ) = δΠ (T ) (3.7) for each j ∈ S\{p}. In what follows, Problem (P ) refers to Problem (P ) satisfying (3.7).

64 3.3.1 Non-Existence of Symmetric Stationary Perfect Equilibrium with Pure Proposals

In this subsection, we show that a sequential bargaining game with pure proposals does not have a symmetric stationary perfect equilibrium if there are at least four agents and they are suﬃciently patient. Consider a subgame where T is the set of remaining agents and p ∈ T is the initial proposer. Lemma 3.1 implies that, in a symmetric stationary perfect equilibrium, agent p P makes proposal (S, G, x) which solves Problem (P ) and satisﬁes Πp(G, x) = Π¯ (T ). First, we provide the necessary conditions for a proposal (S, G, x) to be a solution of Problem (P ).

Lemma 3.2. Let T be the set of remaining agents with t ≥ 2. Suppose that (S, G, x) be a R solution of Problem (P ) with Πp(G, x) ≥ Π¯ (T ). Then, it satisﬁes

(a) S = T and G is connected in T ,11

(b) if xi < 1 for some i ∈ S, xj = 1 for all j ∈ S\{i},

c(x) = c(1) − d, (3.8)

R (d) if xp > 0, Πj(G, x) = Π¯ (T ) for some j ∈ S\{p},

0 0 (e) if xj = 1 and |L(j, G)| > |L(j ,G)| for some j, j ∈ S\{p}, |L(p, G)| = 1,

(f) if xp = 1, |L(p, G)| ≤ |L(j, G)| for all j ∈ S\{p},

(g) |L(j, G)| = 1 for some j ∈ S, and

(h) if |L(p, G)| > 1 and |L(i, G)| > |L(j, G)| for some i, j ∈ S\{p}, |L(k, G)| = |L(j, G)| = 1 for all k ∈ S\{p, i, j}.

Proof. For each of (a), (b), and (c), it is enough to construct a proposal (T,G0, x0) with which p increases his payoﬀ and the other agents in T get the payoﬀ at least Π¯ R(T ), provided that each condition is violated. The arguments are similar to those in Section 3.4, so omitted. R For (d), suppose that xp > 0 and Πj(G, x) > Π¯ (T ) for all j ∈ S\{p}. Since xp > 0 R and v is continuous, there is ε > 0 satisfying Πj(G, x) − Π¯ (T ) > v(xp) − v(xp − ε) > 0 0 0 0 for all j ∈ S\{p}. Let G = G and let xp = xp − ε and xj = xj for j 6= p. Then, 0 0 0 0 R Πp(G , x ) − Πp(G, x) > 0 and Πj(G , x ) = Πj(G, x) + v(xp − ε) − v(xp) > Π¯ (T ) for j ∈ S\{p}. This contradicts that (S, G, x) is a solution of Problem (P ).

11It is not necessary that G is minimally connected in T for (S, G, x) to be a solution of Problem (P ).

65 0 0 For (e), suppose that xj = 1 and |L(j, G)| > |L(j ,G)| for some j, j 6= p and |L(p, G)| ≥ 2. From (a) and (b), we know that S = T and G is connected in T . Then, there exists an agent i0 such that pi0 ∈ G, and p and j0 are connected in G − pi0. Let x0 = x and G0 = 0 0 0 0 0 0 P G−pi +i j . Note that G is also connected in T . Since Πj0 (G , x ) = v(xk)−c(xj0 )− P k∈S 0 ¯ R 0 0 (|L(j ,G)| + 1)d ≥ k∈S v(xk) − c(1) − |L(j, G)|d = Πj(G, x) ≥ Π (T ) and Πi(G , x ) = R 0 0 0 Πi(G, x) ≥ Π¯ (T ) for all i 6= j , p,(T,G , x ) will be accepted by T . However, since 0 0 Πp(G , x ) − Πp(G, x) = d > 0, (S, G, x) cannot be a solution of Problem (P ).

For (f), Let (S, G, x) be a solution of Problem (P ) with xp = 1. By (a) and (b), S = T and G is connected in T . Suppose that |L(p, G)| > |L(j, G)| for some j ∈ S\{p}. Then, since |L(p, G)| ≥ 2, there is an agent i ∈ L(p, G) such that G − pi + ij is connected in 0 0 0 0 R T . Let G = G − pi + ji and x = x. Then, Πj(G , x ) = Πp(G, x) + d ≥ Π¯ (T ) and 0 0 R 0 0 Πk(G , x ) = Πk(G, x) ≥ Π¯ (T ) for each k ∈ S\{p, j}. Since Πp(G , x ) − Πp(G, x) = d > 0, we have a contradiction. For (g), suppose that |L(k, G)| 6= 1 for all k ∈ S. Since G is connected in T , |L(k, G)| > 1 for all k ∈ S. Let G0 be a line-shaped network on T where |L(p, G)| = 1 and x0 = x.12 0 0 0 0 Since |L(p, G )| < |L(p, G)| and |L(k, G )| ≤ |L(k, G)| for each k 6= p, we have Πp(G , x ) > 0 0 R Πp(G, x) and Πk(G , x ) > Πk(G, x) ≥ Π¯ (T ) for each k 6= p. This is a contradiction. For (h), let (S, G, x) be a solution of Problem (P ) in which |L(p, G)| > 1 and |L(i, G)| > |L(j, G)| for some i, j ∈ S\{p}. Suppose that |L(k, G)|= 6 |L(j, G)| for some k ∈ S\{p, i, j}.

From (a), S = T and |L(k, G)| ≥ 1 for all k ∈ T . If xi = 1, then |L(i, G)| > |L(j, G)| and

(e) imply |L(p, G)| = 1, which is a contradiction. So, xi < 1 which implies xk = 1 and xj = 1. Since |L(k, G)| 6= |L(j, G)| and xk = xj = 1, (e) implies |L(p, G)| = 1, which is a contradiction. Therefore, we have |L(k, G)| = |L(j, G)| for all k ∈ S\{p, i, j}. Suppose that |L(j, G)| > 1. Since |L(k, G)| = |L(j, G)| for all k ∈ S\{p, i, j}, |L(k, G)| > 1 for all k ∈ S\{p, i}. Since |L(p, G)| > 1 and |L(i, G)| > |L(j, G)| > 1, we have |L(k, G)| > 1 for all k ∈ S, which contradicts (g).

Note that the continuity of c(·) and 0 < c(1) − d < c(1) ensures the existence of x satisfying (3.8) and x ∈ (0, 1). Throughout the paper, the deﬁnition of x in (3.8) is preserved. Before moving on to Proposition 3.1, consider a sequential bargaining game with N = {1, 2, 3}. Let δ satisfy δ ≤ δ < 1 where δ = (3v(1) − c(1) − 2d)/(3v(1) − c(1) − d). Then, there exist x∗ ∈ [x, 1] and x∗∗ ∈ (0, 1) satisfying 2v(1) + v(x∗) − c(x∗) − 2d = δ(2v(1) + v(x∗) − c(1) − d) and v(1) + v(x∗∗) − c(1) − d = δ(v(1) + v(x∗∗) − c(x∗∗) − d). Consider a stationary strategy such that each agent i makes a proposal (N, G, x) where ∗ G = {ij, jk} and (xi, xj, xk) = (1, x , 1) when N is the set of remaining agents and he has

12A line-shaped network is a network G, which can be obtained from GL = {12, 23,..., (n − 1)n} by relabeling agents.

66 to make a proposal. In addition, he makes a proposal ({i, j},G0, x0) where G0 = {ij} and 0 0 ∗∗ (xi, xj) = (x , 1) when he has to make a proposal with two remaining agents, i and j. If he remains alone in the game, he leaves the game with (∅, 0). By construction of (G, x) and 0 0 (G , x ), we have Πk(G, x) = δΠi(G, x) = δΠj(G, x) when N is the set of remaining agents 0 0 0 0 and Πj(G , x ) = δΠi(G , x ) when {i, j} is a set of remaining agents. One can show that these actions constitute a symmetric stationary perfect equilibrium for sufficiently patient agents. So, we have a positive result that, if n ≤ 3, there is a symmetric stationary perfect equilibrium for sufficiently patient agents.13 However, if there are at least four agents and they are sufficiently patient, a symmetric stationary perfect equilibrium no longer exists.

Proposition 3.1. Let n ≥ 4. There exists δˆ ∈ (0, 1) such that, for all δ ∈ [δ,ˆ 1), there does not exist a symmetric stationary perfect equilibrium.

Proof. See Appendix A.

The proof of Proposition 3.1 is in Appendix A. Here, we provide the sketch of the proof. Suppose that an allocation (G, x) is sustained by a symmetric stationary perfect equilibrium. From Lemmas 3.1 and 3.2, we know that the initial proposer p makes an accepted proposal (N, G, x) where G is connected. Suppose, in addition, that |L(p, G)| ≥ 2. If G is a star- 14 shaped network with p in the center, Lemma 3.2 (c) implies that xp ≥ x. Then, the payoff of the initial proposer p is less than those of the other agents. Therefore, G cannot be a star-shaped network, which may imply that there are agents, i 6= p and j 6= p, such that 0 0 xi ≥ x, |L(i, G)| ≥ 2, and |L(j, G)| = 1. Consider an allocation (G , x ) in which j has 0 0 0 exactly two links and x = x. If xi = 1, then Πj(G , x ) ≥ Πi(G, x). Since (G, x) is accepted by i and every agent employs a symmetric stationary strategy, (G0, x0) is also accepted by j. 0 0 If xi < 1, then Lemma 3.2 (b) implies xp = 1 and so Πj(G , x ) ≥ Πp(G, x). Since strategies are symmetric and stationary, j will accept (G0, x0). Then, we can find a connected network G0 in which |L(j, G0)| = 2, |L(p, G0)| < |L(p, G)|, and |L(k, G0)| ≤ |L(k, G)| for all k 6= p, j. This means that the initial proposer p can improve his payoff by making an accepted offer (G0, x0). This contradicts the fact that (G, x) is sustained as an equilibrium. Therefore, we have |L(p, G)| = 1. Since n ≥ 4, there is an agent i such that |L(i, G)| ≥ 3, or an agent j such that |L(j, G)| ≥ 2 and xj = 1. If there is an agent i such that |L(i, G)| ≥ 3, we have

Πp(G, x) > Πi(G, x), and if there is an agent j such that |L(j, G)| ≥ 2 and xj = 1, we have

13To construct an equilibrium, we have to determine each agent’s actions when he has to respond to each proposal. Indeed, we can construct those actions in a similar way to strategy σ∗, which we will construct in Section 3.3.2. The same arguments as in Section 3.3.2 can be applied to show that, if there are three agents and they are suﬃciently patient, these actions as respondent together with those as proposer constitute a symmetric stationary perfect equilibrium. 14A star-shaped network is a network G such that there is an agent k satisfying ij ∈ G if and only if i = k or j = k, and we say that agent k is in the center.

67 Πp(G, x) > Πj(G, x). These observations reveals that, if they are sufficiently patient, there is an agent who rejects the proposal (N, G, x). Therefore, no allocation can be sustained as a symmetric stationary perfect equilibrium. It has to be mentioned that this negative result comes from the discreteness of G that makes the utility possibility frontier non-connected around 45 degree line. An usual way to obtain a connected utility possibility frontier is to introduce a side payment among the agents with quasi-linear payoff functions. Then, we may have a TU bargaining game P (N, ν) where ν is a characteristic function defined by ν(S) = max(G,x)∈G|S ×X i∈S Πi(G, x). Since (N, ν) is strictly convex, the existence of stationary perfect equilibrium follows from Chatterjee et al. (1993).15 In Section 3.3.2, we provide another way to make the utility possibility frontier connected. Note that Proposition 3.1 does not rule out the existence of asymmetric stationary perfect equilibrium. If n = 2, we can easily see that the utility possibility frontier is 2 symmetric and connected in R+. In this case, it is not dicult to show that there is a unique stationary equilibrium and it is symmetric. On the other hand, we can construct a sequential bargaining game whose utility possibility frontier is not connected and which has an asymmetric stationary perfect equilibrium but not a symmetric stationary perfect equilibrium. In our environment with n ≥ 4, it seems difficult to verify the existence of asymmetric stationary perfect equilibrium.

3.3.2 Existence of Symmetric Stationary Perfect Equilibrium with Mixed Proposals

In this subsection, we allow agents to make a mixed proposal which is a distribution on the set of feasible allocations. By allowing the agents to make mixed proposals, we have the utility possibility frontier connected around the 45 degree line. The connectedness of the utility possibility frontier ensures the existence of symmetric stationary perfect equilibrium. In particular, there is a symmetric stationary perfect equilibrium in which each proposer oﬀers a distribution on the set of equivalent allocations. The procedure of a sequential bargaining game with mixed proposals is almost same as the procedure with pure proposals. The only diﬀerence is that a proposal consists of a coalition and a distribution on the allocations achievable by the coalition. Consider a subgame where T is the set of remaining agents and p ∈ T has to propose. For each S ⊂ T , let

AS = {(G, x) ∈ G × X | G ∈ G|S and xk = 0 for all k∈ / S}, in which Borel σ-algebra is embedded. In words, AS is the set of allocations which S

15A bargaining game (N, ν) is strictly convex if, for all S, T ⊂ N with T \S and S\T non-empty, ν(S ∪T ) > ν(S) + ν(T ) − ν(S ∩ T ).

68 can achieve without cooperating with others. Let ∆(AS) be the set of distributions αS satisfying supp(αS) ⊂ AS where supp(αS) denotes the support of αS. Note that, for any 0 0 S, S ⊂ N with S ⊂ S , AS ⊂ AS0 and ∆(AS) ⊂ ∆(AS0 ). A mixed proposal consists of a coalition S ⊂ T and a distribution αS ∈ ∆(AS). In a sequential bargaining game with mixed proposals, each respondent j chooses his action depending on his expected payoff. Let Π¯ P (T ) be the expected payoff of agent p when he is the proposer in a subgame with T being the set of remaining agents. Let Π¯ R(T ) = δΠ¯ P (T ). From the similar arguments as before, we know that p does not make a rejected proposal and his payoff Π¯ P (T ) is the value of the problem Z

max Πp(G, x)dαS subject to (S,αS ) p ∈ S S ⊂ T (M)

αS ∈ ∆(AS) Z R if S\{p} 6= ∅, Πk(G, x)dαS ≥ Π¯ (T ) for all k ∈ S\{p}.

We will construct an equilibrium where each proposal is a random mixture on an equivalent class of allocations. An allocation (G0, x0) is equivalent to (G, x) if it can be obtained from (G, x) by relabeling the ex-ante symmetric agents. Formally, (G0, x0) is equivalent to 0 0 (G, x) if there is a bijection π : N → N for which G = {π(i)π(j) | ij ∈ G} and xπ(k) = xk for all k ∈ N. Here, we say that (G0, x0) is a permutation of (G, x). For (G, x), an equivalent class of (G, x), denoted by A(G,x), is the set of allocations which are equivalent to (G, x). Note that (G0, x0) ∈ A(G,x) implies A(G,x) = A(G0,x0) and that (G0, x0) ∈/ A(G,x) implies A(G,x) ∩ A(G0,x0) = ∅. If (G0, x0) ∈ A(G,x) with a bijection π, it is satisfied that 0 0 ∗ ∗ Ππ(k)(G , x ) = Πk(G, x) for all k ∈ N. Furthermore, for (G, x) ∈ AT and (G , x ) ∈ AT P P ∗ ∗ ∗ ∗ which are equivalent, k∈T Πk(G, x) = k∈T Πk(G , x ) holds. Therefore, if (G , x ) ∈ AT (G∗,x∗) and αT ∈ ∆(AT ) satisfy supp(αT ) ⊂ A , the aggregate expected payoff for T at αT is equal to the aggregate payoff for T at (G∗, x∗), that is, X Z X ∗ ∗ Πk(G, x)dαT = Πk(G , x ). k∈T k∈T

∗ ∗ (G∗,x∗) ∗ ∗ Next, we deﬁne a proposal (T, αT ) satisfying αT ∈ A for some (G , x ) ∈ AT for each δ ∈ (0, 1) and each t ≥ 2. Consider a subgame where T is the set of remaining agents

69 and p has to propose. For each t ≥ 2, let

(t − 1)2v(1) − (t − 1)c(1) − (2t − 3)d δ = , and t (t − 1)2v(1) − (t − 1)d (3.9) t(t − 1)v(1) − (t − 1)c(1) − (2t − 3)d δ¯ = . t t(t − 1)v(1) − (t − 1)c(1) − (t − 1)d

It is not diﬃcult to see that

δ < 0 < δ¯t = 1 for t = 2, 0 < δ < δ¯t < 1 for t ≥ 3, t t (3.10) ¯ ¯ 0 < δt < δt+1 < 1 for t ≥ 3, 0 < δt < δt+1 < 1 for t ≥ 3.

Case 1 (t = 2 and 0 < δ < 1). Let T = {p, j}. Since 0 < c(1) − v(1) + d < v(1), there o o ¯ exists x ∈ (0, 1) satisfying v(x ) = c(1) − v(1) − d. In addition, δt < 0 < δ < 1 = δt ensures ∗ o ∗ ∗ ∗ the existence of xp ∈ (x , 1) satisfying v(1) + v(xp) − c(1) − d = δ(v(1) + v(xp) − c(xp) − d). ∗ ∗ ∗ Let G ∈ G|T be a network which is (minimally) connected in T . Let xj = 1 and xk = 0 ∗ for k∈ / T . Deﬁne αT ∈ ∆(AT ) by ( 1 if (G, x) = (G∗, x∗) ∗ αT (G, x) = 0 otherwise

∗ ∗ ∗ Case 2 (t ≥ 3 and δ¯t ≤ δ < 1). Let (G , x ) ∈ AT satisfy that G is a star-shaped ∗ ∗ network with p in the center, xk = 1 for all k ∈ T , and xk = 0 for all k∈ / T . Let ( ¯ ) ¯ (G, x) ¯ (G, x) is a permutation of (G∗, x∗) such that A2 = ¯ . ∈ G × X ¯ (G, x) ∈ AT and |L(p, G)| = 1

Note that |A2| = t − 1. Furthermore, for each (G, x) ∈ A2,Πp(G, x) = tv(1) − c(1) − d. ∗ Since δ¯t ≤ δ < 1, there exists a ∈ (0, 1] such that

1 X a∗(tv(1) − c(1) − d) + (1 − a∗)Π (G∗, x∗) = Π (G∗, x∗) p 1 + δ(t − 1) k k∈T

∗ Deﬁne αT ∈ ∆(AT ) by

 ∗  a  if (G, x) ∈ A2 ∗ t − 1 α (G, x) = ∗ ∗ ∗ T  1 − a if (G, x) = (G , x )  0 otherwise

70 ¯ ∗ Case 3 (t ≥ 3 and δt < δ < δt). Let G ∈ G|T be a star-shaped network such that ∗ ¯ ∗ |L(p, G )| = 1. Since v and c are continuous and δt < δ < δt, there exists x such that

1 X Π (G∗, x∗) = δΠ (G∗, x∗) t − 1 k p k∈T \{p}

∗ ∗ ∗ with xk = 1 for k ∈ T \{p}, xk = 0 for k∈ / T and xp ∈ (0, 1). Let ( ¯ ) ¯ (G, x) ¯ (G, x) is a permutation of (G∗, x∗) such that A3 = ¯ ∗ . ∈ G × X ¯ (G, x) ∈ AT , |L(p, G)| = 1, and xp = xp

∗ Note that |A3| = t − 1. Deﬁne αT ∈ ∆(AT ) by   1 ∗ if (G, x) ∈ A3 αT (G, x) = t − 1  0 otherwise

∗ Case 4 (t ≥ 3 and 0 < δ ≤ δt). Let G ∈ G|T be a star-shaped network such that ∗ ∗ ∗ ∗ |L(p, G )| = 1. Let xp = 0, xk = 1 for k ∈ T \{p}, and xk = 0 for k∈ / T . Let ( ¯ ) ¯ (G, x) ¯ (G, x) is a permutation of (G∗, x∗) such that A4 = ¯ . ∈ G × X ¯ (G, x) ∈ AT , |L(p, G)| = 1, and xp = 0

∗ Note that |A4| = t − 1. Deﬁne αT ∈ ∆(AT ) by   1 ∗ if (G, x) ∈ A4 αT (G, x) = t − 1  0 otherwise

∗ In Cases 1 to 4, we deﬁne a distribution αT ∈ ∆(AT ) for each t ≥ 2 and for each ∗ ∗ (G∗,x∗) δ ∈ (0, 1). By construction, we can show that αT satisﬁes supp(αT ) ⊂ A for an ∗ ∗ allocation (G , x ) ∈ AT and, for all k ∈ T \{p}, Z Z ∗ ∗ ∗ Πp(G, x)dαT ≥ Πk(G, x)dαT ≥ δ Πp(G, x)dαT > 0.

Thus, for each of Cases 1 to 4, α∗ satisﬁes the constraints of Problem (M) with Π¯ R(T ) = R T ∗ δ Πp(G, x)dαT . Lemma 3.3. Suppose that v is concave and c is convex.16 For each of Cases 1 to 4, α∗ is R T ¯ R ∗ a solution of Problem (M) with Π (T ) = δ Πp(G, x)dαT . 16The assumptions of concave v and convex c can coincide with the maintained assumptions in (3.1) to (3.4). An example of v and c satisfying these assumptions is v(x) = vx and c(x) = cx with 0 < v < c < 2v. The concavity of v and the convexity of c are not required for the results in Section 3.4.

71 Proof. See Appendix B.

The proof of Lemma 3.3 is in Appendix B. Here, the concavity of v and the convexity of c ensure the payoff functions to be concave in effort profile. So, agents are risk averse in effort levels to produce public goods. Notice that, for each case, the effort profiles in ∗ the support of αT are equivalent in the sense that they are obtained just by relabeling the ex-ante symmetric agents. If we drop these assumptions on v and c, the proposer may be able to increase his payoff by making a proposal in which effort profile is random, instead of α∗ . T R ¯ R ∗ For each of Cases 1 to 4, let Π (T ) = δ Πp(G, x)dαT . We define a strategy profile ∗ ∗ 17 σ = (σi )i∈N as follows.

(P1) When agent i has to make a proposal in a history that T with t ≥ 3 is the set of ∗ remaining agents, he proposes αT which is described in Cases 2 to 4 with i = p.

(P2) When agent i has to make a proposal in a history that T with t = 2 is the set of ∗ remaining agents, he proposes αT which is described in Case 1 with i = p.

(P3) When agent i is in a history that T = {i} is the set of remaining agents, he leaves the ∗ game with xi = 0.

(R1) When agent i has to respond to an ongoing proposal (S, αS) which is proposed by p in a history that T with t ≥ 3 being the set of remaining agents, he accepts the proposal R R if Πk(G, x)dαS ≥ Π¯ (T ) for all k ∈ S\{p} who are not respond yet (including i), and rejects it otherwise.

(R2) When agent i has to respond to an ongoing proposal (S, αS) which is proposed by p in a history that T with t = 2 being the set of remaining agents, he accepts the proposal R R if Πi(G, x)dαS ≥ Π¯ (T ), and rejects it otherwise.

Proposition 3.2. Suppose that v is concave and c is convex. For any δ ∈ (0, 1), there exists a symmetric stationary perfect equilibrium in which the only source of mixing is a probabilistic choice on an equivalent class of an allocation.

∗ Proof. Consider the strategy σi of agent i. For the proof, we will use the one deviation property of perfect equilibrium. Consider a subgame with T being the set of remaining ∗ agents. For T with t ≥ 2, Lemma 3.3 implies that σi gives the highest payoﬀ to agent i among the accepted proposals when he has to make a proposal, given the strategy σ∗ of R −i ¯ R ∗ others. If he makes a rejected proposal, his payoﬀ is at most Π (T ). Since Πp(G, x)dαT >

17 ∗ The strategy σi we construct is very similar to that in Chatterjee et al. (1993), and so the proof of Proposition 3.2 is similar to the arguments in Chatterjee et al. (1993).

72 ¯ R ∗ Π (T ), agent i cannot improve his payoff by deviating from σi when he has to make a ∗ ∗ proposal. For T = {i}, G = ∅ and xi = 0 is trivially the optimal choice for agent i. Consider agent i to be the last respondent to an ongoing proposal (S, αS). Since his payoff when he rejects the proposal is Π¯ R(T ), it is optimal for agent i to accept R R (S, αS) if and only if Πi(G, x)dαS ≥ Π¯ (T ). Consider agent i to be a respondent but not the last respondent to a proposal (S, α ) offered by p. Suppose that (S, α ) satis- R S S R fies Πk(G, x)dαS ≥ Π¯ (T ) for all k ∈ S\{p} who does not respond yet (including i). Given the others’ strategies σ∗ , it is optimal for agent i to accept (S, α ) since agent −i R S i gets at most Π¯ R(T ) by rejecting it and Π (G, x)dα by accepting it. Suppose that R i S R Πk(G, x)dαS < Π¯ (T ) for some k with r(i; S\{p}) < r(k; S\{p}). Since he gets at most ¯ R ¯ R ∗ Π (T ) by accepting (S, αS) and gets Π (T ) by rejecting it and proposing αT with p = i in ∗ the next period, he cannot improve his payoff by deviating from σi . The above arguments show that each agent does not have an incentive to deviate from σ in any history. Then, by construction, σ∗ is a symmetric stationary perfect equilibrium and the only source of mixing is a probabilistic choice on an equivalent class.

In the proof of Proposition 3.2, we show that σ∗ is a symmetric stationary perfect ∗ ∗ equilibrium. The outcome of σ is that the initial proposer p offers (N, αN ) in each case ∗ and all other agents accept the proposal. If δ ≥ δ¯n, σ yields a minimally connected network and every agent making the full effort with probability one. Thus, under the equilibrium σ∗, the aggregate expected payoff of all agents is X Z ∗ 2 Πi(G, x)dαN = n v(1) − nc(1) − 2(n − 1)d. (3.11) i∈N

Furthermore, as δ converges to one, all agents receive the same expected payoff under σ∗. It is worthwhile to mention that the concavity of v and the convexity of c are required not for the existence of symmetric stationary perfect equilibrium but for the strategy σ∗ to be an equilibrium. For the existence of symmetric stationary perfect equilibrium, the assumptions on v and c are not needed, since the Theorem of Maximum and the Brouwer’s Fixed Point Theorem may imply the result. For details, see Chatterjee et al. (1993). However, this approach does not show what the equilibrium looks like. By assuming that v is concave and c is convex, we can find and describe an equilibrium σ∗. Note that σ∗ is not uniquely determined and there can be symmetric stationary perfect equilibria which do not have the features of σ∗. However, Proposition 3.3 shows that a symmetric stationary perfect equilibrium is unique in terms of payoffs.

Proposition 3.3. Let n ≥ 3. Let p be the initial proposer in the game. Suppose that v is concave and c is convex. Let σ¯ be a symmetric stationary perfect equilibrium. If R ¯ ∗ δn ≤ δ < 1, the expected payoﬀ of p is Πp(G, x)dαN and every other agent’s expected payoﬀ

73 R is δ Π (G, x)dα∗ where α∗ is defined in Case 2 with T = N. If δ < δ < δ¯ , the expected p R N N n Rn ∗ ∗ payoff of p is Πp(G, x)dαN and every other agent’s expected payoff is δ Πp(G, x)dαN , where α∗ is defined in Case 3 with T = N. If 0 < δ ≤ δ , the expected payoff of p is R N n ∗ ∗ Πp(G, x)dαN , where αN is defined in Case 4 with T = N. Proof. See Appendix B.

An allocation (G, x) is eﬃcient if, for all (G0, x0) ∈ G × X , X X 0 0 Πk(G, x) ≥ Πk(G , x ). k∈N k∈N

Note that the efficiency in this definition concerns aggregate payoffs rather than a Paretian notion.18 It is not difficult to see that an allocation (G, x) is efficient if and only if G is minimally connected and xi = 1 for all i ∈ N. In addition, if (G, x) is efficient, the aggregate P 2 19 payoff satisfies k∈N Πk(G, x) = n v(1) − nc(1) − 2(n − 1)d. Proposition 3.4 states that, if agents are sufficiently patient, inefficient allocations cannot be realized in a symmetric stationary perfect equilibrium.

Proposition 3.4. Let n ≥ 3. Suppose that v is concave and c is convex. If δ¯n ≤ δ < 1, eﬃcient allocations are obtained with probability one in any symmetric stationary perfect equilibrium.

Proof. See Appendix B.

In Proposition 3.4, since δ¯n increases and converges to 1 as n increases, we can say that it is more diﬃcult to achieve eﬃciency with more agents in the sense that more patience is required. Furthermore, one can show that, if v is concave, c is convex, and 0 < δ <

δ¯n, eﬃcient allocations cannot be obtained with a positive probability in any symmetric stationary equilibrium.

3.4 Allocations in the Core

In the previous section, we investigate a procedure of making a binding agreement through sequential bargaining games. Our interest in this section is to characterize the core allocation, which is another approach to study the allocations possible from a binding agreement. We can interpret a core allocation as an allocation which is stable to coalitional deviations.20

18Since payoffs are not transferable across players in our environment, this definition is not equivalent to Pareto efficiency. The former is stronger than the latter. 19For details, see Lemma 3.4 in Section 3.4. 20Note that our definition of the core is different from those of strong stability in Dutta and Mutuswami (1997) and Jackson and van den Nouweland (2005). Here, we implicitly assume that any coalitional deviation breaks down the present contract completely, while this is not assumed in their environments.

74 The formal deﬁnition of the core is as follows. A coalition S blocks an allocation (G, x) if there exists an allocation (G0, x0) such that

0 (B1) G ∈ G|S, and

0 0 0 0 (B2) Πi(G , x ) ≥ Πi(G, x) for all i ∈ S, and Πi(G , x ) > Πi(G, x) for some i ∈ S.

Here, (B1) means that G0 should be achievable for S without cooperating with others. This implicitly assume that, if S deviates from a contract stipulating (G, x), the contract breaks down immediately. (B2) means that (G0, x0) should be a beneﬁcial deviation for S. An allocation (G, x) is in the core (or, a core allocation) if there does not exist a coalition S which blocks (G, x). Proposition 3.5 provides the necessary conditions for core allocations.

Proposition 3.5. Let n ≥ 2 and let an allocation (G, x) be in the core. Then, (G, x) satisﬁes

(i) G is minimally connected,

(ii) if xi < 1 and |L(i, G)| = 1 for some i ∈ N, then xi > 0 and xj = 1 for all j 6= i, and

(iii) if xi < 1 and |L(i, G)| ≥ 2 for some i ∈ N, then xi ≥ x where x is deﬁned in (3.8)

and xj = 1 for all j 6= i.

Proof. See Appendix C.

The formal proof of Proposition 3.5 appears in Appendix C. Here, we provide the in- tuition. If (G, x) has a redundant link ij in G, breaking ij reduces the link costs of i and j without harming the others. If agents i and j are not connected in (G, x), making a link between i and j with xi = xj = 1 improves i and j without harming other agents. Therefore, a network which is not minimally connected cannot survive in a core allocation. Furthermore, (3.4) ensures that there is at most one agent who does not make the full eﬀort. If the agent i with one link does not make an eﬀort at all, then a coalition excluding i can improve his neighbor without harming the others in the coalition, which implies (ii). To understand (iii), let n = 4 and consider the change of allocation from (G, x) to (G0, x0) which are described in Figure 3.1. If (G, x) and (G0, x0) satisfy

0 0 NB1 ≡ Π1(G , x ) − Π1(G, x) 0 0 0 = v(x1) + v(x2) − c(x1) − [v(x1) + v(1) − c(x1)] + d > 0,

0 0 NB2 ≡ Π2(G , x ) − Π2(G, x) 0 0 0 = v(x1) + v(x2) − c(x2) − [v(x1) + v(1) − c(1)] − d > 0, and

0 0 NBi ≡ Πi(G , x ) − Πi(G, x)

75 0 0 x1 x2 = 1 x1 x2

0 0 x4 = 1 x3 = 1 x4 = 1 x3 = 1

(a) (G, x) (b) (G0, x0)

Figure 3.1: (G, x) and (G0, x0)

0 0 = v(x1) + v(x2) − [v(x1) + v(1)] > 0, (for i = 3, 4)

0 0 every agent prefers (G , x ) to (G, x). Let (G, x) satisfy that x1 < x. Figure 3.2 (a) describes 0 0 the set of (x1, x2) such that NBi = 0 for each i. Note that the assumptions (3.1) to (3.4) ensure that NB1 = 0 and NB2 = 0 have positive slopes and NB2 = 0 has a steeper slope than NB1 = 0. In addition, NBi = 0, i = 3, 4 has a negative slope and passes 0 0 0 0 (x1, x2) = (1, x1) and (x1, x2) = (x1, 1). Furthermore, we can show that NBi = 0, i = 3, 4 0 has an intersection with NB2 = 0 at x2 = x, and that the intersection of NB1 = 0 and 0 0 0 x1 = 1 is lower (in view of x2) than the intersection of NB2 = 0 and x1 = 1. In addition, 0 0 0 0 0 0 0 0 (G , x ) with (x1, x2) above the line NB1 = 0 is preferred by agent 1, (G , x ) with (x1, x2) 0 0 0 0 below the line NB2 = 0 is preferred by agent 2, and (G , x ) with (x1, x2) above the line 0 0 0 0 NBi = 0, i = 3, 4 is preferred by agent i = 3, 4. Therefore, (G , x ) with (x1, x2) in the shaded area is preferred by all agents, which means N blocks (G, x).21 The conditions in Proposition 3.5 are not suﬃcient for an allocation to be in the core. Example 1 and 2 provide counterexamples for n = 2 and n = 3.

Example 1. Let N = {1, 2} and G = {12}. Let an effort profile x = (x1, x2) satisfy x2 = 1 o o o and 0 < x1 < x where x ∈ (0, 1) satisfies v(x ) = c(1) − v(1) + d. Then, (G, x) satisfies the conditions in Proposition 3.5, but S = {2} blocks (G, x) with (G0, x0) = (∅, (0, 0)).

Example 2. Let N = {1, 2, 3}. Since 0 = c(x) − c(1) + d < v(x) − v(1) + d < v(1), there exists xoo ∈ (0, 1) such that v(xoo) = v(x) − v(1) + d. Consider a network G = {12, 23} oo and an eﬀort proﬁle x = (x1, 1, 1) with 0 < x1 < x . Since 0 < v(x1) + v(1) − d < oo ∗ ∗ ∗ v(x ) + v(1) − d = v(x), there exists x such that v(x ) = v(x1) + v(1) − d and 0 < x < x. oo Since 0 = c(1) − c(x) − d < c(1) − v(1) − v(x ) < c(1) − v(1) − v(x1) < c(1) − v(1),

21 0 0 Given (G, x) with x1 > x, Figure 3.2 (b) describes the set of (x1, x2) for which NB2 and NBi, i = 3, 4 0 0 2 are zero. Since there is no (x1, x2) ∈ [0, 1] for which NB2 ≥ 0 and NBi ≥ 0 for i = 3, 4, N cannot block (G, x). This is not enough to say that (G, x) is in the core. To show that (G, x) is in the core, we should consider the all coalitions and all achievable allocations for each coalition.

76 0 0 x2 x2

NB2 = 0 = 0 1 NB2 1 NB1 = 0 x1 x x

x1 NBi, i = 3, 4 NBi = 0, i = 3, 4

x 0 x 0 0 1 1 x1 0 1 1 x1

(a) when x1 < x (b) when x1 > x

0 0 Figure 3.2: (x1, x2) satisfying NBi = 0

∗∗ ∗∗ ∗∗ ∗∗ there exists x such that c(x ) − v(x ) = c(1) − v(1) − v(x1) and 0 < x < 1. Then, ∗∗ ∗∗ oo ∗∗ c(x ) − v(x ) = c(1) − v(1) − v(x1) > c(1) − v(1) − v(x ) = c(x) − v(x), so x > x. Since ∗ ∗∗ 0 ∗ 0 ∗∗ 0 0 0 x < x , there exists x3 such that x < x3 < x . Let G = {23} and x = (0, 1, x3). Then, we can see that S = {2, 3} blocks (G, x) with (G0, x0), though (G, x) satisﬁes the conditions in Proposition 3.5.

We next establish the sufficient conditions for core allocations. Indeed, it turns out that, if there are at least four agents, the conditions in Proposition 3.5 are sufficient for core allocations. To establish the sufficient conditions, we may have to show that no coalition blocks an allocation which satisfies the conditions by considering all coalitions and all allocations achievable for each coalition. Lemma 3.4 helps us avoid this difficulty.

Lemma 3.4. Let S be a coalition with s ≥ 2 and i ∈ S. Let G ∈ G|S. If x ∈ X satisﬁes xi ≤ x¯i for some x¯i ∈ [0, 1], then X Πk(G, x) ≤ s(s − 1)v(1) − (s − 1)c(1) + sv(¯xi) − c(¯xi) − 2(s − 1)d. (3.12) k∈S P Proof. Let X¯ be the set of x such that xi ≤ x¯i. Since X¯ is compact and Πk(G, ·) is P k∈S ∗ ∗ ¯ ∗ continuous in x, there is (G , x ) maximizing k∈S Πk(G, x) on G|S ×X . Trivially, G does not have a redundant link in S. Suppose that G∗ is not connected in S. Let G0 = G∗ + jk ∗ 0 where jk is a link connecting two components in G and xj = 1 for j 6= i. Then, we have a contradiction that X X 0 0 ∗ ∗ ∗ ∗ Πk(G , x ) − Πk(G , x ) ≥ 2v(1) − c(1) − (v(x ) − c(x )) − 2d k∈S k∈S

77 ≥ 2v(1) − c(1) − 2d > 0.

∗ Therefore, G is minimally connected in S. Note that, for any G ∈ G|S which is minimally P ∗ connected in S, i∈S |L(i, G)| = 2(s−1) holds. Given that G is minimally connected in S, P ∗ P since k∈S Πk(G , x) = k∈S(sv(xk) − c(xk)) − 2(s − 1)d and sv(xk) − c(xk) is increasing in xk, we have the result.

Lemma 3.4 provides the upper bound for the aggregate payoﬀ for S when the eﬀort of an agent is bounded. Let S be a subset of agents with s ≥ 2. Lemma 3.4 also implies that, for any (G, x) ∈ G|S × X , X 2 Πk(G, x) ≤ s v(1) − sc(1) − 2(s − 1)d. (3.13) k∈S and, for any (G, x) ∈ G × X , X 2 Πk(G, x) ≤ n v(1) − nc(1) − 2(n − 1)d. (3.14) k∈N

Recall that an allocation (G, x) is efficient if and only if it satisfies (3.14) with equality. Proposition 3.6 states that, if n ≥ 4, the conditions in Proposition 3.5 are sufficient for core allocations.

Proposition 3.6. Let n ≥ 4 and let an allocation (G, x) satisfy followings:

(i) G is minimally connected,

(ii) if xi < 1 and |L(i, G)| = 1 for some i ∈ N, then xi > 0 and xj = 1 for all j 6= i, and

(iii) if xi < 1 and |L(i, G)| ≥ 2 for some i ∈ N, then xi ≥ x where c(x) = c(1) − d and

xj = 1 for all j 6= i.

Then, (G, x) is in the core.

Proof. See Appendix C.

If a coalition S blocks an allocation, then the aggregate payoﬀ for S increases. To understand Proposition 3.6, we consider two kinds of allocations. An allocation (G, x) of the ﬁrst kind is one in which G is minimally connected and there is an agent i satisfying (ii) in Proposition 3.6. For example, let n = 4 and consider an allocation (G, x) where

G = {12, 24, 34}, x1 > 0 and xj = 1 for all j 6= 1. Figure 3.3 (a) describes (G, x). Since the contribution of each agent j 6= 1 to the aggregate payoﬀ via his information is greater

78 x1 > 0 x2 = 1 x1 ≥ x x2 = 1

x4 = 1 x3 = 1 x4 = 1 x3 = 1

(a) (b)

Figure 3.3: Examples of (G, x) for Proposition 3.6 than that via his link cost, excluding agent j 6= 1 from the coalition may not increase their aggregate payoff. Also, agent 1 cannot be excluded. The reason is as follows. Since excluding agent 1 from the coalition decreases agent 3’s payoff, we have to compensate agent 3 for this loss. The only way to compensate agent 3 is to decrease his effort, which decreases agent 4’s payoff. However, it is not possible to compensate agent 4 without reducing the payoff of others. Next, suppose that N blocks (G, x) with (G0, x0). Since increasing agent 0 1’s effort always decreases his payoff, we should have xi ≤ xi for each i. However, Lemma 3.4 implies that the aggregate payoff of N at (G0, x0) cannot be greater than the aggregate payoff at (G, x), which contradicts that N blocks (G, x) with (G0, x0). The second kind of allocations includes allocations (G, x) where G is minimally connected and there is an agent i satisfying in Proposition 3.6. For example, let n = 4 and consider the allocation (G, x) such that G = {12, 14, 34}, x1 ≥ x and x2 = x3 = x4 = 1. Figure 3.3 (b) describes this allocation. From the same arguments as before, agent i 6= 1 cannot be excluded from the coalition. Since each agent i 6= 1 gets benefit from agent 1’s information and they cannot reduce their link cost without agent 1, they cannot increase their aggregate payoff by excluding agent 1. So, a coalition S 6= N cannot block (G, x). 0 0 0 Suppose that N blocks (G, x) with (G , x ). If x1 > x1, then agent 1 should have less links 0 and agent i 6= 1 should have more links in G than in G. However, since x1 ≥ x, it is impossible to compensate agent i for the loss from increasing his links without harming 0 other agents. So, we have xi ≤ xi for each i and Lemma 3.4 implies a contradiction. If n = 2, (iii) in Proposition 3.6 is redundant. In addition, it is easy to show that Proposition 3.6 holds if we replace (ii) with (ii2) for n = 2 and (ii3) for n = 3.

2 o o o (ii ) if xi < 1 and |L(i, G)| = 1 for some i ∈ N, then xi ≥ x where x satisﬁes v(x ) =

c(1) − v(1) + d and xj = 1 for all j 6= i.

3 oo oo (ii ) if xi < 1 and |L(i, G)| = 1 for some i ∈ N, then xi ≥ x where x satisﬁes oo v(x ) = v(x) − v(1) + d and xj = 1 for all j 6= i. Propositions 3.5 and 3.6 fully characterize the core allocations for n ≥ 4. That is, an

79 allocation (G, x) is in the core if and only if it satisfies (i) to (iii) in Proposition 3.6.22 Trivially, the core is not empty. An interesting feature of the core is that its characterization does not depend on v explicitly although the assumptions on v as well as c are crucial in characterizing the core. In addition, the lower bound of effort for an agent who does not make the full effort in the core does not depend on the number of agents as long as n ≥ 4. Another feature is that, when we consider the core as a correspondence of d, it is upper hemi-continuous but not lower hemi-continuous. Let n ≥ 4. Since the core depends on x which converges to 1 as d goes to zero, the core shrinks as d goes to zero and the limit is the set of allocations (G, x) where G is minimally connected and xi < 1 for some i implies |L(i, G)| = 1 and xj = 1 for all j 6= i. However, if d = 0, the core is the set of allocations (G, x) in which G is connected

(not necessarily minimally connected) and xi < 1 for some i implies xj = 1 for all j 6= i. Note that the core contains all efficient allocations. As mentioned in Section 3.3.2, an allocation (G, x) such that G is minimally connected and xi = 1 for all i ∈ N is efficient and Proposition 3.6 implies that it is in the core. Notice that the allocation which maximizes the aggregate payoff is not unique. If n ≥ 4, (ii) in Proposition 3.6 implies that the core is not closed and so it dose not have an allocation which minimizes the aggregate payoff.23 Before leaving this section, we provide Corollary 3.1 which illustrates a relationship between allocations in the core and those in symmetric stationary perfect equilibrium.

Corollary 3.1. Let n ≥ 3 and δ¯n ≤ δ < 1. Suppose that v is concave and c is convex. Core allocations are obtained with probability one in any symmetric stationary perfect equilibrium.

Proof. From Propositions 3.4 and 3.6, eﬃcient allocations are in the core and they are realized with probability one in any symmetric stationary perfect equilibrium. In this way, we ﬁnd our result.24

3.5 Concluding Remarks

This paper considers the situation in which agents make a binding contract both on the amount of local public goods and on the form of networks through which the benefits of public goods are shared. We first consider sequential bargaining games with two different types of proposals and find that, for sufficiently patient agents, sequential bargaining games with pure proposals do not have a symmetric stationary perfect equilibrium, but that sequential bargaining with mixed proposals have it. Next, we characterize the core allocation,

22We can also fully characterize the core allocation for n ≤ 3 by replacing (ii) with (ii2) or (ii3) in Propositions 3.5 and 3.6. 23However, from (ii2) and (ii3), we can see that the core is closed for n ≤ 3. 24If n = 3, condition (ii) in Proposition 3.6 is replaced with (ii3).

80 which consists of a minimally connected network and an effort profile in which at most one agent does not make a full effort. As discussed in Hojman and Szeidl (2008), we frequently observe that many economic and social networks have core-periphery structures. However, in our model, any minimally connected network can survive in a core allocation or in a symmetric stationary perfect equilibrium. Recall that all agents are symmetric in the value and the cost of information and the link cost. In the real world, agents are not symmetric. Some agents can acquire high quality of information with low cost and some agents can contact others more efficiently. In this paper, it is also assumed that there is no decay of information as the number of links through which information passes increases. But, there can be the loss of information if it is conveyed through many people. If we consider these aspects in our model, we may be able to find some specific structures of networks which survive in the core or equilibria. We leave this for future research. We assume that, if there are two connected agents who do not make the socially optimal efforts, increasing their efforts by the same amounts is beneficial to all agents in the coalition. This assumption allows at most one agent whose effort is not socially optimal in a core allocation. One may want to replace this assumption with one that v is concave and c is convex so that the socially optimal effort is achieved in interior. Then, we may have to change the efforts of agents, if their efforts are not socially optimal, by different amounts in order to increase the payoffs of all agents in a coalition. The amounts of efforts we have to change depend on the curvature of v and c, which brings difficulty in characterizing core allocations. We also leave it for future research. Finally, we note that, in Section 3.3.1, the non-existence of symmetric stationary perfect equilibrium comes from the fact that the utility possibility frontier is not connected. To overcome this property, we can also allow agents to transfer side payments instead of introducing mixed proposals. In the sequential bargaining games with side payments, we may have results similar to those in Section 3.3.2: there exists a symmetric stationary perfect equilibrium and any symmetric stationary perfect equilibrium results in an efficient outcome. One difference we conjecture is that the assumption of sufficiently patient agents is not required for the efficiency of symmetric stationary perfect equilibria.

Appendix A: Proofs in Section 3.3.1

In proving Proposition 3.1, Claims 3.1 to 3.3 are useful. Let

nv(1) − c(1) − d (n − 1)v(1) + v(x) − c(1) − 2d δˆ = and δˇ = . nv(1) − c(1) (n − 1)v(1) + v(x) − c(1) − d

It is easy to see that δˆ > δˇ for all n ≥ 4.

81 Claim 3.1. Let N be the set of agents with n ≥ 3 and let δ ≥ δˆ. Let p be the initial proposer in N.

Suppose that an allocation (G, x) satisﬁes xp ≤ xj and |L(p, G)| < |L(j, G)| for some j ∈ N\{p}. Then, (G, x) cannot be sustained by a symmetric stationary perfect equilibrium.

Proof. Suppose that (G, x) described in Claim 3.1 is sustained by a symmetric stationary perfect R equilibrium. By Lemmas 3.1 and 3.2, G is connected and Πj(G, x) ≥ Π¯ (T ) = δΠp(G, x). However, δ ≥ δˆ implies

Πj(G, x) − δΠp(G, x) ≤ (1 − δ)(nv(1) − c(1) − |L(p, G)|d) − d < (1 − δ)(nv(1) − c(1)) − d ≤ 0, which contradicts Πj(G, x) ≥ δΠp(G, x).

Claim 3.2. Let N be the set of agents with n ≥ 4 and p be the initial proposer in N. Let δ > δˇ. Suppose that (G, x) is an allocation satisfying |L(p, G)| + 1 < |L(j, G)| for some j ∈ N\{p}. Then, (G, x) cannot be sustained by symmetric stationary perfect equilibrium.

Proof. Suppose that (G, x) is sustained by a symmetric stationary perfect equilibrium and satisfy

|L(p, G)|+1 < |L(j, G)| for some j ∈ N\{p}. By Lemmas 3.1 and 3.2, G is connected and Πj(G, x) ≥

δΠp(G, x). Furthermore, since 2 ≤ |L(p, G)| + 1 < |L(j, G)|, we have xj ≥ x. If xj = 1 then, since ˆ ˇ δ ≥ δ implies δ > δ, the result follows from Claim 3.1. If xj < 1, xk = 1 for all k 6= j. Then, since δ ≥ δˆ, we have

Πj(G, x) − δΠp(G, x) ≤ (n − 1)v(1) + v(x) − c(x) − |L(j, G)|d − δ ((n − 1)v(1) + v(x) − c(1) − |L(p, G)|d) ≤ (n − 1)v(1) + v(x) − c(x) − (|L(p, G)| + 2)d − δ((n − 1)v(1) + v(x) − c(1) − |L(p, G)|d) ≤ (n − 1)v(1) + v(x) − c(1) − 2d − δ [(n − 1)v(1) + v(x) − c(1) − d] < 0, which is a contradiction.

Claim 3.3. Let N be the set of agents with n ≥ 3 and p be the initial proposer in N. Suppose that an allocation (G, x) satisﬁes |L(p, G)| > |L(j, G)| for all j 6= p. Then, for any δ ∈ (0, 1), (G, x) cannot be sustained by a symmetric stationary perfect equilibrium.

Proof. Suppose that (G, x) satisﬁes |L(p, G)| > |L(j, G)| for all j 6= p and is sustained by a symmetric stationary perfect equilibrium.By Lemmas 3.1 and 3.2, G is connected and Πj(G, x) ≥ δΠp(G, x).

Furthermore, since |L(p, G)| > |L(j, G)| for all j 6= p, we have |L(p, G)| ≥ 2 and xp ≥ x. If xp = 1, then we have δΠp(G, x) < Πp(G, x) < Πj(G, x) for all j 6= p, which contradicts Lemma 3.2 (d).

Therefore, we get x ≤ xp < 1 and, by Lemma 3.2, xj = 1 for all j ∈ N\{p}. Let j0 be an agent such that |L(j0,G)| ≥ |L(j, G)| for all j ∈ N\{p}. Then, by Lemma 3.2, we have Πj(G, x) ≥ Πj0 (G, x) = δΠp(G, x) for all j 6= p. However, since x ≤ xp < 1, we have

Πj0 (G, x) − δΠp(G, x)

82 ≥ (t − 1)v(1) + v(x) − c(1) − |L(j0,G)|d − δ[(t − 1)v(1) + v(x) − c(x) − |L(p, G)|d] ≥ (1 − δ)[(t − 1)v(1) + v(x) − c(1) − |L(p, G)|d] > 0.

This contradiction completes the proof.

Now, we are ready to prove Proposition 3.1.

Proof of Proposition 3.1. Suppose that an allocation (G, x) is sustained by a symmetric stationary perfect equilibrium. Let an agent p be the initial proposer in N. Since Lemmas 3.1 and 3.2 imply that G is connected, it is enough to consider the following four cases. Suppose that |L(p, G)| ≥ 2 and that |L(i, G)| > |L(j, G)| for some i, j 6= p. Lemma 3.2 (h) implies |L(k, G)| = 1 for all k 6= i, p. Furthermore, since |L(i, G)| > |L(j, G)| for some i, j ∈ S\{p} and |L(p, G)| ≥ 2, Lemma 3.2 (b) and (e) imply xi < 1 and xp = 1. By Lemma 3.2 (f), we have |L(p, G)| ≤ |L(j, G)| for all j 6= p, which contradicts |L(p, G)| ≥ 2 and |L(k, G)| = 1 for all k 6= i, p. Suppose that |L(p, G)| ≥ 2 and |L(i, G)| = |L(j, G)| for all i, j 6= p. By Lemma 3.2 (g), we have |L(k, G)| = 1 for all k ∈ S\{p}. Since |L(p, G)| > |L(k, G)| = 1 for all k ∈ S\{p}, Claim 3.3 implies (G, x) cannot be sustained by symmetric stationary perfect equilibrium. Suppose that |L(p, G)| = 1 and that |L(i, G)| ≥ 2 and |L(j, G)| ≥ 2 for some i, j 6= p. From

Lemma 3.2 (b), we know that xi = 1 or xj = 1. Without loss of generality, let xj = 1. Since xp ≤ xj and |L(p, G)| < |L(j, G)|, a contradiction comes from Claim 3.1. Suppose that |L(p, G)| = 1 and that |L(i, G)| ≥ 2 for some i 6= p and |L(j, G)| = 1 for all j 6= p, i. Since G is connected, |L(i, G)| ≥ (n − 1) ≥ |L(p, G)| + 2. Since δ ≥ δˆ implies δ > δˇ, Claim 3.2 implies a contradiction.

Appendix B: Proofs in Section 3.3.2

In proving Lemma 3.3 and Propositions 3.3 and 3.4, Claims 3.4 is useful.

Claim 3.4. Let T be the set of remaining agents with t ≥ 2 and Π¯ R(T ) ≥ 0. Let (S, α ) satisfy R S the constraints of Problem (M) and Π (G, x)dα ≥ Π¯ R(T ). Then, there exists α0 ∈ ∆(A ) such R R R p S T T 0 0 ¯ R 0 that Πp(G, x)dαT ≥ Πp(G, x)dαS, Πk(G, x)dαT ≥ Π (T ) for all k ∈ T \{p}, and supp(αT ) ⊂ {(G, x) ∈ G × X | G is connected in T }. In addition, if S 6= T , there exists α0 ∈ ∆(A ) such R R R T T 0 0 ¯ R 0 that Πp(G, x)dαT > Πp(G, x)dαS, Πk(G, x)dαT ≥ Π (T ) for all k ∈ T \{p}, and supp(αT ) ⊂ {(G, x) ∈ G × X | G is connected in T }.

Proof. The proof consists of two steps. R R Step 1. If S 6= T, there exists a distribution α ∈ ∆(A ) such that Π (G, x)dα > Π (G, x)dα R T T p T p S R and Πk(G, x)dαT ≥ Π¯ (T ) for all k ∈ T \{p}.

First, we deﬁne a function f : G × X → G × X . For each (G, x) ∈ AS, f is deﬁned by 0 0 0 0 0 0 f(G, x) = (G , x ) where G = G ∪ {pj | j ∈ T \S}, xp = 1, xk = xk for k ∈ S\{p}, and xk = 1 for k ∈ T \S. For each (G, x) ∈/ AS, let f(G, x) = (G, x). Then, f is measurable on G × X and, for each

(G, x) ∈ AS,Πp(f(G, x)) > Πp(G, x), Πj(f(G, x)) > Πp(G, x), and Πk(f(G, x)) ≥ Πk(G, x) for all k ∈ S\{p}.

83 Deﬁne α ∈ ∆(A ) to satisfy α (A) = α (f −1(T )) for each A ⊂ G × X which is measurable. T T R T RS S R Then, for each k ∈ T \S, Π (G0, x0)dα = Π (f(G, x))dα > Π (G, x)dα ≥ Π¯ R(T ) and, for R j R T j R S p S each k ∈ S\{p}, Π (G0, x0)dα = Π (f(G, x))dα ≥ Π (G, x)dα ≥ Π¯ R(T ). Furthermore, R k R T k R S k S 0 0 since Πp(G , x )dαT = Πp(f(G, x))dαS > Πp(G, x)dαS, we completes Step 1. R R 0 0 Step 2. For any αT ∈ ∆(AT ), there exists αT ∈ ∆(AT ) such that Πk(G, x)dαT ≥ Πk(G, x)dαT 0 for all k ∈ T , and supp(αN ) ⊂ {(G, x) ∈ G × X | G is connected in T }.

Deﬁne a function f : G × X → G × X as follows: If (G, x) ∈ AT and G is not connected in T , let 0 0 0 0 0 0 0 0 f(G, x) = (G , x ) where (G , x ) ∈ AT , Πk(G , x ) ≥ Πk(G, x) for all k ∈ T, and (G , x ) is connected in T . From Section 3.4, we can see that such (G0, x0) exists. Otherwise, let f(G, x) = (G, x). 0 0 −1 Let αT be a distribution on G × X deﬁned by αT (A) = αT (f (A)) for all A ⊂ G × X which is measurable. Then, we have supp(α0 ) ⊂ {(G, x) ∈ G × X | G is connected in T } and R R R N 0 0 0 Πk(G , x )dαT = Πk(f(G, x))dαT ≥ Πk(G, x)dαT for each k ∈ T . This completes Step 2. Then, the result follows from Steps 1 and 2.

Note that Claim 3.4 implies that (S, αS) with S 6= T cannot be a solution of Problem (M).

Proof of Lemma 3.3. We prove Lemma 3.3 for each case.

Case 1 (t = 2 and 0 < δ < 1). Let T = {p, j} and suppose that there exists (S, α ) which R R S satisﬁes all constraints in Problem (M) and Π (G, x)dα > Π (G, x)dα∗ . By Claim 3.4, there R p R S p R T 0 0 ∗ 0 ¯ R exists αT ∈ ∆(AT ) such that Πp(G, x)dαT > Πp(G, x)dαT , Πj(G, x)dαT ≥ Π (T ), and (G, x) ∈ supp(α0 ) implies G is connected in T , that is G = G∗. Since v is concave and c is convex, T R 0 ∗ Jensen’s inequality and Πp(G, x)dαT > Πp(G, x)dαT imply µZ ¶ µZ ¶ µZ ¶ 0 0 0 ∗ ∗ v xjdαT + v xpdαT − c xpdαT − d > v(1) + v(xp) − c(xp) − d.

R R P R Since x dα0 ≤ 1, we should have x dα0 < x∗. However, since Π (G, x)dα0 > R j T p T p k∈T k T P ∗ k∈T Πk(G, x)dαT , Jensen’s inequality implies µZ ¶ µZ ¶ µZ ¶ µZ ¶ 0 0 0 0 2v xjdαT − c xjdαT + 2v xpdαT − c xpdαT − 2d

∗ ∗ > 2v(1) − c(1) + 2v(xp) − c(xp) − d, R 0 ∗ and so we have xpdαT > xp, This contradiction proves Case 1.

Case 2 (t ≥ 3 and δ¯ ≤ δ < 1). Suppose that there exists (S, α ) which satisﬁes all constraints Rt R S R in Problem (M) and Π (G, x)dα > Π (G, x)dα∗ . Since Π¯ R(T ) = Π (G, x)dα∗ ≥ 0 for p S p T k R T each k ∈ T \{p}, Claim 3.4 implies that there exists α0 ∈ ∆(A ) such that Π (G, x)dα0 > R R R T T p T ∗ 0 ∗ Πp(G, x)dαT and Πk(G, x)dαT ≥ Πk(G, x)dαT for each k ∈ T \{p}. Then, Z X X Z 0 0 Πk(G, x)dαT = Πk(G, x)dαT k∈T k∈T

84 X Z ∗ 2 > Πk(G, x)dαT = t v(1) − tc(1) − 2(t − 1)d, k∈T

0 0 0 P 0 0 which means that there exists an allocation (G , x ) ∈ supp(αT ) such that k∈T Πk(G , x ) > t2v(1) − tc(1) − 2(t − 1)d. This contradicts (3.13).

Case 3 (t ≥ 3 and δ < δ < δ¯ ). Suppose that there exists (S, α ) which satisﬁes all constraints R t t R S in Problem (M) and Π (G, x)dα > Π (G, x)dα∗ . By Claim 3.4, there exists α0 ∈ ∆(A ) such R Rp S R p T R T T 0 ∗ 0 ∗ that Πp(G, x)dαT > Πp(G, x)dαT , Πj(G, x)dαT ≥ Πj(G, x)dαT for each j ∈ T \{p}, and (G, x) ∈ supp(α0 ) implies G is connected in T . So, for all (G, x) ∈ supp(α0 ), |L(p, G)| ≥ 1 and T R R T P 0 ∗ k∈T |L(k, G)| ≥ 2(t − 1). Then, Jensen’s inequality and Πp(G, x)dαT > Πp(G, x)dαT imply X µZ ¶ µZ ¶ µZ ¶ Z 0 0 0 0 v xkdαT + v xpdαT − c xpdαT − (|L(p, G)|d)dαT k∈T \{p} X ∗ ∗ > v(1) + v(xp) − c(xp) − d. k∈T \{p} R R P R Since (|L(p, G)|d)dα0 ≥ d, we have x dα < x∗. However, since Π (G, x)dα0 > R T p T p k∈T k T P ∗ k∈T Πk(G, x)dαT , we have X µ µZ ¶ µZ ¶¶ 0 0 tv xkdαT − c xkdαT k∈T \{p} Ã ! µ µZ ¶ µZ ¶¶ Z X 0 0 0 + tv xpdαT − c xpdαT − |L (k, G) |d dαT k∈T X ¡ ¡ ∗¢ ¡ ∗¢¢ > (tv(1) − c(1)) + tv xp − c xp − 2 (t − 1) d. k∈T \{p} R R P 0 0 ∗ Then, ( k∈T |L(k, G)|d)dαT ≥ 2(t − 1)d implies that xpdαT > xp. This is a contradiction. R Case 4 (t ≥ 3 and 0 < δ ≤ δ ). Note that Π (G, x)dα∗ = (t − 1)v(1) − d and it is the t R p R T ∗ maximum payoﬀ for agent p on AT . Thus, Πp(G, x)dαS ≤ Πp(G, x)dαT for any (S, αS).

Proof of Proposition 3.3. Letσ ¯ be a symmetric stationary perfect equilibrium and let Π¯ P (N) be an equilibrium payoff to the initial proposer p. From same arguments as in Lemma 3.1,σ ¯ satisfies R P no-delay property. Then, by Claim 3.4, Π¯ (N) = Πp(G, x)dαN holds, where (N, αN ) is the initial proposal of p. R R R Suppose that δ¯ ≤ δ < 1. If Π (G, x)dα > Π (G, x)dα∗ holds, then Π (G, x)dα ≥ R n R p N p N k N δ Π (G, x)dα > δ Π (G, x)dα∗ for all k 6= p. So, (N, α ) is not a solution of the problem with p R N p N N Π¯ R(T ) = δ Π (G, x)dα∗ , which contradicts Lemma 3.3. In the equilibrium, any proposal (N, α0 ) p N R R R N such that, for each k 6= p, Π (G, x)dα0 ≥ δ Π (G, x)dα will be accepted. If Π (G, x)dα < R R k N R p N R p N Π (G, x)dα∗ , then Π (G, x)dα∗ = δ Π (G, x)dα∗ ≥ δ Π (G, x)dα for each k 6= p. So, p p N k N p N p N R can improve his payoff by offering (N, α∗ ), which is a contradiction. Therefore, Π (G, x)dα = R N p N ∗ Πp(G, x)dαN .

85 R R ∗ If Πk(G, x)dαN > δ Πp(G, x)dαN for some k ∈ N\{p}, then Z X X Z Πk(G, x)dαN = Πk(G, x)dαN k∈N k∈N X Z ∗ > Πk(G, x)dαN k∈N = n2v(1) − nc(1) − 2(n − 1)d, R R ∗ which contradicts (3.14). Therefore, Πk(G, x)dαN = δ Πp(G, x)dαN for all k 6= p. Suppose that δ < δ < δ¯ . From the same arguments as for δ¯ ≤ δ < 1, we have Π (G, x)dα = R n n R R n p N Π (G, x)dα∗ . Suppose that Π (G, x)dα > δ Π (G, x)dα∗ for some k 6= p. By Step 2 in p N k N R p N R 0 P 0 P ∗ Claim 3.4, there is αN ∈ ∆(AN ) such that k∈N Πk(G, x)dαN > k∈N Πk(G, x)dαN and supp(α0 ) ⊂ {(G, x) ∈ G × X | G is connected}. Since v is concave and c is convex, the same N R R arguments in the proof of Lemma 3.3 leads us a contradiction that x dα0 > x∗ and x dα0 ≤ x∗. R R p N p p N p ∗ Therefore, Πk(G, x)dαN = δ Πp(G, x)dαN for all k 6= p. Suppose that 0 < δ < δ . The same arguments as for δ¯ ≤ δ < 1 result in Π (G, x)dα = R n n p N ∗ Πp(G, x)dαN . ¯ Proof of Proposition 3.4. Let δn ≤ δ < 1 andσ ¯ be a symmetric stationary perfect equilibrium. From Lemma 3.1, we know that the initial proposer p oﬀers (N, α¯ ) and all other agents accept this N P oﬀer. Since (3.11), (3.14), and Proposition 3.3 imply that, for any (G, x) ∈ G × X , Π (G, x) ≤ R P i∈N i i∈N Πi(G, x)dα¯N , we have the result.

Appendix C: Proofs in Section 3.4

Proof of Proposition 3.5. Suppose that (G, x) is in the core. If H and H0 with H 6= H0 are com- 0 0 0 0 0 ponents in G, let G = G + ij where i ∈ H and j ∈ H and xi = xj = 1 and xk = xk for k 6= i, j. 0 0 0 0 0 0 Then, we can see that Πi(G , x ) > Πi(G, x), Πj(G , x ) > Πj(G, x), and Πk(G , x ) ≥ Πk(G, x) for all k 6= i, j. Thus, G is connected. If G−ij is connected for some ij ∈ G, then Πi(G−ij, x)−Πi(G, x) =

Πj(G−ij, x)−Πj(G, x) = d > 0, and Πk(G−ij, x) = Πk(G, x) for all k 6= i, j. Hence, G is minimally 0 0 connected. If xi < 1 and xj < 1 for some i, j ∈ N with i 6= j, let xi = xi + ε, xj = xj + ε for 0 0 suﬃciently ε, and xk = xk for all k 6= i, j. Then, (3.4) implies Πk(G, x ) > Πk(G, x) for all k ∈ N, which contradicts that (G, x) is in the core. Therefore, xi < 1 for some i implies xj = 1 for all j 6= i. 0 0 Suppose that xi = 0 and L(i, G) = {ij} for some i ∈ N. Let G = G − ij and x = x. Then, 0 0 0 0 Πj(G , x )−Πj(G, x) = d > 0 and Πk(G , x )−Πk(G, x) = 0 for all k 6= i, j. So, N\{i} blocks (G, x).

Suppose that there exists an agent i such that xi < x and |L(i, G)| ≥ 2. Let ij, ik ∈ L(i, G). Since G is minimally connected, jk∈ / G. Let G0 = G−ij +jk. Then, G0 is also minimally connected and, from the previous arguments, we know xj = xk = 1. Since v(·) is continuous and strictly increasing and 0 < v(1) − d < v(1), there existsx ˆ ∈ (0, 1) such that

v(ˆx) = v(1) − d.

Then, since c(ˆx) − v(ˆx) < c(1) − v(1) = c(x) − v(ˆx), we havex ˆ < x. We will consider two cases,

86 xi < xˆ andx ˆ ≤ xi < x. First, consider the case of xi < xˆ. Since v(xi) < v(xi) + d < v(ˆx) + d = v(1), there exists x∗ such that ∗ ∗ xi < x < 1 and v(x ) = v(xi) + d.

∗∗ If c(1) − v(1) > c(xi) − v(xi) + d, then there exists x ∈ (0, 1) such that

∗∗ ∗∗ c(x ) − v(x ) = c(xi) − v(xi) + d.

∗∗ ∗∗ ∗∗ ∗∗ Since c(x ) − v(x ) > c(xi) − v(xi), we have x > xi. In addition, since v(x ) − v(xi) > ∗∗ ∗∗ ∗ ∗ ∗∗ c(x ) − v(x ) − (c(xi) − v(xi)) = v(x ) − v(xi), we have x < x . If c(1) − v(1) ≤ c(xi) − v(xi) + d, ∗∗ ∗ ∗∗ 0 let x = 1. Since x < x whichever c(1)−v(1) > c(xi)−v(xi)+d or not, there exists xi such that ∗ 0 ∗∗ 0 0 0 x < xi < x . Let xl = xl for all l 6= i. Then, it is not diﬃcult to see that Πk(G , x ) > Πk(G, x) for all k ∈ N. Therefore, N blocks (G, x).

Next, consider the case ofx ˆ ≤ xi < x. Since c(x) − v(x) < c(1) − d − v(xi) < c(1) − v(1), there exists x∗∗ such that

∗∗ ∗∗ ∗∗ x < x < 1 and c(x ) − v(x ) = c(1) − d − v(xi).

∗∗ ∗ Furthermore, since v(x) < v(xi) − c(xi) + c(1) − d < v(x ), there exists x such that

∗ ∗∗ ∗ x < x < x and v(x ) = v(xi) − c(xi) + c(1) − d.

∗ ∗∗ 0 ∗ 0 ∗∗ 0 Then, since x < x , there exists xk such that x < xk < x . Let xl = 1 for all l 6= k. Then, it 0 0 is not diﬃcult to see that Πk(G , x ) > Πk(G, x) for all k ∈ N. Therefore, N blocks (G, x). This contradiction completes the proof.

0 0 Proof of Proposition 3.6. Suppose that Πi(G, x) > 0 for all i ∈ N and S blocks (G, x) with (G , x ). 0 0 Since Πi(∅, x ) ≤ 0 for all x ∈ X , S cannot be a singleton. Furthermore, if S with s ≥ 2 blocks 0 0 0 (G, x), we can always ﬁnd (G , x ) ∈ GS × X such that G is minimally connected in S and S blocks (G, x) with (G0, x0). In the proof, if we say that S blocks (G, x) with (G0, x0), we let G0 be a minimally connected network in S. The proof of Proposition 3.6 consists of two steps. o Step 1. Suppose that a network G ∈ G is minimally connected and that xio > 0 for some i ∈ N o o with |L(i ,G)| = 1 and xi = 1 for all i 6= i . Then (G, x) is in the core. Let (G, x) satisfy the conditions in Step 1 and let io be an agent described in Step 1. Suppose 0 0 that S blocks (G, x) with (G , x ). Since Πi(G, x) > 0 for all i ∈ N, we know that s ≥ 2. Suppose in addition that io ∈ S. Since

0 0 0 0 0 ≤ Πio (G , x ) − Πio (G, x) ≤ v(xio ) − c(xio ) − (v(xio ) − c(xio )),

0 we have xio ≤ xio . Then, by Lemma 3.4, X 0 0 Πk(G , x ) ≤ s(s − 1)v(1) + sv(xio ) − (s − 1)c(1) − c(xio ) − 2(s − 1)d k∈S

≤ s(n − 1)v(1) + sv(xio ) − (s − 1)c(1) − c(xio ) − 2(n − 1)d

87 X ≤ Πk(G, x) k∈S which is a contradiction. Therefore, S with io ∈ S does not block (G, x). Suppose that io ∈/ S. If there is an agent j∈ / S with j 6= io, then X 0 0 Πk(G , x ) < s(n − 1)v(1) − sc(1) − 2(n − 2)d k∈S ≤ s(n − 1)v(1) − sc(1) − [2(n − 1) − |L(io,G)| − |L(j, G)|]d X ≤ Πk(G, x), k∈S which is a contradiction. Thus, S = N\{io}. Let jo be an agent such that iojo ∈ G. Suppose o 0 o 0 P 0 that |L(j ,G )| < |L(j ,G)|. Since G and G are minimally connected in S, o |L(k, G )| ≥ P k∈S\{j } k∈S\{jo} |L(k, G)| is satisﬁed. Since s ≥ 3, simple computation leads us to a contradiction that P 0 0 P o 0 o k∈S\{jo} Πk(G , x ) < k∈S\{jo} Πk(G, x). Thus, we have |L(j ,G )| ≥ |L(j ,G)|, which implies

0 0 0 0 0 ≤ Πjo (G , x ) − Πjo (G, x) < v(xjo ) − c(xjo ) − (v(1) − c(1)). (3.15)

Furthermore, since G is minimally connected, there exists an agent i ∈ S such that |L(i, G)| = 1. Since |L(i, G0)| ≥ 1, we have

0 0 0 0 0 ≤ Πi(G , x ) − Πi(G, x) < v(xi) − c(xi) − (v(1) − c(1)). (3.16)

0 0 0 0 0 0 From (3.15) and (3.16), we have xjo < 1 and xi < 1. Since xjo ≤ xi implies Πi(G , x ) < Πi(G, x) and 0 0 0 0 o xi < xjo implies Πjo (G , x ) < Πjo (G, x), we have a contradiction to j ∈ S and i ∈ S. Therefore, S with io ∈/ S cannot block (G, x). This proves Step 1. o Step 2. Suppose that a network G ∈ G is minimally connected and that xio ≥ x for some i ∈ N o o with |L(i ,G)| ≥ 2 and xi = 1 for all i 6= i . Then (G, x) is in the core. Let (G, x) satisfy the conditions in Step 2 and io be an agent described in Step 2. Let S block 0 0 (G, x) with (G , x ). Since Πi(G, x) > 0 for all i ∈ N, we know that s ≥ 2. Suppose that io ∈/ S. Since |L(io,G)| ≥ 2, by Lemma 3.4, we have X 0 0 2 Πk(G , x ) ≤ s v(1) + sc(1) − 2(s − 1)d k∈S ≤ s(n − 1)v(1) + sc(1) − (2(n − 1) − |L(io,G)|) d X ≤ Πk(G, x), k∈S which contradicts that S blocks (G, x). Suppose that io ∈ S. If there is an agent j∈ / S with j 6= io, then we have a contradiction that X 0 0 2 Πk(G , x ) ≤ s v(1) − sc(1) − 2(s − 1)d k∈S < s(n − 1)v(1) − sc(1) − (2(n − 1) − |L(io,G)| − |L(j, G)|)d

88 X ≤ Πk(G, x). k∈S P P Here, the last inequality comes from k∈S |L(k, G)| ≤ k∈N\{io,j} |L(k, G)|, and so we have S = N. 0 0 0 0 Suppose in addition that xio > xio . Since Πio (G , x ) > Πio (G, x) and G and G are minimally connected, io has less links in G0 than in G and there is an agent j ∈ S such that |L(j, G0)| > |L(j, G)|. For this agent j, the following holds:

0 0 0 ≤ Πj(G , x ) − Πj(G, x) 0 0 0 ≤ v(xio ) + v(xj) − c(xj) − (v(xio ) + v(1) − c(1))) − d (3.17) 0 0 < v(1) + v(xj) − c(xj) − (v(x) + v(1) − c(1))) − d 0 0 = v(xj) − c(xj) − (v(x) − c(x)) .

Furthermore, since G is minimally connected and n ≥ 4, there exists an agent j0 ∈ S such that j0 6= j and |L(j0,G)| = 1. Note that G0 is minimally connected, so |L(j0,G0)| ≥ 1 = |L(j0,G)|. Then, we have 0 0 0 0 0 ≤ Πj0 (G , x ) − Πj0 (G, x) < v(xj0 ) − c(xj0 ) − (v(1) − c(1)). (3.18)

0 0 0 0 0 0 From (3.17) and (3.18), we have xj < x < 1 and xj0 < 1. If xj0 ≤ xj,(3.4) implies Πj(G , x ) < 0 0 0 0 Πj(G, x) which contradicts j ∈ S. If xj < xj0 ,(3.4) implies Πj0 (G , x ) < Πj0 (G, x) which contradicts 0 0 j ∈ S. Thus, we have xio ≤ xio . However, Lemma 3.4 and S = N imply a contradiction that P 0 0 P o k∈S Πk(G , x ) ≤ k∈S Πk(G, x). Therefore, S with i ∈ S cannot block (G, x). This proves Step 2 and completes the proof of Proposition 3.6.

89 Chapter 4

Eﬃcient Structure of Organization with Heterogeneous Workers

4.1 Introduction

Hierarchy is a typical form of organizations, and there are two main approaches to hierarchical organizations in economic theory: the incentives approach and the computational approach. The first approach focuses on internal compensation schemes that align the incentives of owners and managers. The need for monitoring arises, for example in Holmstrom (1982), to alleviate the moral hazard problem in producing joint outputs. This approach is adopted in Melumad et al. (1995), McAfee and McMillan (1995), and Mookherjee and Reichelstein (2001), which study the possibility of implementing second-best allocations (incentive constrained allocations) in the hierarchy.1 The second approach, older in origin, considers design of an organization given exogenous constraints. Recently, Radner (1993) gave a fillip to this approach and Prat (1997), Van Zandt (2004), and Schulte and Grüner (2007) adopt it to investigate efficient structure of organizations. In those studies, an organization performs information processing rather than operates with workers who have an incentive to shirk without being monitored. The approach taken in this paper uses one insight from the incentives approach – everyone is required to be monitored to prevent shirking – but focuses on the computational approach to investigate efficiency for organizations in which workers are required to be monitored to prevent shirking. An organization in our interests has the following features. The organization hires workers to achieve its goal. For example, a business firm maximizes its profit by selling its product, the Gallup collects public opinions, and the police wants to make its area

1McAfee and McMillan (1995) provide an example in which the hierarchy cannot achieve the second-best outcome. Melumad et al. (1995) and Mookherjee and Reichelstein (2001) show that, if contracts occur sequentially from top to down, the second-best outcome can be obtained in the hierarchy.

90 safe by patrolling the area. Workers differ in their abilities, which is measured by the number of tasks he can deal with. In each task, a worker either supervises his subordinate (supervisory task) or works to achieve the goal of the organization (productive task). That is, an employee in a business firm sells a product, a worker in the Gallup contacts people, and a policeman patrols a district if they have remaining resources (e.g., time) after supervising their subordinates. Since each worker except the president should have a supervisor, the organization has a hierarchical structure which can be represented as a rooted tree. The value of an organization is determined by the benefit from its performance and the wage cost, which are exogenously given. More precisely, the benefit depends on the total number of productive tasks carried out and the length of chain of supervision in the hierarchy. Having the benefit depend on the length of chain of supervision, we try to capture the efficiency loss such as communication cost, loss of information, and distortion of command. In addition, we consider a wage scheme which depends on workers’ hierarchical positions not on their abilities. Although a widespread belief of economists is that wage is determined by productivity, we frequently observe that many organizations, such as govern- ments and armed forces, pay wages based on hierarchical position rather than productivity. Furthermore, it will turn out that, in an organization of efficient structure, abler workers are assigned to the higher positions and so paid more than less able workers. Since a usual way to compensate abler workers is to pay a higher wage by appointing them to higher positions in the hierarchy, the wage scheme based on hierarchical levels can be compatible with the belief of economists.2 The main aim of this paper is to show how tasks can be optimally allocated among workers to generate endogenous hierarchies. We first consider workers in an organization to be fixed and construct an efficient structure of hierarchy, which maximizes the value of organization. The efficient hierarchy we construct has the following features: (i) abler workers are assigned to hierarchical positions no lower than less able workers, (ii) if worker i is abler than worker j, then i’s manager (or, subordinates) is also abler than j’s manager (or, subordinates), and (iii) all productive tasks are assigned to workers with lower abilities. After considering an organization with a fixed set of workers, we explore the optimal size of organization and find that the hierarchy with the optimal size should be balanced in the sense that workers performing productive tasks have the same distance from the head of the organization. Finally, we show that the structure we construct is the unique (up to equivalent class) one which is efficient and satisfies an axiom, called lower ability consistency.

2An experimental result in Mahoney (1979) indicates that one’s hierarchical level has a greater influence on his wage than does the number of his direct subordinates. The results in Medoff and Abraham (1981) reveal that, in a company, there are experience-wage differentials among the workers and productivity plays a significantly smaller role in explaining these differentials.

91 Lower ability consistency requires that, if an organization additionally hires workers who are less able than current workers, it does not change the previous hierarchical relationship among the current workers. Efficient structure of organization is explored in many previous studies. Williamson (1967), Calvo and Wellisz (1978), and, Qian (1994) consider the hierarchical models of moral hazard and provide many interesting features for efficient structure of organizations. Although those studies share some results with our model (e.g., abler workers being in higher hierarchical positions), there are some differences between those studies and our paper. For instance, they assume that the productive tasks are done by workers in the lowest level and that all other workers use their efforts to detect shirking of their subordinates, which we obtain as a result. In addition, they do not allow workers with different abilities in a hierarchical level, although we occasionally see that workers in a hierarchical level have different abilities. Furthermore, since those studies consider a hierarchy as multiple layers rather than a rooted tree, they do not explore individual hierarchical relationships (e.g., who are the manager and subordinates of a worker). There are other papers which share our interpretation of an organization in that the hierarchy can be represented as a rooted tree. For example, Radner (1993), Bolton and Dewatripont (1994), Prat (1997), Van Zandt (2004), and Schulte and Grüner (2007) study efficiency for information processing hierarchies. In these papers, it takes time for an agent to process information in order to make a decision, and the delay of decision brings ineffi- ciency.3 Although they share the interpretation of hierarchy with this paper, their models have some aspects different from this paper. Radner (1993) and Schulte and Grüner (2007) have identical agents and Bolton and Dewatripont (1994) do not take wage costs into account in their analysis. In Prat (1997), wages depend on workers’ abilities, not on hierarchical levels and, in Van Zandt (2004), wages are constant.4 The remainder of this paper is organized as follows. Section 4.2 explains the model to analyze the hierarchy. In Section 4.3, we construct an efficient structure of an organization. In Section 4.4, we discuss the optimal size of organization and, in Section 4.5, we characterize a structure rule which is efficient and satisfies lower ability consistency. We conclude in Section 4.6. 3The delay in decision-making is introduced by Keren and Levhari (1979), which investigates the optimal structure of organization. In Keren and Levhari (1979), a structure of organization is characterized by the number of workers in each rank, not by a rooted tree. 4All those papers including this paper consider the need to monitor workers and the desire to parallelize information processing as an explanation for the popularity of hierarchy as a form of organizations. Another explanation for the popularity of hierarchy is the desire to economize on skilled or knowledgeable workers, which is discussed in Beggs (2001) and Garicano and Rossi-Hansberg (2006).

92 4.2 Model

An organization hires n ≥ 2 workers and wants to arrange them in a hierarchy. Each worker i is characterized by his ability ai ∈ N, which represents the number of tasks he can manage.5 In each task, he controls his subordinate or does a productive task which yields a beneﬁt to the organization. Let a = (a1, . . . , an) be an ability vector. Without loss of generality, we let a1 ≥ ... ≥ an ≥ 2. Given an ability vector a = (a1, . . . , an), we denote by 6 Na the set of workers corresponding to a. That is, Na = {1, . . . , n}.

A directed graph G on Na is a subset of Na × Na such that (i, j) ∈ G implies i 6= j. For directed graph G, we write ij for an element (i, j) ∈ G and call a link. Since G is directed graph, ij ∈ G is not equivalent to ji ∈ G. For convenience, we let G + ij = G ∪ {ij} for ij∈ / G and G − ij = G\{ij} for ij ∈ G.

Given an ability vector a, a structure Ga of organization is a directed graph on Na satisfying the following:

(O1) there exists a unique h ∈ Na,

(O2) |{j ∈ Na : ji ∈ Ga}| = 1 for each i ∈ Na\{h},

(O3) for each i ∈ Na, |{j ∈ Na : ij ∈ Ga}| ≤ ai, and

(O4) there does not exist a sequence of workers i1, . . . , im such that ilil+1 ∈ Ga for all 7 l = 1, . . . , m − 1 and imi1 ∈ Ga.

We call such a worker h ∈ Na in (O1) a head in Ga and write h(Ga). If ij ∈ Ga, we say that a worker i is a manager of j and that j is a subordinate of i in Ga. We denote 8 p(j; Ga) as the manager of j and s(i; Ga) as the set of subordinates of i in Ga. That is, if ij ∈ Ga, i = p(j; Ga) and j ∈ s(i; Ga) where s(i; Ga) = {k ∈ Na : ik ∈ Ga}. 0 Furthermore, for each i ∈ Na, we deﬁne r-th order manager of i in Ga as p (i; Ga) = i and r r−1 p (i; Ga) = p(p (i; Ga); Ga) for each r ≥ 1.

For i, j ∈ Na with i 6= j, if there is a subset {i1i2, . . . , il−1il} of Ga such that i1 = i, il = j, we call the subset as a path from i to j. Note that (O1) to (O4) imply that, for each i, j ∈ Na, the path from i to j is uniquely determined. If there is a path from i to j in Ga,

5Some previous studies such as Beckmann (1988), Qian (1994), and Prat (1997) refer to ability as a span of control or a capacity. 6We first explore the efficient structure of organization with workers and their abilities being fixed. Then, in Section 4.4, we allow the organization to choose the set of workers. Next, in Section 4.5, we explore structure rules which assign an ability vector to a structure of organization. We use the notation Na, rather than N, for the analysis in Sections 4.4 and 4.5. 7 Note that a directed graph G is an irreflexive binary relation on Na. If we consider a structure Ga of organization as a irreflexive binary relation on Na, (O4) means that Ga is acyclic. 8Here, p and s stand for predecessor and successor, respectively.

93 we say that i controls j. Let c(i; Ga) be the set of workers who agent i controls, including i. That is,

c(i; Ga) = {j ∈ Na : i controls j} ∪ {i}.

For i ∈ Na and j ∈ c(i; Ga)\{i}, if ij ∈ Ga, i is said to directly control j, and otherwise, i is said to indirectly control j in Ga. (O1) and (O2) describe that there is only one head h in an organization and every other worker has only one manager. Note that (O2) implies that, for any i, j ∈ Na with i 6= j, s(i; Ga) ∩ s(j; Ga) = ∅. (O3) is equivalent to |s(i; Ga)| ≤ ai for each i ∈ Na, and means that the number of each worker’s subordinates cannot be greater than the number of tasks he can deal with. (O4) implies that if a worker i (directly or indirectly) controls j, j cannot (directly or indirectly) control i.

For i ∈ Na and j ∈ c(i; Ga), the distance from i to j in Ga, denoted di(j; Ga), is the number of links in the path from i to j. Note that di(i; Ga) = 0 for all i ∈ N. For d convenience, we let dh(i; Ga) = dh(Ga)(i; Ga). Notice that, for each i ∈ Na, p (i; Ga) > 1 for d (i;Ga) all d < dh(i; Ga) and p h (i; Ga) = 1. In addition, we deﬁne the height of Ga by

R(Ga) = max{dh(i; Ga)}. i∈Na

Let Ga denote the set of all possible structures of an organization for ability vector a.

From (O1) to (O4), it is easy to see that any Ga ∈ Ga can be represented as a rooted tree in which the root is h(Ga) and the leaves are workers who do not have subordinates. Next, we will specify the value of the organization which depends on the structure of the organization. The value of the organization consists of the beneﬁt from the productive tasks the organization performs and the wage cost. More precisely, the beneﬁt of the organization is X B(Ga) = (ai − |s(i; Ga)|)v(dh(i; Ga)), (4.1)

i∈Na where v(·) is a decreasing function of d. Here, v(d) represents the unit beneﬁt from worker’s productive task when he is controlled by h through d−1 managers. The assumption that v is decreasing in d captures that there is information loss as the number of mediators increases, or that a command from head for a productive task is more likely to be distorted as the number of managers increases, which may decrease the productivity of an organization. In addition, we assume that v satisﬁes, for each d ≥ 0, v(d) ≥ 0 and

2v(d + 1) − v(d) > 2v(d + 2) − v(d + 1).9 (4.2)

9It will be clear in Lemma 4.1 that this assumption is required for eﬃciency to be obtained in the structure where productive tasks are assinged to the workers in the lower hierarchical levels.

94 An example of v satisfying (4.2) is v(d) =v ¯ +cδ ¯ d wherev ¯ ≥ 0,c ¯ > 0, and δ ∈ (1/2, 1).10 Since we assume that each worker has to use all of his abilities in doing productive tasks or supervising his subordinates, ai − |s(i; Ga)| is the number of productive tasks that worker i carries out. In this paper, it is assumed that the wage for each worker depends only on his hierarchical position in an organization. A rank system, which determines the hierarchical level of a worker, is deﬁned by a function ρ : Na × Ga → Z. That is, ρ(i; Ga) > ρ(j; Ga) means that worker i is in the higher position than worker j in the hierarchy of Ga. A reasonable rank system ρ may have to satisfy the following: for any Ga ∈ Ga,

(R1) if i controls j, ρ(i; Ga) > ρ(j; Ga),

0 0 (R2) if {i ∈ Na : ρ(i; Ga) = r} = ∅, {i ∈ Na : ρ(i; Ga) = r } = ∅ for all r > r, and

(R3) min{ρ(i; Ga): i ∈ N} = 0.

If ρ(i; Ga) = r, then worker i is said to be in rank r (or, in hierarchical level r). (R1) captures that a manager is in a higher hierarchical position than his subordinates.

Since the head of the organization controls all other workers, ρ(h(Ga); Ga) > ρ(i; Ga) for all i 6= h(Ga). That is, h(Ga) takes the highest position in the hierarchy of the organization. (R2) implies that there is no gap in hierarchical levels in the sense that, if there are workers in ranks r and r + 2, there should be a worker in rank r + 1. (R3) normalizes the rank system of organization at the bottom. One may want to normalize rank systems at the top by letting ρ(h(Ga); Ga) =r ¯ for sufficiently larger ¯ ∈ N. However, it seems more reasonable and realistic to normalize rank systems at the bottom, since we frequently observe that workers in the lowest hierarchical level are treated more equally than CEOs are treated in firms with different sizes. There are two rank systems which satisfy (R1) to (R3): counting up rank system and counting down rank system.11

Counting down rank system:

ρ(i; Ga) = max{dh(j; Ga)} − dh(i; Ga) j∈Na

Counting up rank system:

ρ(i; Ga) = max {di(j; Ga)} j∈c(i;Ga)

10Given that a is fixed, another example of v satisfying (4.2) is v(d) =v ¯ − cd wherev ¯ > 0 is sufficiently high and c > 0 is sufficiently small so that v(d) > 0 is satisfied for all d satisfying 0 ≤ d ≤ maxGa∈Ga R(Ga). 11These rank systems are suggested by Beckmann (1988). Note that there are rank systems other than counting up and down rank systems satisfying (R1) to (R3).

95 R(Ga) = 2 2 2 N2(Ga) 3 1 1 N1(Ga)

4 3 4 N0(Ga)

(a) Counting Down (b) Counting Up

Figure 4.1: Counting up and down rank systems

Given a rank system ρ, note that ρ(h(Ga); Ga) = R(Ga). For a rank system ρ, Nr(Ga) denotes the set of workers whose ranks are r. That is,

Nr(Ga) = {i ∈ Na : ρ(i; Ga) = r}.

0 Note that, under the counting down rank system, Nr(Ga) = Dr0 (Ga) with r = R(Ga) − r for each r ≥ 0. In addition, we denote by N r(Ga) the set of workers who are in a rank higher than or equal to rank r. That is, [ N r(Ga) = Nρ(Ga). ρ≥r

Figure 4.1 illustrates the counting down and counting up systems for a = (3, 2, 2, 2) and

Ga = (21, 23, 14). Throughout the paper, a rank system is referred to as a counting up rank system unless otherwise noted. At the end of Section 4.3, we will discuss the reason why we choose the counting up rank system for analysis. We assume that the wage for each rank is exogenously given and workers in the same rank receive the same wage. Let wr denote the wage for a worker in rank r. Since the wage is positive and the workers in a higher rank are paid more than those in a lower rank, ∞ 0 < wr < wr+1 for each r ≥ 0 is satisﬁed. We denote by w the wage sequence {wr}r=0. The cost of organization with structure Ga is the sum of workers’ wages. That is, X X

C(Ga) = wρ(i;Ga) = |Nr(Ga)|wr. (4.3) i∈Na r≥0

From (4.1) and (4.3), we deﬁne the value of the organization with Ga by

V (Ga) = B(Ga) − C(Ga).

96 ∗ ρ(i; Ga) = 2 1 ∗ ∗ ρ(i; Ga) = 1 ρ(i; Ga) = 0 2 3 4 5

6 7 8 9 10

∗ Figure 4.2: Example of Ga

0 0 Given a, a structure Ga ∈ Ga is eﬃcient if V (Ga) ≥ V (Ga) for all Ga ∈ Ga.

∗ 4.3 Eﬃcient Structure Ga

∗ ∗ In this section, we construct a structure Ga for each a and show that Ga is eﬃcient. Given ∗ ∗ an ability vector a, the structure Ga of an organization is deﬁned as follows: ij ∈ Ga if and only if

i = 1 and j ∈ {2,..., 1 + a1} ∩ Na, or ( ) Xi−1 Xi i ∈ Na\{1} and j ∈ 2 + ak,..., 1 + ak ∩ Na. k=1 k=1 ∗ ∗ It is easy to see that Ga satisﬁes all conditions (O1) to (O4) where h(Ga) = 1. Figure 4.2 ∗ describes the structure Ga for a = (4, 3, 3, 3, 2, 2, 2, 2, 2, 2). 0 0 ∗ Let i, i ∈ Na satisfy i < i . By construction of Ga, it is not diﬃcult to see that j ∈ ∗ 0 0 ∗ 0 0 ∗ 0 ∗ s(i; Gi ) and j ∈ s(i ; Ga) imply j < j . In addition, if i 6= 1 and i 6= 1, p(i; Ga) ≤ p(i ; Ga) ∗ 0 holds. Thus, under Ga, if worker i has higher ability than i has, then i’s manager has higher ability than i0’s manager and i’s subordinates have at least as high ability as i0’s 0 ∗ 0 ∗ subordinates have. Furthermore, if 1 < i < i , then p(i; Ga) ≤ p(i ; Ga) and so

∗ 0 ∗ dh(i; Ga) ≤ dh(i ; Ga). (4.4)

Let µ be the number of productive tasks the organization can do. Since all abilities of workers in an organization should be used to control n−1 workers or to perform productive tasks, we have X µ = ak − (n − 1), (4.5)

k∈Na and it does not depend on the structure of organization.

97 ∗ Given the structure Ga, we deﬁne ir for each r ≥ 0 iteratively as follows: i0 = n ∗ ∗ ∗ and ir = p(ir−1; Ga) for each r with 1 ≤ r ≤ R(Ga). By construction of Ga, we have ∗ ∗ ∗ N ∗ (G ) = {1} and N (G ) = {i + 1, . . . , i } for each r ≥ 1, and so |N (G )| = R(Ga) a r a r+1 r r a ir − ir+1. Note that the workers with lower ability cannot be in a higher hierarchical level 0 0 ∗ 0 ∗ 12 than those with higher ability. That is, for i, i ∈ Na with i < i , ρ(i; Ga) ≥ ρ(i ; Ga). ∗ 0 Under Ga, worker i in rank r ≥ r uses his ability either to control the subordinates who are in rank r00 ≤ r − 1 or to do productive tasks, which implies

Xn Xr X Xr−1 ∗ ak = ak ≤ µ + |Nρ(Ga)|. (4.6) ∗ k=ir+1+1 ρ=0 k∈Nρ(Ga) ρ=0

∗ Furthermore, under Ga, since each worker k < ir+1 has to use his full ability to manage Pir+1−1 the workers j with 1 < j < ir, we have k=1 ak < ir − 1. Then, simple calculation with (4.5) leads us to Xr−1 Xn ∗ µ + |Nρ(Ga)| < ak. (4.7) ρ=0 k=ir+1

∗ ∗ To show the efficiency of Ga, we first show that Ga maximizes B on Ga, and then show ∗ that Ga minimizes C. Lemmas 4.1 and 4.2 provide the necessary conditions for a structure to maximize B. That is, in an efficient structure of organization, productive tasks are assigned to workers in the lower hierarchical levels and workers with higher ability cannot be in hierarchical level lower than those with lower ability.

Lemma 4.1. Suppose that Ga ∈ Ga maximizes B on Ga. If dh(i; Ga) + 1 < dh(j; Ga) for some i, j ∈ Na, then |s(i; Ga)| = ai.

Proof. Let Ga ∈ Ga and suppose that dh(i; Ga) + 1 < dh(j; Ga) and |s(i; Ga)| < ai for some 0 i, j ∈ Na. Let j be a worker which maximizes dh(j; Ga) subject to dh(i; Ga)+1 < dh(j; Ga). 0 0 Then, s(j ; Ga) = ∅ and dh(j ; Ga) > 1. 0 0 0 0 Let Ga = Ga − p(j ; Ga)j + ij . Then,

0 B(Ga) − B(Ga) 0 0 = aj0 v(dh(i; Ga) + 1) + v(dh(j ; Ga) − 1) − (aj0 v(dh(j ; Ga)) + v(dh(i; Ga)))

12Recall that the the rank system is referred to as a counting up rank system. If we adopt the counting down rank system instead of the counting up rank system, this property is also satisﬁed.

98 i i

0 0 p(j ; Ga) j

aj0

aj0 0 (a) Ga (b) Ga

0 Figure 4.3: Example of Ga and Ga for Lemma 4.1

¡ 0 ¢ ¡ 0 ¢ = aj0 v(dh(i; Ga) + 1) − v(dh(j ; Ga)) − v(dh(i; Ga)) − v(dh(j ; Ga) − 1) ¡ 0 ¢ ¡ 0 ¢ ≥ 2 v(dh(i; Ga) + 1) − v(dh(j ; Ga)) − v(dh(i; Ga)) − v(dh(j ; Ga) − 1) ¡ 0 0 ¢ = 2v(dh(i; Ga) + 1) − v(dh(i; Ga)) − 2v(dh(j ; Ga)) − v(dh(j ; Ga) − 1) > 0, which is a contradiction. The last inequality comes from (4.2). Figure 4.3 provides an 0 example of Ga and Ga in the proof.

Lemma 4.2. Suppose that Ga ∈ Ga maximizes B on Ga. Then, for any i, j ∈ Na with aj > ai, dh(j; Ga) ≤ dh(i; Ga).

Proof. Suppose that Ga ∈ Ga satisfies that, for some i, j ∈ Na, aj > ai and dh(j; Ga) > dh(i; Ga). Let Sj be a subset of s(j; Ga) such that |Sj| = min{|s(j; Ga)|, aj − ai}. Let S C = c(k; G ). Note that S is empty if s(j; G ) = ∅. If S is not empty, s(k; G ) = ∅ j k∈Sj a j a j a for some k ∈ Cj. Let π : Na → Na be a permutation of Na defined by π(i) = j, π(j) = i, 00 and π(k) = k for all k 6= i, j. Define Ga = (Ga\{jk : k ∈ Sj}) ∪ {ik : k ∈ Sj} and 0 00 0 0 Ga = {π(i)π(j): ij ∈ Ga}. Then, |s(j; Ga)| = |s(i; Ga)| + |Sj|, |s(i; Ga)| = |s(j; Ga)| − |Sj|, 0 0 00 and dh(j; Ga) = dh(i; Ga) and dh(i; Ga) = dh(j; Ga). Note that Ga ∈ Ga may not be 00 satisfied since |Sj| > 0 and |s(i; Ga)| = ai imply |{k ∈ Na : ik ∈ Ga}| = ai + |Sj| > ai. 0 0 However, Ga ∈ Ga is always satisfied. In addition, for each k ∈ Cj, |s(k; Ga)| = |s(k; Ga)| 0 and dh(k; Ga) < dh(k; Ga). Since either aj − ai − |Sj| > 0 or ak − |s(k; Ga)| > 0 for some k ∈ Cj holds, we have

0 B(Ga) − B(Ga) X X 0 0 = (ak − |s(k; Ga)|)v(dh(k; Ga)) − (ak − |s(k; Ga)|)v(dh(k; Ga)) k∈Cj ∪{i,j} k∈Cj ∪{i,j}

99 h h h i i j

Sj Sj j j i

C Cj Sj j

00 0 (a) Ga ∈ Ga (b) Ga ∈/ Ga (c) Ga ∈ Ga

00 0 Figure 4.4: Example of Ga, Ga, and Ga for Lemma 4.2

X ¡ 0 ¢ = (ak − |s(k; Ga)|) v(dh(k; Ga)) − v(dh(k; Ga)) k∈Cj

+ (aj − ai − |Sj|)(v(dh(i; Ga)) − v(dh(j; Ga))) > 0.

00 0 Figure 4.4 provides an example of Ga, Ga, and Ga in the proof.

Lemma 4.3 states that the necessary conditions in the previous lemmas are suﬃcient for ∗ the beneﬁt maximizing structure. In Proposition 4.1, we show that Ga maximizes B on Ga.

Lemma 4.3. If Ga ∈ Ga satisﬁes the conditions in Lemma 4.1 and 4.2, then Ga maximizes

B on Ga.

Proof. Given Ga, let Dd(Ga) denote the set of workers i such that dh(i; Ga) = d for each d ≥ 0. Note that Na is partitioned into {Dd(Ga) : 0 ≤ d ≤ R(Ga)}. Suppose that Ga ∈ Ga satisﬁes the conditions in Lemma 4.1 and 4.2. Then, iteratively, it is not diﬃcult to show ∗ ∗ P that R(Ga) = R(G ) and, for each d ≥ 0, |Dd(Ga)| = |Dd(G )| and ak = P aP a k∈Dd(Ga) ∗ ak. Since |s(k; Ga)| = |Dd(Ga)| for each d ≥ 0, we have k∈Dd(Ga) k∈Dd−1(Ga) X B(Ga) = (ak − |s(k; Ga)|)v(dh(k; Ga))

k∈Na RX(Ga) X = (ak − |s(k; Ga)|)v(d) d=0 k∈D (G )  d a  RX(Ga) X X =  ak − |s(k; Ga)| v(d)

d=0 k∈Dd(Ga) k∈Dd(Ga)

100   RX(Ga) X =  ak − |Dd+1(Ga)| v(d) d=0 k∈D (G )  d a  ∗ RX(Ga) X  ∗  = ak − |Dd+1(Ga)| v(d) ∗ d=0 k∈Dd(Ga)

Since B has a maximum on Ga and the conditions in Lemmas 4.1 and 4.2 are necessary, this completes the proof.

∗ Proposition 4.1. Ga maximizes B on Ga.

∗ ∗ Proof. From (4.4), Ga satisﬁes the conditions in Lemma 4.2. Suppose that dh(i; Ga) + 1 < ∗ ∗ ∗ ∗ dh(j; Ga) and |s(i; Ga)| < ai for some i, j ∈ Na. Then, by construction of Ga, |s(k; Ga)| = 0 ∗ ∗ for all k > i. Since dh(j; Ga) ≥ 2, there is a worker k ∈ Na\{1} such that j ∈ s(k; Ga) and ∗ ∗ ∗ dh(i; Ga) < dh(k; Ga). Then, (4.4) implies that i < k. Since |s(k; Ga)|= 6 0 and k > i, we have a contradiction. Therefore, the condition in Lemma 4.1 is satisﬁed. The result follows from Lemma 4.3.

∗ Proposition 4.2 shows that Ga minimizes C on Ga and Lemma 4.4 providing the suﬃcient condition for the cost of organization to be minimized is useful to prove Proposition 4.2.

13 Since wr − wr−1 > 0 for all r ≥ 1, we have the results.

∗ Proposition 4.2. Under the counting up rank system, Ga minimizes C on Ga. 13This proof is a simple modiﬁcation of the proof in Beckmann (1988). We include this proof for com- pleteness.

101 Pr ∗ Proof. Because of Lemma 4.4, it is enough to show that for each r ≥ 1, ρ=1 |Nρ(Ga)| ≥ Pr |Nρ(Ga)| for all Ga ∈ Ga. Let µ be deﬁned as in (4.5). Then, each Ga ∈ Ga should ρ=1 P satisfy a ≤ µ and i∈N0(Ga) i

Xr X Xr−1 ak ≤ µ + |Nρ(Ga)| (4.9)

ρ=0 k∈Nρ(Ga) ρ=0

Pr ∗ Pr Therefore, ρ=0 |Nρ(Ga)| ≥ ρ=1 |Nρ(Ga)| for all Ga ∈ Ga.

∗ The eﬃciency of Ga is obtained as a corollary of Proposition 4.1 and 4.2.

∗ Corollary 4.1. Under the counting up rank system, Ga maximizes V on Ga.

Proof. The result follows from Propositions 4.1 and 4.2.

∗ Corollary 4.1 does not claim that Ga is the unique efficient allocation in Ga. Any ∗ ∗ structure Ga ∈ Ga which is equivalent to Ga in the sense that Ga is obtained from Ga by relabeling the workers with same abilities is also efficient.14 Furthermore, for some a, there ∗ can be an efficient structure Ga which is not equivalent to Ga. For example, let n = 5 and a = (3, 3, 3, 2, 2). Then, it is not difficult to see that Ga = {12, 13, 14, 45} is also efficient, ∗ though not equivalent to Ga. ∗ ∗ To establish the efficiency of Ga, we showed that Ga maximizes B and minimizes C on Ga. We note that maximizing B is independent of maximizing C. To see this, let a = (3, 2).

Then, Ga = {21} trivially minimizes C but does not maximize B. For a = (2, 2, 2, 2, 2), we can see that Ga = {12, 13, 24, 35} maximizes B but does not minimize C.

14We will deﬁne an equivalent structure formally in Section 4.5.

102 ∗ 0 N G N2 G 2( a) 1 4 ( a) ∗ N1(Ga) 0 N1(Ga) 2 3 4 5 5 2 ∗ N0 G 0 ( a) N0(Ga) 6 7 8 6 7 8 3 1

∗ 0 (a) Ga (b) Ga

Figure 4.5: Counting down rank system

In Section 4.2, we introduced the counting down rank system as well as the counting up rank system. Since we focused on the counting up rank system, one may attempt to find an efficient structure under the counting down rank system. Consider the following ∗ 0 example. Let n = 8 and a = (4, 3, 3, 2, 2, 2, 2, 2). Consider the structures Ga and Ga ∗ described in Figure 4.5. Under the counting down rank system, Ga maximizes B but it 0 does not minimize C, while Ga minimizes C but it does not maximize B. Thus, an efficient structure may depend on v and w, which brings difficulty in finding an efficient structure. In deed, we may need more assumptions on v and w to find an efficient structure under the counting down rank system. Furthermore, it seems to be considered desirable that workers with higher ability get a higher wage than those with lower ability. Under the counting down rank system, there is 0 an ability vector for which this property conflicts with minimizing the cost. Notice that Ga in Figure 4.5 is the unique (up to equivalent class) structure which minimizes C under the 0 counting down rank system. In structure Ga, worker 2 receives a lower wage than worker 4 receives though worker 2 is more able than worker 4. Under the counting up rank system, the wage for workers with higher ability is no less than the wage for those with lower ability ∗ in Ga which minimizes the cost. Proposition 4.3 provides another reason for considering the counting up rank system. Since the total wage under the counting up rank system is no greater than that under the counting down rank system, an organization prefers the former rather than the later. Recall that workers are assumed to follow the organization’s decision.

Proposition 4.3. For any Ga ∈ Ga, the counting up rank system yields a wage cost smaller than or equal to the wage cost the counting down rank system yields.

Proof. Let ρ and ρ be counting up and down rank systems, respectively. For each i ∈ Na, µ ¶

ρ(i; Ga) − ρ(i; Ga) = max{di(j; Ga)} − max{dh(j; Ga)} − dh(i; Ga) j∈Na j∈Na

103 = max{dh(i; Ga) + di(j; Ga)} − max{dh(j; Ga)} j∈Na j∈Na

= max{dh(j; Ga): i controls j} − max{dh(j; Ga)} j∈Na j∈Na ≤ 0.

Then, we have ωρ(i;Ga) ≤ ωρ(i;Ga) for all i ∈ Na, which implies the result.

4.4 Optimal Size of Organization

In the previous section, we let workers and their abilities be given and found an efficient structure of an organization. In this section, we allow an organization to choose its workers from the labor pool and explore the optimal number of workers in the organization. Let N = {1, 2,..., } be the set of potential workers. We assume that, for each i ≥ 1, 15 ai ≥ ai+1 and ai ≥ 2 are satisfied. For a finite subset N of potential workers, let a(N) denote the ability vector corresponding to N. That is, a(N) = (ai)i∈N . The organization wants to choose N and Ga(N) ∈ Ga(N) maximizing its value. Because of Section 4.3, we ∗ can restrict our attention to N = {1, . . . , n} and Ga(N) = Ga(N). For convenience, we n ∗ ∗ ∗ let a(n) = (ai)i=1, Gn = Ga(n), and R(n) = R(Gn). Then, the organization’s problem is ∗ ∗ reduced to finding n such that N = {1, . . . , n } and Ga(N) ∈ Ga(N) which maximize V . Suppose that the organization hiring n − 1 workers considers whether to hire worker n or not. The marginal benefit MB(n) and the marginal cost MC(n) of worker n are given as follows:

∗ ∗ MB(n) = B(Gn) − B(Gn−1) = anv(R(n)) − v(R(n) − 1) ∗ ∗ MC(n) = C(Gn) − C(Gn−1)

Since

MB(n) − MB(n + 1)

= anv(R(n)) − v(R(n) − 1) − (an+1v(R(n + 1)) − v(R(n + 1) − 1))

≥ an+1v(R(n)) − v(R(n) − 1) − (an+1v(R(n + 1)) − v(R(n + 1) − 1)) ≥ 2v(R(n)) − v(R(n) − 1) − (2v(R(n + 1)) − v(R(n + 1) − 1)) ≥ 0,

15 This assumption seems somewhat strong since it requires that ai is bounded and the set of workers i such that ai > aj for some j ∈ N is ﬁnite. However, if the set of workers i such that ai > aj for some j ∈ N is inﬁnite, we can ignore the workers who has lower ability than ai since the organization never hires ∞ them at the optimal. Thus, we can include this assumption without loss of generality, given that {ai}i=1 is assumed to be bounded.

104 MB(n) is non-increasing in n, while MC(n) is neither increasing nor decreasing in n since

MC(n) = wR(n) if hiring worker n increases the height of the organization (that is, R(n) >

R(n − 1)) and MC(n) < wR(n) otherwise (that is, R(n) = R(n − 1)). Non-monotonicity of marginal cost in n makes it diﬃcult to determine the optimal size n∗ of organization. Furthermore, the assumptions on v and w in Section 4.2 are not enough to ensure the existence of the optimal size n∗. For example, let v and w satisfy v(d) =v ¯ +cδ ¯ d and wr =v ¯ − 1/(r + 1) with 1 < v¯. Then, v and w satisfy the assumptions in Section 4.2, but hiring an additional worker is always beneﬁcial since MB(n) > MC(n) for all n. ∗ For the existence of the optimal n ≥ 2, we assume that a2v(1) + (a1 − 1)v(0) > w0 + w1 and (an¯ − 1) v(R(¯n) − 1) ≤ w0 for somen ¯. Then since

MB(k) ≤ (ak − 1) v(R(k) − 1) ≤ (an¯ − 1) v(R(¯n) − 1) ≤ w0 ≤ MC(k), for all k ≥ n¯, there exists the optimal n∗ satisfying 2 ≤ n∗ ≤ n¯. Propositions 4.4 and 4.5 provide the properties of the optimal size n∗ with additional ∗ assumptions. Since we restrict structures to Gn, the properties in Propositions 4.4 and 4.5 describe the structure of organization which has the optimal size.

Proposition 4.4. Given v and w, let n∗ be the optimal size of an organization. If ∗ ∗ ∗ Pp∗ ∗ ∗ ∗ MB(n ) − MC(n ) 6= 0 and an∗ = an∗+1, then n = k=1 ak + 1 where p = p(n ; Gn∗ ).

Proof. Since n∗ is the optimal size of the organization, MB(n∗) − MC(n∗) 6= 0 implies ∗ ∗ ∗ ∗ ∗ ∗ Pp∗ ∗ MB(n ) − MC(n ) > 0. Let p = p(n ; Gn∗ ). If n < k=1 ak + 1, then Gn∗+1 = ∗ ∗ ∗ ∗ ∗ ∗ ∗ Gn∗ + p (n + 1) and ρ(k; Gn∗ ) = ρ(k; Gn∗+1) for all k ≤ n . Then, since MC(n ) ≥ ∗ ∗ ∗ ∗ ∗ w0 = MC(n + 1) and MB(n ) = MB(n + 1), we have MB(n + 1) − MC(n + 1) ≥ MB(n∗)−MC(n∗) > 0. This contradicts that n∗ is the optimal size of the organization.

Note that MB(n∗) − MC(n∗) 6= 0 in Proposition 4.4 is generically satisﬁed and, if the organization is large enough to hire the lowest ability workers, an∗ = an∗+1 is also satisﬁed. ∗ ∗ Proposition 4.4 implies that under Gn∗ , there exists p ∈ Na such that, for all i with ∗ ∗ ∗ ∗ ∗ 1 ≤ i ≤ p , |s(i; Gn∗ )| = ai and, for all i with p < i ≤ n , |s(i; Gn∗ )| = 0. In other words, each worker uses his full ability either in supervising his subordinates or in performing productive tasks, but not both. In addition, all productive tasks are assigned to the workers ∗ at the bottom of the hierarchy. Furthermore, by construction of Gn∗ , all productive tasks are done by the workers with low ability.16

16Many previous studies, such as Williamson (1967), Calvo and Wellisz (1978), Calvo and Wellisz (1979), and Qian (1994), assume that only the workers in the lowest hierarchical level do productive tasks and that the other workers supervise their subordinates. Note that we derive this property as a result of maximizing the value of organization.

105 ∗ Proposition 4.5. Suppose that ak = ao ≥ 2 for all k ∈ N . If n is the optimal size of the ∗ PR r ∗ organization, then n = r=0 ao where R = R(Gn∗ ).

Pr−1 ρ Pr ρ Proof. Let D0 = {1} and Dr = { ρ=1 ao +1,..., ρ=1 ao} for each r ≥ 0. Since aj = ao for ∗ ∗ all j ∈ N , we have Dr = {j ∈ N : d1(j; Gn) = r} for any n ≥ j. Let n be the optimal size ∗ PR r ∗ of the organization and R = R(Gn∗ ). For convenience, we letn ¯ = r=0 ao. Then, n ∈ DR PR−1 r ∗ and so r=1 ao + 1 ≤ n ≤ n¯. Note that, for all n ∈ DR, MB(n) = aov(R) − v(R − 1). ∗ ∗ ∗ ∗ ∗ ∗ Suppose that n < n¯. Let p = max{p ∈ N : n ∈ c(p; Gn¯) and n + 1 ∈ c(p; Gn¯)}. ∗ ∗ ∗ ∗ Let k = min {k ∈ N : k ∈ s(p ; Gn¯)}. For convenience, for each k ∈ s(p ; Gn¯), let e(k) = ∗ ∗ min{j ∈ N : j ∈ c(k; Gn¯) and j ∈ DR} andm ¯ (k) = max{j ∈ N : j ∈ c(k; Gn¯) and ∗ ∗ j ∈ DR} − e(k). Since aj = ao for all j ∈ N , we can letm ¯ (k) =m ¯ for all k ∈ s(p ; Gn¯). ∗ ∗ Then, we can see that, for all k ∈ s(p ; G ) with k 6= k, MC(e(k)) ≥ w ∗ ∗ > MC(e(k)) n¯ ρ(p ;Gn¯ ) and MC(e(k) + m) = MC(e(k) + m) for m satisfying 1 ≤ m ≤ m¯ . Thus, we have Pe(k)+m ¯ Pe(k)+m ¯ ∗ ∗ j=e(k) MC(j) > j=e(k) MC(j) for all k ∈ s(p ; Gn¯)\{k}. ∗ ∗ ∗ ∗ ∗∗ ∗ ∗ Let k be the worker in s(p ; Gn¯) who controls n and k be the worker in s(p ; Gn¯)) ∗ ∗ 17 ∗ ∗ ∗ Pe(k )+m ¯ who controls n + 1. Suppose that k = k. Then, e(k ) +m ¯ = n and j=e(k∗) MC(j) > ∗∗ ∗ Pe(k )+m ¯ ∗ Pe(k )+m ¯ j=e(k∗∗) MC(j). Since n is the optimal size of the ﬁrm, 0 ≤ j=e(k∗) [MB(j) − MC(j)] 0 0 holds. Then, since MB(j) = MB(j ) for all j, j ∈ DR, we have

∗ ∗∗ e(kX)+m ¯ e(kX)+m ¯ 0 ≤ [MB(j) − MC(j)] < [MB(j) − MC(j)] , j=e(k∗) j=e(k∗∗) which contradicts that n∗ is the optimal size of the organization. Suppose that k∗ > k. Pk∗ Pe(k)+m ¯ Pe(k)+m ¯ Pe(k)+m ¯ Then, 0 ≤ k=k j=e(k) [MB(j) − MC(j)] and j=e(k) MC(j) > j=e(k) MC(j) for ∗ ∗ each k ∈ s(p ; Gn¯)\{k} imply

∗ ∗∗ e(kX)+m ¯ e(kX)+m ¯ 0 < [MB(j) − MC(j)] = [MB(j) − MC(j)] , j=e(k∗) j=e(k∗∗) which contradicts that n∗ is the optimal size of the ﬁrm. Therefore, we have n∗ =n ¯. ∗ ∗ ∗∗ ∗ ∗ Figure 4.6 provides an example of p , k , k , and e(k ) when ao = 3 and n < n¯.

Here, the thick lines represent the marginal cost induced by a worker in DR. For instance, ∗ ∗ ∗ ∗ ∗∗ ∗ MC(n ) is worker n ’s wage in Gn∗ , MC(n + 1) is worker k ’s wage in Gn∗+1, and so on.

Beckmann (1988) refers to a structure in which workers in the lowest rank have the same distance from the head of organization as a balanced structure.18 Proposition 4.5

17 ∗∗ ∗ ∗ ∗∗ ∗ ∗ ∗ ∗ ∗∗ ∗ Note that k = k +1 and k , k ∈ s(p ; Gn¯ ). Furthermore, we have e(k )+m ¯ = n and e(k ) = n +1. 18Beckmann (1988) also refers to a balanced structure with constant span of control as a regular structure.

106 h

p∗

∗ k = k k∗∗

∗ ∗∗ e(k ) n∗ n∗ + 1 e(k ) +m ¯ n¯

Figure 4.6: Example for the proof of Proposition 4.5 implies that, if all workers have the same ability, the organization with the optimal size has a structure such that all the productive tasks are assigned to the workers in the lowest rank and that it is balanced. If all workers have the same ability ao, the number of workers for ∗ ∗ each rank r in a balanced structure satisﬁes |Nr(Gn∗ )| = ao|Nr+1(Gn∗ )|. Note that we can replace the same ability assumption in Proposition 4.5 with one such that all workers who ∗ have the same distance from the head in Gn∗ have the same ability.

4.5 Characterization of Structure Rule

∗ ∗ In Section 4.3, we showed that Ga is an efficient structure for each a. Recall that Ga is not the unique efficient structure for some a. The purpose of this section is to characterize the ∗ structure Ga with some axioms. We first define a structure rule. Let A be the set of ability vectors a = (a1, . . . , an) S satisfying n ≥ 2 and ai ≥ ai+1 ≥ 2 for all i.A structure rule is a function ϕ : A → a∈A Ga ∗ S satisfying ϕ(a) ∈ Ga for each a ∈ A. An example of a structure rule is ϕ : A → a∈A Ga defined by ∗ ∗ ϕ (a) = Ga for each a ∈ A (4.10)

∗ where Ga is a structure constructed in Section 4.3. For structure rules, we are interested in two axioms: efficiency and lower ability consistency. S Efficiency. A structure rule ϕ : A → a∈A Ga is efficient if, for each a ∈ A, V (ϕ(a)) ≥ V (Ga) for all Ga ∈ Ga. S Lower Ability. A structure rule ϕ : A → a∈A Ga satisfies lower ability consistency if,

Since we assume that ai = ao for all i ∈ N , the resulting structure in Proposition 4.5 is also regular.

107 0 0 0 for any a, a ∈ A such that a = (a1, . . . , an) and a = (a; an+1, . . . , an+m), ϕ(a )|Na = ϕ(a) 0 0 where ϕ(a )|Na = {ij ∈ ϕ(a ): i ∈ Na and j ∈ Na}.

In other words, a structure rule is efficient if it assigns a structure which maximizes the value of an organization given the set of workers. Lower ability consistency requires that, when an organization hires new workers who have lower ability than the current workers, the organization does not change the hierarchical relationship among current workers.19 Since newcomers tend to be less able (in terms of experience or knowledge) than current workers in an organization and there is a cost to rearrange the workers, lower ability consistency seems to be a desirable property for structure rules. Since we allow workers to have same ability, it is possible that structures are equivalent in the sense that one is obtained from the other by changing the names of workers who have the same ability. To define an equivalent structure formally, let a = (a1, . . . , an) ∈ A be given. An equivalent permutation π is a bijection on Na satisfying that π(i) = j implies 0 ai = aj. Given a ∈ A, Ga ∈ Ga is equivalent to Ga ∈ Ga if there is an equivalent permutation 0 π : Na → Na such that Ga = {π(i)π(j): ij ∈ Ga}. For Ga ∈ Ga, E(Ga) denotes the set of structures which are equivalent to Ga. In Proposition 4.6, we show that a structure rule satisfying efficiency and lower ability ∗ consistency always yields a structure which is equivalent to Ga.

Proposition 4.6. Let ϕ be an eﬃcient structure rule satisfying lower ability consistency. ∗ Then, for all a ∈ A, ϕ(a) ∈ E(Ga).

Proof. See the Appendix.

The proof is found in the Appendix. In the proof, we consider a structure rule satisfying ∗ efficiency and lower ability consistency and show that, if ϕ(a) ∈ E(Ga) for a = (a1, . . . , an) ∈ 0 0 ∗ 0 ∗ A and a = (a1, . . . , an, an+1) ∈ A, then ϕ(a ) ∈ E(Ga0 ). Indeed, if ϕ(a ) ∈ E(Ga0 ) is violated, lower ability consistency implies that there exists a00 ∈ A for which ϕ(a00) is ∗ not efficient. Since efficiency ensures that ϕ(a) ∈ E(Ga) for a = (a1, a2) ∈ A, iterative arguments imply the result. It has to be mentioned that Proposition 4.6 is not a complete characterization of the ∗ structure rule. We can easily construct a structure rule ϕ which satisfies ϕ(a) ∈ E(Ga) for each a but does not satisfy lower ability consistency. Note that any structure rule ϕ ∗ satisfying ϕ(a) ∈ E(Ga) for each a ∈ A is efficient. Furthermore, although it is trivial to show that the structure rule ϕ∗ in (4.10) is efficient and satisfies lower ability consistency, it is not a unique structure rule satisfying efficiency and lower ability consistency. However,

19We can also interpret lower ability consistency as follows: when an organization ﬁres the workers who have lowest ability, the organization does not change the current structure among the remaining workers.

108 1 1 1

2 3 4 2 3 4 2 3 4

5 5 6

(a) a = (3, 3, 2, 2) (b) a0 = (3, 3, 2, 2, 2) (c) a00 = (3, 3, 2, 2, 2, 2)

Figure 4.7: Structure ruleϕ ¯

Proposition 4.6 implies that any efficient and lower ability consistent structure rule yields ∗ a structure unique up to equivalent class of Ga. Before concluding this section, we provide examples to establish the independence of axioms. First, consider a structure ruleϕ ˆ such that, for each a ∈ A, ij ∈ ϕˆ(a) if and only if i, j ∈ Na and j = i + 1. Trivially,ϕ ˆ satisfies lower ability consistency. However, for any a ∈ A with |Na| ≥ 3,ϕ ˆ(a) is not efficient.

Next, let a structure ruleϕ ¯ be deﬁned by, for each a = (a1, . . . , an) ∈ A, ( P ∗ ∗ ∗ p(n−1;Ga) Ga − p(n; Ga)n + (n − 1)n if i=1 ai = n − 2 ϕ¯(a) = ∗ Ga otherwise

Then, one can easily show thatϕ ¯ is efficient but does not satisfy lower ability consistency. Figure 4.7 illustrates examples of structures resulting fromϕ ¯, where lower ability consistency is violated. Since the value of an organization is decomposed into benefit and cost, we may want to replace efficiency with benefit maximizing and cost minimizing, where a structure rule

ϕ is benefit maximizing if, for each a ∈ A, B(ϕ(a)) ≥ B(Ga) for all Ga ∈ Ga and cost minimizing if, for each a ∈ A, C(ϕ(a)) ≤ C(Ga) for all Ga ∈ Ga. If that is the case, since cost minimizing and lower ability consistency imply benefit maximizing, the efficiency axiom in Proposition 4.6 can be weakened to the axiom of cost minimizing.

4.6 Conclusion

This paper has studied the efficient structure of an organization in which employees with different abilities work either to supervise their subordinates or to perform productive tasks. The benefit of the organization comes from productive work and the organization has to pay the wage cost. We first find an efficient structure of organization in which workers with higher ability are assigned to higher positions in the hierarchy. Furthermore, at a

109 hierarchical level, workers with high ability have a manager and subordinates more able than the workers with low ability have. In this paper, we also characterize the efficient and lower ability consistent structure rule, which is unique up to equivalent class. Recall that this paper assumes that the wage depends not on the worker’s ability but on the rank assignment, and that workers do not have a bargaining power against the organization. Although some empirical results such as Medoff and Abraham (1981) reveal that the productivity effect on wage is not outstanding, these assumptions do not seem to be desirable. If we consider different wages for workers with various abilities in the same rank, our results break down and more assumptions on wage are needed to characterize efficient structures. In addition, we can think of a model where the wage is determined endogenously from the strategic behavior of the organization and its workers. If workers have productive options, the organization has to guarantee the wage no less than the payoff from their productive options in order to hire them. Furthermore, since the workers with high ability may get a higher payoff from his productive option than those with low ability, worker’s ability may influence the wage. We leave this for future study.

Appendix: Proof of Proposition 4.6

To prove Proposition 4.6, we need Lemma 4.5 and 4.6. Throughout the Appendix, let a structure S rule ϕ : A → a∈A Ga be eﬃcient and satisfy lower ability consistency.

∗ ∗ Lemma 4.5. Let a = (a1, . . . , an) ∈ A. Suppose that ϕ(a) ∈ E(Ga) and that Ga satisﬁes that there o o ∗ 0 0 ∗ exists i ∈ Na such that 0 < |s(i ; Ga)| < aio . If a = (a1, . . . , an, an+1) ∈ A, then ϕ(a ) ∈ E(Ga0 ).

∗ Proof. Without loss of generality, we let ϕ(a) = Ga, if necessary, by relabeling the workers, and will 0 ∗ ∗ ∗ o show that ϕ(a ) ∈ E(Ga0 ). For convenience, let R = R(Ga). Note that |s(i; Ga)| = ai for each i < i ∗ o 0 ∗ 0 and |s(i; Ga)| = 0 for each i > i . Since ϕ satisfies lower ability consistency, ϕ(a ) = Ga + i (n + 1) for some i0 such that io ≤ i0 ≤ n. Suppose that i0 6= io. Since ϕ is efficient, Lemma 4.1 implies 0 0 0 0 ∗ ∗ that dh(i ; ϕ(a )) = R − 1 and dh(n + 1; ϕ(a )) = R. Thus, we have B(ϕ(a )) = B(Ga0 ) = V (Ga) + o o ∗ 0 ∗ o o an+1v(R) − v(R − 1). Let p = max{p ∈ Na : i ∈ c(p; Ga) and i ∈ c(p; Ga)}. Since p < i and ∗ o ∗ ∗ ∗ ρ(i; Ga) = 1, we have ρ(p ; Ga) ≥ 2. Moreover, it is not difficult to see that C(Ga0 )) − C(Ga) = w0 0 ∗ o o ∗ ∗ 0 and C(ϕ(a )) − C(Ga) = wρo−1 where ρ = ρ(p ; Ga). Since C(Ga0 ) − C(ϕ(a )) = w0 − wρo−1 < 0, ∗ 0 we have V (Ga0 ) − V (ϕ(a )) = wρo−1 − w0 > 0 which contradicts that ϕ is efficient. Therefore, 0 ∗ o ∗ ϕ(a ) = Ga + i (n + 1) = Ga0 .

∗ ∗ Lemma 4.6. Let a = (a1, . . . , an) ∈ A. Suppose that ϕ(a) ∈ E(Ga) and that Ga satisﬁes that o ∗ o ∗ o there exists i ∈ Na such that |s(i; Ga)| = ai for all i < i and |s(i; Ga)| = 0 for all i ≥ i . If 0 0 ∗ a = (a1, . . . , an, an+1) ∈ A. Then, ϕ(a ) ∈ E(Ga0 ).

∗ Proof. Without loss of generality, we let ϕ(a) = Ga, if necessary, by relabeling the workers, and will 0 ∗ o ∗ ∗ ∗ o ∗ show that ϕ(a ) ∈ E(Ga0 ). Note that (i − 1)n ∈ Ga, Ga0 = Ga + i (n + 1), and ρ(i; Ga) = 0 for o ∗ d all i ≥ i . Let R = R(Ga). For convenience, for each i ∈ Na, we let p (i) denote the d-th order

110 ∗ ∗ manager of worker i in Ga. For each k ∈ Na, E(k) denotes the set of workers i such that i ∈ c(k; Ga) ∗ and ρ(i; Ga) = 0. 0 ∗ 0 Since ϕ satisfies lower ability consistency and efficiency, Lemma 4.1 implies ϕ(a ) = Ga +i (n+1) 0 0 o 0 ∗ o ∗ o d o d o for some i ∈ Na such that i ≥ i and dh(i ; Ga) = dh(i ; Ga). Let d = min{d : p (i ) = p (i − 1)} o and d0 = min{d : pd(i0) = pd(io − 1)}. Note that do + 1 and d0 + 1 are the ranks of pd (io − 1) and d0 o ∗ p (i − 1), respectively, in Ga. o 0 ∗ o 0 d0 o Suppose that d 6= d . By construction of Ga, we have d < d which implies the rank of p (i −1) do o ∗ 0 ∗ is higher than the rank of p (i − 1) in Ga. Then, V (ϕ(a )) − V (Ga0 ) = wro−1 − wr0−1 < 0 where 0 o r0 = ρ(pd (io − 1)) and ro = ρ(pd (io − 1)). Since this contradicts that ϕ is efficient, we have o o o pd (io) = pd (i0) = pd (io − 1). Furthermore, since do = 0 implies a contradiction that io = io − 1, we have do ≥ 1. o P o o 0 P Let m = j∈E(pdo−1(io)) aj and M = {n + 1, . . . , n + m }, and let m = j∈E(pdo−1(i0)) aj and M 0 = {n + 1, . . . , n + m0}. That is, mo (resp. m0) means the number of productive tasks worker do−1 o do−1 0 ∗ o 0 p (i )(resp. p (i )) controls. By the construction of Ga, we have m ≥ m . o 0 00 Suppose that m > m and let a = (a; an+1, . . . , an+mo ) satisfy that an+m = an+1 for all m such o 00 o that 1 ≤ m ≤ m . Note that a ∈ A and Na00 = Na ∪ M . Since ϕ satisfies lower ability consistency o 00 ∗ Pm ∗ o ∗ and efficiency, Lemma 4.1 implies ϕ(a ) = Ga + m=1 im(n + m) where dh(im; Ga) = dh(i ; Ga) o 0 and im ≥ i for each im and i1 = i . r o P r o 0 do o 00 For Ga00 ∈ Ga00 , let p (M ; Ga00 ) = k∈M o p (k; Ga00 ). Since m > m , we have |p (M ; ϕ(a ))| > do o ∗ do−1 o ∗ 1 and p (M ; Ga00 ) = {p (i )}. Notice that, for any Ga00 ∈ Ga00 such that Ga00 = Ga + o Pm ∗ o ∗ m=1 im(n + m) where dh(im; Ga) = dh(i ; Ga) for each m, we have

∗ r o N r(Ga00 ) = N r(Ga) ∪ p (M ; Ga00 ) for each r ≥ 0.

r o ∗ o Furthermore, p (M ; Ga00 ) ∩ N r(Ga) = ∅ for each r < d , and so

∗ r o o |N r(Ga00 )| = |N r(Ga)| + |p (M ; Ga00 )| for each r < d .

o r ∗ o The reason is as follows: Let r ≤ d and suppose that p (n+m; Ga00 ) ∈ N r(Ga) for some n+m ∈ M . r−1 ∗ r ∗ o ∗ Then, p (im; Ga) = p (n + m; Ga00 ) ∈ N r(Ga) for im ≥ i such that im(n + m) ∈ Ga00 and ∗ o ∗ o r−1 r−1 o r−1 ∗ dh(im; Ga) = dh(i ; Ga). Since im ≥ i implies p (im) ≥ p (i ), we have ρ(p (im); Ga) ≤ r−1 o ∗ r−1 o ∗ ∗ ρ(p (i ); Ga) and so p (i ) ∈ N r(Ga) is satisﬁed. However, by the construction of Ga, we have r−1 o ∗ o a contradiction that p (i ) ∈ Nr−1(Ga) for all r ≤ d . ∗ ∗ By the construction of Ga00 we can show that, for any Ga00 ∈ Ga00 such that Ga00 = Ga + Pm¯ ∗ o ∗ m=1 im(n + m) where dh(im; Ga) = dh(i ; Ga) for each m, it is satisﬁed that

r o r o ∗ |p (M ; Ga00 )| ≥ |p (M ; Ga00 )| for each r ≥ 0.

Then, since 00 ∗ r o 00 |N r(ϕ(a ))| = |N r(Ga)| + |p (M ; ϕ(a ))| ∗ r o ∗ ∗ ≥ |N r(Ga)| + |p (M ; Ga00 )| = |N r(Ga00 )|

111 o pd (io − 1), do = 2

do−1 0 do−1 o p i p (i ) ( )

io − 1 0 io i

n o m0 m n + 1

Figure 4.8: Example for the proof of Lemma 4.6

for each r < do,

00 ∗ do o 00 |N do (ϕ(a ))| = |N do (Ga)| + |p (M ; ϕ(a ))| ∗ do o ∗ ∗ > |N do (Ga)| + |p (M ; Ga00 )| = |N do (Ga00 )|, and 00 ∗ r o 00 |N r(ϕ(a ))| = |N r(Ga) ∪ p (M ; ϕ(a ))| ∗ ∗ ≥ |N r(Ga)| = |N r(Ga00 )| for each r > do, Lemma 4.4 implies that ϕ(a00) is not efficient. Therefore, we have mo = m0. o do−1 o ∗ 0 do−1 0 ∗ o ∗ Suppose that ajo 6= aj0 for some j ∈ c(p (i ); Ga) and j ∈ c(p (i ); Ga) with ρ(j ; Ga) = 0 ∗ ∗ ρ(j ; Ga) =r ¯. Then, by the construction of Ga, we have ajo > aj0 and, iteratively, this implies that X X ak > ak do−1 o ∗ ∗ do−1 0 ∗ ∗ k∈c(p (i );Ga):ρ(k;Ga)=r k∈c(p (i );Ga):ρ(k;Ga)=r for each r with 0 ≤ r ≤ r¯. Since this implies mo > m0 contradicting mo = m0, for jo ∈ do−1 o ∗ 0 do−1 0 ∗ o ∗ 0 ∗ c(p (i ); Ga) and j ∈ c(p (i ); Ga) satisfying ρ(j ; Ga) = ρ(j ; Ga), we have ajo = aj0 . o o From the previous arguments, we know that pd −1(io) and pd −1(i0) have the same manager do o do−1 o ∗ do−1 0 ∗ p (i − 1) and that the sub-structures with c(p (i ); Ga) and c(p (i ); Ga) are equivalent. 0 Then, it is not difficult to find an equivalent permutation π : Na0 → Na0 for which ϕ(a ) = {π(i)π(j): ∗ do−1 o ∗ do−1 0 ∗ o 0 ij ∈ Ga0 }. Indeed, π satisfies π(k) = k for k∈ / c(p (i ); Ga) ∪ c(p (i ); Ga), π(i ) = i , π(n + 1) = n + 1, π(pd(io)) = pd(i0) for each d, and so on. This completes the proof of Lemma 4.6. Figure 4.8 provides an example for Lemma 4.6.

S Proof of Proposition 4.6. Let ϕ : A → a∈A Ga be an eﬃcient structure rule satisfying lower ability t consistency. Let a = (a1, . . . , an) ∈ A with n ≥ 2. For each t with 2 ≤ t ≤ n, a = (a1, . . . , at). If ∗ 2 ∗ a1 = a2, each Ga2 ∈ Ga2 is in E(Ga2 ). Thus, ϕ(a ) ∈ E(Ga2 ). If a1 > a2, the eﬃciency of ϕ implies 2 ∗ ∗ ϕ(a ) = Ga2 ∈ E(Ga2 ). Then, the result follows from Lemmas 4.5 and 4.6.

112 References

Bala, V. and Goyal, S. (2000). A noncooperative model of network formation. Econometrica, 68(5):1181–1230.

Beckmann, M. J. (1988). Tinbergen Lectures on Organization Theory. Springer-Verlag.

Beggs, A. W. (2001). Queues and hierarchies. Review of Economic Studies, 68(2):297–322.

Ben-Porath, E. and Kahneman, M. (1996). Communication in repeated games with private monitoring. Journal of Economic Theory, 70(2):281–297.

Ben-Porath, E. and Kahneman, M. (2003). Communication in repeated games with costly monitoring. Games and Economic Behavior, 44(2):227–250.

Bhaskar, V. (1998). Informational constraints and the overlapping generations model: Folk and anti-folk theorems. Review of Economic Studies, 65(1):135–149.

Bhaskar, V. and Obara, I. (2002). Belief-based equilibria in the repeated prisoners’ dilemma with private monitoring. Journal of Economic Theory, 102(1):40–69.

Bogomolnaia, A. and Moulin, H. (2004). Random matching under dichotomous preferences. Econometrica, 72(1):257–279.

Bolton, P. and Dewatripont, M. (1994). The ﬁrm as a communication network. Quarterly Journal of Economics, 109(4):809–839.

Bramoull´e,Y. and Kranton, R. (2007). Public goods in networks. Journal of Economic Theory, 135(1):478–494.

Calvo, G. A. and Wellisz, S. (1978). Supervision, loss of control, and the optimum size of the ﬁrm. Journal of Political Economy, 86(5):943–952.

Calvo, G. A. and Wellisz, S. (1979). Hierarchy, ability, and income distribution. Journal of Political Economy, 87(5):991–1010.

Chatterjee, K., Dutta, B., Ray, D., and Sengupta, K. (1993). A noncooperative theory of coalitional bargaining. Review of Economic Studies, 60(2):463–477.

113 Chatterjee, K. and Sabourian, H. (2000). Multiperson bargaining and strategic complexity. Econometrica, 68(6):1491–1510.

Cho, M. (2007a). Cooperation in the repeated prisoner’s dilemma game with local interaction and local communication. Mimeo. The Pennsylvania State University.

Cho, M. (2007b). Public randomization in the repeated prisoner’s dilemma game with local interaction. Mimeo. The Pennsylvania State University.

Compte, O. (1998). Communication in repeated games with imperfect private monitoring. Econometrica, 66(3):597–626.

Dutta, B. and Mutuswami, S. (1997). Stable networks. Journal of Economic Theory, 76(2):322–344.

Ellison, G. (1994). Cooperation in the prisoner’s dilemma with anonymous random matching. Review of Economic Studies, 61(3):567–88.

Ely, J. C. and Valimaki, J. (2002). A robust folk theorem for the prisoner’s dilemma. Journal of Economic Theory, 102(1):84–105.

Friedman, J. W. (1971). A non-cooperative equilibrium for supergames. Review of Economic Studies, 38(113):1–12.

Fudenberg, D., Levine, D. I., and Maskin, E. (1994). The folk theorem with imperfect public information. Econometrica, 62(5):997–1039.

Fudenberg, D. and Maskin, E. (1986). The folk theorem in repeated games with discounting or with incomplete information. Econometrica, 54(3):533–554.

Fudenberg, D. and Maskin, E. (1990). Evolution and cooperation in noisy repeated games. American Economic Review, 80(2):274–279.

Galeotti, A. and Goyal, S. (2008). The law of the few. Mimeo. University of Essex.

Garicano, L. and Rossi-Hansberg, E. (2006). Organization and inequality in a knowledge economy. The Quarterly Journal of Economics, 121(4):1383–1435.

Granovetter, M. (1974). Getting a Job: A Study of Contacts and Careers. Chicago: Uni- versity of Chicago Press.

Green, E. J. and Porter, R. H. (1984). Noncooperative collusion under imperfect price information. Econometrica, 52(1):87–100.

Gul, F. (1989). Bargaining foundations of Shapley value. Econometrica, 57(1):81–95.

Haag, M. and Lagunoﬀ, R. (2006). Social norms, local interaction, and neighborhood planning. International Economic Review, 47(1):265–296.

Hippel, E. (1988). The Sources of Innovation. New York: Oxford University Press.

114 Hojman, D. A. and Szeidl, A. (2006). Endogenous networks, social games, and evolution. Games and Economic Behavior, 55(1):112–130.

Hojman, D. A. and Szeidl, A. (2008). Core and periphery in networks. Journal of Economic Theory, 139(1):295–309.

Holmstrom, B. (1982). Moral hazard in teams. Bell Journal of Economics, 13(2):324–340.

H¨orner,J. and Olszewski, W. (2006). The folk theorem for games with private almost- perfect monitoring. Econometrica, 74(6):1499–1544.

Jackson, M. O. and van den Nouweland, A. (2005). Strongly stable networks. Games and Economic Behavior, 51(2):420–444.

Jackson, M. O. and Watts, A. (2002). On the formation of interaction networks in social coordination games. Games and Economic Behavior, 41(2):265–291.

Jackson, M. O. and Wolinsky, A. (1996). A strategic model of social and economic networks. Journal of Economic Theory, 71(1):44–74.

Kandori, M. (1992a). Repeated games played by overlapping generations of players. Review of Economic Studies, 59(1):81–92.

Kandori, M. (1992b). Social norms and community enforcement. Review of Economic Studies, 59:63–80.

Kandori, M. and Matsushima, H. (1998). Private observation, communication and collusion. Econometrica, 66(3):627–652.

Keren, M. and Levhari, D. (1979). The optimum span of control in a pure hierarchy. Management Science, 25(11):1162–1172.

Kreps, D. M. and Wilson, R. (1982). Sequential equilibria. Econometrica, 50(4):863–894.

Mahoney, T. A. (1979). Organizational hierarchy and position worth. Academy of Manage- ment Journal, 22(4):726–737.

Mailath, G. J. and Morris, S. (2002). Repeated games with almost-public monitoring. Journal of Economic Theory, 102(1):189–228.

McAfee, R. P. and McMillan, J. (1995). Organizational diseconomies of scale. Journal of Economics and Management Strategy, 4:399–426.

Medoﬀ, J. L. and Abraham, K. G. (1981). Are those paid more really more productive? The case of experience. Journal of Human Resources, 16(2):186–216.

Melumad, N. D., Mookherjee, D., and Reichelstein, S. (1995). Hierarchical decentralization of incentive contracts. RAND Journal of Economics, 26(4):654–672.

Miyagawa, E., Miyahara, Y., and Sekiguchi, T. (2008). The folk theorem for repeated games with observation costs. Journal of Economic Theory, 139(1):192–221.

115 Mookherjee, D. and Reichelstein, S. (2001). Incentives and coordination in hierarchies. Advances in Theoretical Economics, 1(1). Available at: http://www.bepress.com/bejte/advances/vol1/iss1/art4.

Obara, I. (2007). Folk theorem with communication. Levine’s Bibliography 784828000000000351, UCLA Department of Economics.

Prat, A. (1997). Hierarchies of processors with endogenous capacity. Journal of Economic Theory, 77(1):214–222.

Qian, Y. (1994). Incentives and loss of control in an optimal hierarchy. Review of Economic Studies, 61(3):527–544.

Radner, R. (1993). The organization of decentralized information processing. Econometrica, 61(5):1109–1146.

Ray, D. and Vohra, R. (1999). A theory of endogenous coalition structures. Games and Economic Behavior, 26(2):286–336.

Renault, J. and Tomala, T. (1998). Repeated proximity games. International Journal of Game Theory, 27(4):539–559.

Salop, S. C. (1979). Monopolistic competition with outside goods. Bell Journal of Eco- nomics, 10(1):141–156.

Schulte, E. and Gr¨uner,H. P. (2007). Speed and quality of collective decision making: Imperfect information processing. Journal of Economic Theory, 127(1):138–154.

Sekiguchi, T. (1997). Eﬃciency in repeated prisoner’s dilemma with private monitoring. Journal of Economic Theory, 76(2):345–361.

Van Zandt, T. (2004). Balancedness of real-time hierarchical resource allocation. CEPR Discussion Papers 4276, C.E.P.R. Discussion Papers.

Williamson, O. E. (1967). Hierarchical control and optimum ﬁrm size. Journal of Political Economy, 75(2):123–138.

Xue, J. (2004). Essays on Cooperation, Coordination, and Conformity. PhD thesis, The Pennsylvania State University.

116 Vita Myeonghwan Cho Department of Economics Phone: (Oﬃce) 814 865 1108 608 Kern Graduate Building (Home) 814 862 2988 The Pennsylvania State University E-mail: [email protected] University Park, PA 16802 Homepage: http://my.win.psu.edu/muc152/

Citizenship Republic of Korea. Education Ph.D. Economics, The Pennsylvania State University, expected August 2008. M.A. Economics, Seoul National University, Korea, 2003. B.A. Economics, Seoul National University, Korea, 1997. Fields of Interest Primary: Microeconomics, Game Theory. Secondary: Industrial Organization, Public Economics. Ph.D. Thesis Essays on Networks and Hierarchies. Thesis Advisor: Professor Kalyan Chatterjee. Teaching Experience Teaching Assistant, The Pennsylvania State University, 2003 – 2007. Graduate (5 semesters): Introduction to Mathematical Economics, Microeconomic Analysis, Advanced Microeconomic Theory. Undergraduate (3 semesters): Intermediate Microeconomic Analysis, Strategy. Teaching Assistant, Seoul National University, Korea, 2001 – 2002. Undergraduate (4 semesters): Microeconomics, Fiscal Policy.

Research Experience Research Assistant: Kalyan Chatterjee, The Pennsylvania State University, 2007. Sophie Bade, The Pennsylvania State University, 2005. Youngsub Chun, Seoul National University, Korea, 2002 – 2003. Presentations 18th International Conference on Game Theory, Stony Brook University, 2007. PET07 Conference, Vanderbilt University, 2007. Micro Theory Seminar, The Pennsylvania State University, 2007. Honor and Fellowships Robert C. Daniels Graduate Fellowship, 2007 – 2008. Will E. Mason Memorial Endowment Award, 2007. David W. and Carolyn P. Grow Graduate Fellowship, 2006.