Classifications of Asynchronous Shared Memory Objects

David Yu Cheng Chan

A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science University of Toronto

Classiﬁcations of Asynchronous Shared Memory Objects

David Yu Cheng Chan Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2019

Since the early days of the shared memory model for distributed computing, researchers have sought a simple and precise characterization of an object’s ability to implement other objects in a wait-free manner.

The ﬁrst candidate for such a characterization was the consensus number of an object [24]. But a characterization based on consensus numbers is not precise: there are pairs of objects with the same consensus number that are not equivalent (i.e., one object cannot be wait-free implemented by instances of the other and registers) [2, 42].

A more recent candidate for such a characterization is the set agreement power of an object [15]. In this thesis, we show that this characterization is not precise even for the case of deterministic objects: there are pairs of deterministic objects with the same set agreement power that are not equivalent.

Next, we show that there are uncountably many objects with distinct set agreement power. Conse- quently, a precise characterization of an object’s ability to implement other objects in a wait-free manner must classify objects into uncountably many cells. This suggests that there is no simple and precise characterization of objects, since the cells of an uncountable classiﬁcation cannot be labeled using simple integers (as in Herlihy’s hierarchy) or even ﬁnite sequences of integers.

Finally, we introduce a generalization of consensus called bounded disagreement, which diﬀers from set agreement in that it restricts the number of disagreeing processes rather than the number of disagreeing values. More precisely, the `-bounded disagreement task for n processes has the following requirement: there is a value v such that at most ` processes (the disagreers) decide values that are not v. We then show that for every integer n ≥ 2, there is a bounded disagreement object that has consensus number n, but is not equivalent to the n-consensus object. Prior to this work, the only objects known to have this unusual characteristic for n ≥ 2 (which shows that the characterization of objects via consensus numbers is not precise) were artiﬁcial objects crafted solely for the purpose of exhibiting this behaviour [2, 42].

ii Acknowledgements

I thank my supervisors, Sam Toueg and Vassos Hadzilacos, for all their guidance and support. I thank the additional members of the supervisory committee, Allan Borodin and Eyal de Lara, for their feedback and encouragement. I also thank Maurice Herlihy, Aleksandar Nikolov, and Doug Stephan for being members of my ﬁnal oral exam committee. In addition, I thank the Natural Sciences and Engineering Research Council of Canada for their ﬁnancial support. I also thank the University of Toronto for allowing the use of their facilities for this research. Finally, I thank my family and friends for all of the support they have given me.

iii Contents

1 Introduction 1 1.1 The Imprecision of the Set Agreement Power Classification ...... 2 1.2 The Linearizability of Set Agreement ...... 2 1.3 On the Robustness of Object Classifications ...... 3 1.4 The Uncountability of the Precise Classification ...... 4 1.5 Bounded Disagreement ...... 4 1.6 Thesis Organization ...... 5

2 Model 6 2.1 Shared Memory Objects ...... 6 2.2 Non-Ported Objects ...... 8 2.3 Steps and Histories ...... 9 2.4 Linearizability ...... 9 2.4.1 Atomic Steps ...... 10 2.5 Wait-Freedom ...... 10 2.6 Object Implementations ...... 10 2.6.1 Ported Objects Versus Non-Ported Objects ...... 10 2.7 Problems and Problem Solutions ...... 11 2.7.1 The n-Consensus and (n, k)-Set Agreement Problems ...... 12

3 The Imprecision of the Set Agreement Power Classiﬁcation 13 3.1 n-P&G Objects ...... 14 3.2 An Equivalent n-P&G Object Sequential Speciﬁcation ...... 16 3.3 The Set Agreement Power of the n-P&G Object ...... 19 3.3.1 Lower Bound ...... 19 3.3.2 Upper Bound ...... 20 3.4 (n − 1)-Consensus Cannot Implement the n-P&G Object ...... 63 3.5 Objects Versus Tasks ...... 67 3.6 BG Simulation and Extensions ...... 67 3.7 Conclusion ...... 68

4 The Linearizability of Set Agreement 69

4.1 The An,k Object ...... 70

4.2 The Equivalence of the An,k Object and the (n, k)-Set Agreement Problem ...... 71

iv 4.3 The Linearizability of Sets of Set Agreement ...... 76

5 On the Robustness of Object Classiﬁcations 79 5.1 The Extended Set Agreement Partial Order Theorem ...... 79 5.2 The Robustness of Objects Equivalent to Their Set Agreement Power ...... 80 5.3 An Alternate Proof of the Imprecision of the Set Agreement Power Classiﬁcation . . . . . 82

6 The Uncountability of the Precise Classiﬁcation 84 6.1 Concluding remark ...... 87

7 Bounded Disagreement 88 7.1 Using Bounded Disagreement Problems to Solve Set Agreement Problems ...... 88 7.2 Bounded Disagreement Objects ...... 94 7.3 Unsolvability of (m − 2` + 1)-Consensus by (m, `)-BD Objects ...... 95 7.4 An Unusual Property of Bounded Disagreement ...... 98 7.4.1 BD Objects are not Equivalent to Rachman’s Objects ...... 99 7.4.2 BD Objects are not Equivalent to Afek et al.’s Objects ...... 99 7.5 Conclusion ...... 100

8 Conclusion 102

Bibliography 103

A A Comparison of Consensus Objects 107

v Chapter 1

Introduction

Since the early days of the asynchronous shared memory model for distributed computing, researchers have sought a simple and precise characterization of an object’s ability to implement other objects in a wait-free manner when processes may fail by crashing, i.e., stopping to take steps.1 The first candidate for such a characterization was based on the well-known consensus problem in which each of n processes has an input value and must decide on one of the input values so that all decision values are identical [17]. The consensus problem plays a central role in distributed computing, as many fundamental appli- cations of distributed computing require some form of consensus. For example, in distributed database systems, processes must agree (reach consensus) on the changes applied to the database, otherwise the database would become inconsistent. More generally, arbitrary objects can be shared among multiple processes using a “state machine approach” in which processes reach consensus on the sequence of operations applied to the shared object [36]. Although the consensus problem can be solved in synchronous systems, Fischer et al. [17] proved that in an asynchronous message passing system, the consensus problem cannot be solved among two or more processes even when at most one process can crash. Loui and Abu-Amara [39] then proved that the same result holds in an asynchronous shared memory system with only registers. On the other hand, they showed that when the system contains Test & Set objects, the consensus problem can be solved among two processes, but not among three or more processes.2 Naturally, this raises the issue of identifying which shared memory objects can be used to solve the consensus problem in an asynchronous shared memory system. Herlihy helped to address this issue by introducing the consensus hierarchy: a classification of shared memory objects into levels based on their ability to solve the consensus problem [24]. He defined the consensus number of an object as the maximum number of processes for which instances of the object and registers can solve the consensus problem, then placed each object into the level of the consensus hierarchy that corresponds to its consensus number. Herlihy then presented a result that illustrates the significance of an object’s consensus number: an object with consensus number n can implement any object shared by up to n processes. Thus the consensus number of an object served as an important characterization of an object’s ability to implement other objects in a wait-free manner.

1Unless otherwise speciﬁed, we consider objects, algorithms, and implementations that are wait-free [24, 35], i.e., processes that crash cannot block the other processes from progressing. Moreover, we assume that all algorithms and implementations can use registers. 2A Test & Set object returns 1 to the ﬁrst operation applied and 0 to all subsequent operations.

1 Chapter 1. Introduction 2

The consensus number of an object, however, is not a precise characterization of the object’s ability to implement other objects: some objects have the same consensus number but are not equivalent (we say two objects are equivalent if each can be implemented by instances of the other and registers in a wait-free manner). Herlihy [23] first showed that there are (non-deterministic) objects with consensus number 1 that are not equivalent to each other. Rachman [42] then showed that for every integer n ≥ 2, there are also non-deterministic objects with consensus number n that are not equivalent to each other. More recently, Afek et al. [2] showed that for every integer n ≥ 2, there are deterministic objects with consensus number n that are not equivalent to each other. Since the consensus number of an object does not fully characterize its ability to implement other objects, one may ask whether some natural extension of the consensus number, in particular the set agreement power of an object [15, 11], does so (a conjecture along these lines was given in [15]). The set agreement power of an object is the infinite sequence (n1, n2, . . ., nk,...) where nk is the maximum number of processes for which instances of the object and registers can solve the k-set agreement problem. In the (n, k)-set agreement problem [13], each of n processes has an input value and must decide on one of the input values such that there exist at most k distinct decision values. Note that the (n, 1)-set agreement problem is identical to the n-consensus problem, so the first component of the set agreement power of an object is simply its consensus number.

1.1 The Imprecision of the Set Agreement Power Classiﬁcation

Is the set agreement power of an object indeed a precise characterization of its ability to implement other objects? In this thesis, we prove that the answer is no. In fact, we prove the following stronger result: every level n ≥ 2 of Herlihy’s consensus hierarchy [24] has two deterministic objects with the same set agreement power that are not equivalent (i.e., one of these two objects cannot be implemented using instances of the other and registers). The objects at the core of our result are both fairly simple. The ﬁrst is just the n-consensus object, the standard object that allows up to n processes to solve the consensus problem directly. The second, which we call n-P&G (for “Propose&Get”), is an n-consensus object that we weaken as follows: (1) proposing a value and getting the agreed upon value are two separate operations, (2) roughly speaking, the object forces two of the n processes to compete to obtain the agreed upon value and returns ? to losers, and (3) the other n − 2 processes can get the agreed upon value unhindered. We give a precise deﬁnition of the n-P&G object in Section 3.1. We show that for all n > 2: (a) the n-P&G and (n−1)-consensus objects have the same set agreement power, but (b) the n-P&G object cannot be implemented using only (n − 1)-consensus objects and registers. Thus the deterministic n-P&G and (n − 1)-consensus objects have the same set agreement power but are not equivalent.

1.2 The Linearizability of Set Agreement

Our next result does not directly concern the classiﬁcation of shared memory objects, but is a useful step for subsequent results and somewhat surprising in its own right. We prove that for all n ≥ 1 and k ≥ 1, there is a linearizable object, denoted An,k, that is computationally equivalent to the (n, k)-set agreement problem in the following sense: the (n, k)-set agreement problem can be solved using the An,k Chapter 1. Introduction 3

object, and the An,k object can be implemented using any solution to the (n, k)-set agreement problem (and registers).

This is not obvious because the An,k object is not behaviourally equivalent to the (n, k)-set agreement problem. Indeed, any linearizable object for the (n, k)-set agreement problem imposes restrictions that are not inherent to this problem [41, 10]. These issues are discussed further in Chapter 4. We then extend this result to show that given any set C of set agreement problems, there exists a linearizable object O that is equivalent to C in the following sense: all of the problems in C can be solved using O, and O can be implemented using arbitrary solutions to all of the problems in C (and registers).

1.3 On the Robustness of Object Classiﬁcations

Our next result concerns the robustness of the consensus hierarchy and generalizations using set agreement. Roughly speaking, a hierarchy of objects is robust if objects at higher levels of the hierarchy cannot be implemented using instances of objects from lower levels of the hierarchy and registers [34, 38]. When considering the universe of shared memory objects, the consensus hierarchy is not robust [38]. However, the consensus hierarchy is robust for both the set of deterministic read-modify-write and readable objects [43] and the set of deterministic one-shot objects [27]. Notably, it remains an open question whether the consensus hierarchy is robust for the set of deterministic objects. In this thesis, we consider whether the consensus hierarchy (and its set agreement generalizations) is robust for the set of objects that are equivalent to their set agreement power. We say an object O is equivalent to its set agreement power if O is equivalent to the set of set S∞ agreement problems k=1{(nk, k)-set agreement}, where (n1, n2, . . . , nk,...) is the set agreement power of O. Then let USA be the set consisting of every object that is equivalent to its set agreement power. In some sense, USA is a generalization of the family of objects known as Common2 [3, 4], which is the set of objects that are equivalent to the 2-consensus object. Thus USA contains every object in Common2, which includes several common objects such as stack, swap, fetch&add, and test&set [3, 4].

+ + + + Let Z denote the set of all positive integers and let Z∞ = Z ∪ {∞}. For any ` ∈ Z∞, we can partition the set USA into equivalence classes such that two objects are in the same class if and only if the first ` components of their set agreement powers are the same; we call this an `-partition of USA, and denote it P`. Note that the 1-partition of USA is simply Herlihy’s consensus hierarchy restricted to contain only objects in USA. (Similarly, the ∞-partition of USA is simply the set agreement power classification restricted to contain only objects in USA.) + We prove that for all ` ∈ Z∞, the `-partition of USA is robust. Thus the consensus hierarchy (and its set agreement generalizations) is robust for USA. Since the consensus hierarchy is not robust [38], this implies that there are objects that are not in USA. This result, combined with the existence of linearizable set agreement objects, allows us to present a significantly simpler alternate proof that the set agreement power classification is imprecise. However, unlike the original proof, the alternate proof does not hold when the universe of objects is restricted to contain only deterministic objects, as the linearizable set agreement objects we defined are nondeterministic. Chapter 1. Introduction 4

1.4 The Uncountability of the Precise Classiﬁcation

Since the set agreement power of an object is not a precise characterization of its ability to implement other objects, the search continues for a precise object classification scheme, i.e., one that partitions the universe U of all shared objects so that two objects are equivalent if and only if they are in the same cell of the partition. As the next result of this thesis, we prove that there is an uncountable number of (linearizable) objects that are not equivalent to one another (and so they have distinct computational power). Thus any precise classification scheme of the universe U of objects contains an uncountable number of cells, even when the universe of objects is restricted to contain only linearizable objects. So the cells of a precise classification scheme cannot be labeled using simple integers (as in Herlihy’s hierarchy) or finite sequences of integers.

1.5 Bounded Disagreement

In this thesis we introduce and study another natural generalization of consensus, one that limits the number of processes that decide differently from the plurality; we call this the bounded disagreement (BD) problem. Recall that in the (n, k)-set agreement problem [13], each of n processes has an input value and must decide on one of the input values so that there exist at most k distinct decision values. For k = 1, the (n, k)-SA problem is simply the n-consensus problem; in this case, there is only one decision value and hence no disagreers. For k = 2, however, there are up to two decision values, but the number of disagreers (i.e., the processes deciding differently from the plurality) can suddenly jump all to the way to bn/2c. In some settings, it may be more important to limit the number of processes that disagree than limiting the number of different decision values. Thus we introduce the (n, `)-BD problem: each of n processes has an input value and must decide on one of the input values such that for some value v, at most ` processes (the disagreers) decide a value other than v. For ` = 0, the (n, `)-BD problem is just the n-consensus problem. Furthermore, for all ` ≥ 0, having at most ` disagreers implies having at most ` + 1 distinct decision values, so every solution to the (n, `)-BD problem also solves the (n, ` + 1)-SA problem. In this thesis we study the relationship between the consensus, set agreement and bounded disagreement problems, and show that despite their similarity, these problems are fundamentally different. To do so, we first determine values of n, `, m and k such that the (n, `)-BD problem and registers can solve the (m, k)-SA problem. We then determine the consensus number of every instance of the BD problem:3 specifically, we prove that the (n, `)-BD problem has consensus number max(n − 2`, 1). Using the above results and a previously-known impossibility result for set agreement [14], we prove that BD problems have the following uncommon property: for all j ≥ 2, there is a BD problem, namely the (9j, 4j)-BD problem, that has consensus number j but cannot be solved using j-consensus objects and registers. These results imply that there are infinitely many instances of the BD problem that are not equivalent to any consensus problem or any set agreement problem (because for k ≥ 2 all (m, k)-SA problems have consensus number 1).4

3The consensus number of a problem T is the maximum number of processes for which an arbitrary solution to T and registers can be used to solve the consensus problem. 4We say that two problems are equivalent if any solution to one problem, together with registers, can be used to solve the other problem. Similarly, we say that two objects are equivalent if instances of either object, together with registers, can be used to implement the other. Chapter 1. Introduction 5

We also define the (n, `)-BD object that solves the (n, `)-BD problem, as follows: intuitively, processes can “propose” any value to an (n, `)-BD object, and it responds to the first n such proposals with values that satisfy the Validity and `-BD properties; all subsequent proposals return arbitrary responses. It turns out that the above results about BD problems also apply to the corresponding BD objects. In particular, for all j ≥ 2, the (9j, 4j)-BD object has consensus number j, but cannot be implemented using j-consensus objects and registers. Prior to this thesis, however, the only objects known to have this unusual property for j ≥ 2 were artificial objects crafted solely for the purpose of exhibiting this behaviour [2, 42]. Unlike these objects, our BD objects and problems are natural generalizations of consensus. The BD objects and those in [42] are non-deterministic, while the objects in [2] are deterministic.

1.6 Thesis Organization

In Chapter 2, we describe the asynchronous shared memory model used in this thesis. In Chapter 3, we show that the set agreement power classification of shared memory objects is not precise. In Chapter 4, we present the results on the linearizability of set agreement. In Chapter 5, we present the results on the robustness of classifications of objects equivalent to their set agreement power. In Chapter 6, we show that of the precise classification of objects contains an uncountable number of equivalence classes. In Chapter 7, we introduce and analyze the bounded disagreement generalization of consensus. Finally, in Chapter 8, we conclude the thesis with some brief remarks. Chapter 2

Model

In this thesis, we use the asynchronous shared memory model where processes communicate only by performing operations on shared memory objects, and can fail by crashing, i.e., stopping to take steps. In our model, objects have ports [40]: each operation op is invoked on a speciﬁc port of an object, and the response it receives may depend on the port chosen. Processes can choose to invoke an operation on any port of any object, but no two operations may be applied on the same port of an object concurrently. If there are multiple pending operations on the same port of an object, which violates the semantics of a port, these operations are not required to terminate. Thus the number of ports of an object eﬀectively limits the number of processes that may access it concurrently. Henceforth, we consider only algorithms that are well-formed, i.e., algorithms that never apply concurrent operations on the same port of an object.

2.1 Shared Memory Objects

Formally, let P = (Σ, O) be an asynchronous shared memory system, where Σ = {p1, p2,...} is the (finite or infinite) set of processes, and O is the collection of shared memory objects supported by the system P . Each shared memory object O in O has a sequential specification that describes how the object behaves when accessed sequentially. This sequential specification is formally expressed as the tuple (Q, OP, RES, s0, n, δ), where:

• Q is the set of states that O can be in when accessed sequentially.

• OP is the set of operations that can be applied on a port of O.

• RES is the set of responses that O can return.

• s0 is the initial state of O.

• n is the number of ports that O has.

• δ ⊆ Q × OP × [1..n] × Q × RES is a state transition relation such that:

– A tuple (s, op, i, s0, res) is in δ if and only if when an operation op is applied on port i of O while O has state s, O can change its state to s0 and return res as the response.

6 Chapter 2. Model 7

– δ is total: for every reachable state s ∈ Q, every operation op ∈ OP, and every integer i ∈ [1..n], there exists at least one s0 ∈ Q and res ∈ RES such that (s, op, i, s0, res) ∈ δ.

Deﬁnition 2.1 (Deterministic Objects). An object O = (Q, OP, RES, s0, n, δ) is deterministic if and only if for every reachable state s ∈ Q, every operation op ∈ OP, and every integer i ∈ [1..n]:

There exists exactly one state s0 ∈ Q and one response value res ∈ RES (2.1) such that (s, op, i, s0, res) ∈ δ.

Deﬁnition 2.2 (Oblivious Objects). An object O = (Q, OP, RES, s0, n, δ) is oblivious if and only if for every pair of reachable states s ∈ Q and s0 ∈ Q, every operation op ∈ OP, and every response value res ∈ RES:

For all i, j ∈ [1..n], (s, op, i, s0, res) ∈ δ (2.2) if and only if (s, op, j, s0, res) ∈ δ.

Instead of a formal description of the above kind, we will usually describe the sequential specification of an object in pseudocode, using variables whose values define the state of the object, and procedures that describe how each operation modifies the object’s state and the value the operation returns. + For example, for all n ∈ Z∞, an n-consensus object is an object with n ports that support propose(v) operations for all v ∈ Z+.1 All propose(−) operations on an n-consensus object return the value proposed by the first propose(−) operation on the object. The state of an n-consensus object is a single variable val, with val = ⊥ in the initial state. When a propose(v) operation is performed on the object, it sets val to v if val still contains ⊥. Then it returns val. The sequential specification of the n-consensus object is given by Algorithm 1.

Algorithm 1 The sequential speciﬁcation of an n-consensus object. 1: procedure propose(v) applied to port i ∈ [1..n] 2: if val = ⊥ then 3: val ← v 4: return val

+ More precisely, for all n ∈ Z∞, the n-consensus object is a tuple (Q, OP, RES, s0, n, δ) where:

• Q = Z+ ∪ {⊥}

• OP = {propose(v): v ∈ Z+}

• RES = Z+

• s0 = ⊥

• For every reachable state s ∈ Q, every operation op ∈ OP, every integer i ∈ [1..n], every state s0 ∈ Q, and every response res ∈ RES, the tuple (s, op, i, s0, res) is in δ if and only if either

(a) s = ⊥, op = propose(res), and s0 = res, or (b) s0 = res = s 6= ⊥.

Note that the n-consensus object is deterministic (and oblivious).

1 + + + Recall that Z denotes the set of positive integers, and Z∞ = Z ∪ {∞}. Chapter 2. Model 8

2.2 Non-Ported Objects

There is an alternative model of objects that we call the non-ported model, because operations are applied directly on objects instead of via ports. Formally, the behaviour of a non-ported object when operations are applied to it sequentially is defined by a tuple (Q, OP, RES, s0, δ), i.e., the same as in ported objects except that the number of ports n is missing. The components of this tuple are as before, except that the state transition relation has the form δ ⊆ Q × OP × Q × RES, so the next state and the response of the object do not depend on the port. On one hand, non-ported objects are easier to use because processes do not need to coordinate with each other to ensure that no port is accessed by multiple concurrent operations. On the other hand, some objects are more naturally captured by the ported model. For example, a single-writer single- reader (SWSR) register is most naturally captured as an object with two ports, one of which supports only write operations, the other supports only read operations, and at any time at most one operation can be applied at each port. In this thesis, we adopt the ported model of objects because it is at least as general as the non-ported one in the following sense: every non-ported object has a ported counterpart with the same computing power (this is shown later in this chapter). Because of this fact, we can also use non-ported objects, and we do so when convenient. For example, throughout this thesis we will use non-ported registers that support read() and write(v) operations for any value v. The state of a register is a single variable val, with val = ⊥ in the initial state unless otherwise specified. When a read() operation is performed on the object, it returns val. When a write(v) operation is performed on the object, it sets val to v, and returns done. The sequential specification of the (non-ported) register is given by Algorithm 2.

Algorithm 2 The sequential speciﬁcation of a register. 1: procedure read 2: return val

3: procedure write(v) 4: val ← v 5: return done

More precisely, the register is a tuple (Q, OP, RES, s0, δ) where:

• Q = Z+ ∪ {⊥}

• OP = {write(v): v ∈ Z+} ∪ {read()}

• RES = Z+ ∪ {⊥} ∪ {done}

• s0 = ⊥

• For every reachable state s ∈ Q, every operation op ∈ OP, every state s0 ∈ Q, and every response res ∈ RES, the tuple (s, op, s0, res) is in δ if and only if either

(a) op = read() and s0 = res = s, or (b) op = write(s0) and res = done.

Note that the register is deterministic. Chapter 2. Model 9

2.3 Steps and Histories

A process p ∈ Σ that accesses an object O ∈ O does so in two steps: In the first step, denoted (inv, p, op, i, O), p invokes an operation op on a port i of O. In the second step, denoted (res, p, res, i, O), p receives a response res for op from O. The interval of time between the first and second step is called the execution interval of op. The response res received in the second step may differ depending on the steps taken by other processes in Σ, and thus the processes in Σ may communicate by accessing the objects in O. In addition to the above two kinds of steps, a process may also fail by crashing, more precisely a process p may take a crash step, denoted (crash, p), after which it cannot take any subsequent steps.

A history HP of the system P is a (possibly inﬁnite) sequence of invocation, response, and crash steps taken by the processes of Σ, where the invocation and response steps are taken on the objects of O. For O each object O ∈ O, we denote by HP the history of the object O in HP , i.e., the subsequence of steps in HP that are either steps on O or crash steps. We consider only systems where processes are sequential, in the sense that whenever a process invokes an operation, it must wait for the response before invoking another operation. So for all p ∈ Σ, O ∈ O, and i ∈ Z+, whenever process p takes a step of the form (inv, p, −, i, O) in the history HP , the earliest subsequent step of p in HP if it exists, is either of the form (res, p, −, i, O) or of the form (crash, p). Thus the subsequence of steps of each process p ∈ Σ in the history HP is a sequence of matching pairs of invocation and response steps, possibly followed by an unmatched invocation step, and then possibly followed by a crash step. If a process p ∈ Σ crashes in a history H (i.e., takes a crash step), then p is a faulty process in H; otherwise p is a correct process in H. For each operation op that is invoked in a history H, we say that op is complete in H if and only if the invocation step of op has a matching response step in H; otherwise op is incomplete in H. We say that a history H is complete if all operations invoked in H are complete in H.A completion H0 of a history H is a modiﬁcation of H to form a complete history as follows: for each incomplete operation op in H, either (a) the invocation step of op is removed, or (b) a matching response step for op is added after the invocation step of op and if the process that invokes op takes a crash step in H, the crash step is moved to after the added response step.

2.4 Linearizability

In this thesis, we focus on shared memory objects that are linearizable: every operation appears to take effect instantaneously at some time within its execution interval [30]. More precisely, let H be a history and O be an object such that the history HO of O in H is complete. A linearization L of HO is an assignment of a distinct linearization time t to every operation op of O in HO such that t is within the execution interval of op in HO. We say that L respects the sequential specification of O if the operation responses in HO could be those received when applying these operations sequentially, in the order dictated by L, on the object O. A history H is linearizable with respect to the sequential specification of an object O if HO has a completion H0O and a linearization L of H0O that respects the sequential specification of O. An object O is linearizable (with respect to its sequential specification specification) if every history H of O is linearizable with respect to the sequential specification of O. Chapter 2. Model 10

2.4.1 Atomic Steps

By definition, operations on linearizable objects appear to take effect instantaneously, i.e., an entire operation appears to occur in a single atomic step. Thus for convenience, given any history, operations on objects that are a priori known to be linearizable are not expressed in terms of invocation and response steps. Instead, a process p that accesses a linearizable object O does so in a single atomic step denoted (atom, p, op, res, i, O), in which p invokes an operation op on a port i of O, the operation appears to take effect instantaneously at the time of the atomic step, and the response res is returned to p.

2.5 Wait-Freedom

In this thesis, we focus on shared memory objects that are wait-free: every operation eventually receives a response if the process that invokes the operation does not crash. More precisely, we define an infinite history as a history that covers an infinite amount of time, thus allowing processes to execute their algorithms forever (or until they crash, or until their algorithms tell them to halt). Then an object O is wait-free if in every infinite history of O, every operation invocation step taken by a correct process has a matching response step.

2.6 Object Implementations

Consider an arbitrary (wait-free, linearizable) object O with n ports. An implementation of O is an algorithm that specifies for each opoeration op of O and each i ∈ [1..n] how to perform operation op on port i of O. An implementation of O is correct if every history of the implementation is indistinguishable from a history on O, i.e., every history of the implementation where accesses are well-formed is wait- free and linearizable with respect to the sequential specification of O. In this thesis, we consider only implementations that are correct. Consider an arbitrary implementation of O. We say that the objects used by this implementation to perform operations on O are the base objects of the implementation. For convenience, in this thesis we augment all implementation histories with the histories of their base objects. This can make properties such as linearizability significantly simpler to prove.

Deﬁnition 2.3. Given two objects O and O0, we say that O can be implemented using O0 and registers (or more simply O0 implements O), if there is a correct implementation of O that uses only O0 and registers as base objects.

Deﬁnition 2.4. Given two objects O and O0, we say that O is equivalent to O0 if and only if O implements O0 and O0 implements O.

2.6.1 Ported Objects Versus Non-Ported Objects

We now show that every non-ported object has a ported counterpart with the same computing power: given any non-ported object O, we can transform O into an (oblivious) ported object O0 that has an infinite number of ports, each supporting the same set of operations that O supports with the same + 0 sequential behaviour. More precisely, for all i ∈ Z∞, and for each tuple (s, op, s , res) in the state transition relation of the sequential specification of O,(s, op, i, s0, res) is in the state transition relation Chapter 2. Model 11 of the sequential specification of O0. It is straightforward to show that the resulting oblivious object O0 is equivalent to the original object O:

• To implement O0 using O, each process p performs an operation op on a port of O0 by simply performing the same operation op on O and returning the response.

• To implement O using O0, each process p performs an operation op on O by simply performing the same operation op on port p of O0 and returning the response.

Consequently, for each non-ported object O there exists an oblivious, inﬁnite-ported object O0 that is equivalent to O.

2.7 Problems and Problem Solutions

In a problem, every process in a distributed system starts with an input and, under certain conditions, it must compute an output. The problem can be speciﬁed by a set of properties that describe the allowable outputs and the conditions under which a process must produce an output. A solution to an n-process problem T is a set of protocols P[1..n] such that for all i ∈ [1..n], P[i] speciﬁes the actions process i should execute, where each action is either to perform an operation on some object or to output a value. A history of a solution to T is the sequence of steps taken by the n processes executing the protocols

P[1..n], augmented with two vectors ~vI and ~vO of length n that contain the inputs and outputs of the n processes (~vO contains the special value empty for each process that does not output any value within the history). A solution to T is correct if every history of the solution satisﬁes all properties of T . Henceforth when we say that A is a solution to a problem T , we mean that A is a correct solution to T .

Deﬁnition 2.5. Given a problem T and an object O, we say that T can be solved using O and registers (or more simply O solves T ), if there is a solution to T that uses only O and registers as base objects.

Roughly speaking, a solution P[1..n] to a n-process problem T can be thought of as a (not necessarily linearizable) object with n ports that support execute(v) operations, where the response to an execute(v) operation on port i is the output value of process i executing protocol P[i] with input v. Thus, there is a notion of implementing objects using solutions to problems:

Deﬁnition 2.6. Given an object O and a problem T , we say that O can be implemented using an arbitrary solution to T and registers (or more simply T implements O), if there is a correct implementation of O that uses only an arbitrary solution to T and registers as base objects.

Deﬁnition 2.7. Given an object O and a problem T , we say that O is equivalent to T if and only if O solves T and T implements O.

More generally:

Deﬁnition 2.8. Given two sets of objects and/or problems X and X 0, we say that X 0 implements X if for every object/problem X in X there is a correct implementation of X/solution to X that uses only objects in X 0, arbitrary solutions to problems in X 0, and registers as base objects.

Deﬁnition 2.9. Given two sets of objects and/or problems X and X 0, we say that X is equivalent to X 0 if and only if X implements X 0 and X 0 implements X . Chapter 2. Model 12

2.7.1 The n-Consensus and (n, k)-Set Agreement Problems

We now describe two problems to which we will refer often in this thesis. The n-consensus problem is a problem where each of n processes has an input value (its proposal) and must output a value (its decision) such that the following properties are satisﬁed:

• Termination: Every correct process eventually outputs a decision value.

• Validity: Each decision is one of the proposals.

• Agreement: All decisions are identical.

In the (n, k)-set agreement problem [13], the Agreement property of the n-consensus problem is replaced by:

• k-SA: There exist at most k distinct decision values. Chapter 3

The Imprecision of the Set Agreement Power Classiﬁcation

This chapter shows that the set agreement power classification of objects is not precise. In fact, we prove the following stronger result: every level n ≥ 2 of Herlihy’s consensus hierarchy [24] has two deterministic objects with the same set agreement power that are not equivalent (i.e., one of these two objects cannot be implemented using instances of the other and registers). The objects at the core of our result are both fairly simple. The first is just the n-consensus object, the standard object that allows up to n processes to solve the consensus problem directly. The second, which we call n-P&G (for “Propose&Get”), is an n-consensus object that we weaken as follows: (1) proposing a value and getting the agreed upon value are two separate operations, (2) roughly speaking, the object forces two of the n processes to compete to obtain the agreed upon value and returns ? to losers, and (3) the other n − 2 processes can get the agreed upon value unhindered. We show that for all n > 2: (a) the n-P&G and (n−1)-consensus objects have the same set agreement power, but (b) the n-P&G object cannot be implemented using only (n − 1)-consensus objects and registers. To prove (a), we show that the set agreement power of both the n-P&G object and the (n − 1)- consensus object is (n1, n2, . . . , nk,...) where nk = k(n − 1). For the (n − 1)-consensus object, this follows directly from results of Chaudhuri and Reiners [14]. For the n-P&G object, we first show that it solves the (n − 1)-consensus problem, and so it has set agreement power (n1, n2, . . . , nk,...) where nk ≥ k(n − 1). We then prove that it has set agreement power (n1, n2, . . . , nk,...) where nk ≤ k(n − 1), and so nk = k(n − 1). We do so by deriving a novel extension of the Borowsky-Gafni (BG) simulation algorithm [7, 8] that allows the simulated algorithm to use n-P&G objects (in addition to registers); we then apply this simulation algorithm to transform the known impossibility result that k + 1 processes cannot solve the k-set agreement problem using only registers [7, 8, 28, 44], into a new impossibility result that k(n − 1) + 1 processes cannot solve the k-set agreement problem using only n-P&G objects and registers, and so nk ≤ k(n − 1). To prove (b), we introduce a problem called the n-SLC problem and show that (i) the n-SLC problem can be solved using the n-P&G object, and (ii) the n-SLC problem cannot be solved using only (n − 1)- consensus objects and registers. To show (ii), we leverage a result of Hadzilacos and Toueg [22] that (n − 1)-consensus objects and registers cannot solve the n-DAC problem, a problem that is no harder

13 Chapter 3. The Imprecision of the Set Agreement Power Classification 14

Figure 3 Sequential speciﬁcation of the n-P&G object. + 1: procedure propose(v) applied to port i ∈ [1..n] { v ∈ Z } {* Denoted propose(v, i)*} 2: if val = ⊥ then 3: val ← v 4: return done

5: procedure get() applied to port i ∈ [1..n] {* Denoted get(i)*} 6: if i ≤ 2 and pri + i is odd then 7: pri ← pri + 1 8: return ? 9: else 10: return val than the n-SLC problem. Finally, by (a) and (b), the deterministic n-P&G and (n − 1)-consensus objects have the same set agreement power but are not equivalent. The organization of this chapter is as follows. In Section 3.1, we introduce our deterministic P&G objects. In Section 3.2, we present an equivalent deﬁnition of the P&G objects that is useful for linearization proofs. In Section 3.3, we prove that for every integer n ≥ 3, the n-P&G object and the (n − 1)-consensus object have the same set agreement power. In Section 3.4, we prove that an n-P&G object cannot be implemented using only (n − 1)-consensus objects and registers. In Section 3.5, we show that the deterministic n-P&G object is not equivalent to any task. We discuss some related BG simulation work in Section 3.6, and we conclude the chapter with some brief remarks in Section 3.7.

3.1 n-P&G Objects

In this section, we define the deterministic n-P&G objects for every integer n ≥ 3. Intuitively, an n-P&G object is a modified n-ported consensus object, where: (1) proposing a value and getting the agreed upon value are two separate operations, and (2) two of the n processes have to compete to get the agreed upon value, while the other processes can get the agreed value without competition. Roughly speaking, the competition works as follows. A process that tries to obtain the consensus value from port i ∈ {1, 2} of the object, must apply two consecutive get operations on that port: the first acquires a token that gives it the right to obtain the consensus value, and the second obtains the consensus value if the token was not taken away by another process. A process that applies a get operation on any port i > 2 of the object, obtains the consensus value immediately (without competing for the token). We now define the n-P&G object more precisely. The n-P&G object has n ≥ 3 ports. It supports operations propose(v) for v ∈ Z+, and get(), each of which can be applied to any port i ∈ [1..n]. A propose(v) operation applied to port i is denoted propose(v, i); similarly, a get() operation applied to port i is denoted get(i). The sequential specification of the n-P&G object describes how an n-P&G object behaves when it is accessed sequentially. It can be defined as a deterministic automaton (which has a set of states, a set of operations, a set of responses, and a deterministic state transition function) whose behavior is described in Figure 3. The state of an n-P&G object is a tuple (val, pri), such that:

• val ∈ Z+ ∪ {⊥} is initially ⊥. Chapter 3. The Imprecision of the Set Agreement Power Classification 15

• pri ∈ Z+ is initially 1.

The val field is used to store the agreed upon value, and it is set by the first propose(−, −) operation. The pri field is an integer counter that specifies which one of get(1) and get(2) has the token: if pri is odd then get(1) has the token, if it is even then get(2) has the token. From lines 6 to 10 of Figure 3, if a get(i) operation does not have the token and i ≤ 2, it changes the parity of the pri counter (to get the token for the next get(i) operation) and returns ?; otherwise, it simply returns the consensus value stored in val. Intuitively, a get(i) operation is prioritized if it has the token or i > 2 (so it has the right to obtain the consensus value). More precisely, a get(i) operation is prioritized if and only if, immediately before it is applied, either pri + i is even or i > 2. From Figure 3, it is easy to see that:

Lemma 3.1. The sequential speciﬁcation of an n-P&G object satisﬁes the following properties:

1. If a get(−) operation is preceded by a propose(−, −) operation, then it does not return ⊥.

2. All get(−) operations that do not return ⊥ or ? return the proposal value of the ﬁrst propose(−, −) operation.

3. For each i ∈ {1, 2}, if there is a time after which there are no get(i) operations, then only ﬁnitely many get(−) operations return ?.

4. For all i ∈ [3..n], no get(i) operation returns ?.

Proof. From Figure 3, it is clear that the val field contains ⊥ until the first propose(−, −) operation, which permanently sets the val field to its own proposal value. Furthermore, get(−) operations return either ? on line 8 or the value of the val field on line 10. Thus (1) and (2) hold. Next, note that a get(−) operation returns ? (line 8) only if it evaluates the conditional on line 6 as true. Thus (4) holds. Finally, to prove (3), suppose for contradiction that for some i ∈ {1, 2}, there is a time t after which there are no get(i) operations, yet infinitely many get(−) operations return ?. Let i0 ∈ {1, 2} be an integer such that i 6= i0 (i.e., i0 = 3 − i). Since there are no get(i) operations after time t and (4) holds, there must be infinitely many get(i0) operations that return ? after time t. Let op and op0 be the first and second get(i0) operations that return ? after time t. Since op returns ? (line 8), it found that pri + i0 is odd on line 6 and incremented pri on line 7. So pri + i0 becomes even when op is applied. From Figure 3, an operation sets the pri field (line 7) only if it evaluates the conditional on line 6 as true. Thus the pri field is only set by get(1) and get(2) operations. By definition, there are no get(i0) operations between op and op0. Furthermore, since op is applied after time t, there are no get(i) operations between op and op0. So the pri field is not set between op and op0, and thus pri + i0 is still even immediately before op0. Therefore op0 evaluates the conditional on line 6 as false — contradicting 0 the fact that op returns ? (line 8). Lemma 3.1

We consider n-P&G objects that are linearizable (with respect to their sequential specification): they behave as if all operations, including concurrent operations on different ports, are applied sequentially, so that each operation appears to take effect instantaneously at some distinct point between its invocation and response [30]. Chapter 3. The Imprecision of the Set Agreement Power Classification 16

3.2 An Equivalent n-P&G Object Sequential Speciﬁcation

In this section, we show that the sequential specification of the n-P&G object can also be given in terms of the properties that the object must satisfy when it is accessed sequentially. This is useful when we wish to prove the correctness of an implementation of an n-P&G object, as it suffices to show that these properties are satisfied by the implementation. To state these properties, we first define what it means for a get(−) operation to be “prioritized”.

Deﬁnition 3.2. Consider any n-ported object O that supports propose(−) and get() operations that can be applied to any port i ∈ [1..n] of this object. In a sequential history of O we say that:

• A get(2) operation op is prioritized if and only if it is preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op.

• A get(1) operation op is prioritized if and only if it is not preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op.

• Every get(i) operation with i > 2 is prioritized.

The sequential speciﬁcation of the n-P&G object in terms of properties is as follows.

Deﬁnition 3.3. In every sequential execution of an n-P&G object, the following properties hold:

1. If op is a prioritized get(−) operation and op is preceded by a propose(−, −) operation, then op returns the proposal value of the ﬁrst propose(−, −) operation.

2. If op is a prioritized get(−) operation and op is not preceded by a propose(−, −) operation, then op returns ⊥.

3. If op is an unprioritized get(−) operation, then op returns ?.

4. If op is a propose(−, −) operation, then op returns done.

We now show that the sequential speciﬁcation of the n-P&G object given in terms of the above properties is equivalent to the one given by the deterministic state automaton of Figure 3.

Theorem 3.4. Let H be a sequential history of an n-ported object O such that H contains only propose(v, i) and get(i) operations where v ∈ Z+ and i ∈ [1..n].1 Then H satisfies the sequential specification of the n-P&G object as given by the automaton of Figure 3 if and only if H satisfies the sequential specification of the n-P&G object as given by Definition 3.3.

Proof. We prove the following equivalent statement: either H violates both sequential specifications of the n-P&G object (Figure 3 and Definition 3.3), or H satisfies both sequential specifications. Suppose, for contradiction, that H violates exactly one of the two sequential specifications of the n-P&G object (Figure 3 or Definition 3.3). Then let op be the earliest operation in H that violates exactly one of the two sequential specifications of the n-P&G object (Figure 3 or Definition 3.3). By our assumption, all preceding operations do not violate either of the two sequential specifications of the n-P&G object (Figure 3 or Definition 3.3).

1Recall that propose(v, i) denotes a propose(v) operation on port i, and get(i) denotes a get() operation on port i. Chapter 3. The Imprecision of the Set Agreement Power Classification 17

Claim 3.4.1. For each get(i0) operation op0 such that op0 precedes op and i0 ≤ 2, pri + i0 is even immediately after op0.

Proof. Note that since op0 precedes op, it does not violate either of the two sequential speciﬁcations of the n-P&G object (Figure 3 or Deﬁnition 3.3). There are two cases: either pri + i0 is odd immediately before op0, or pri + i0 is even immediately before op0. In the former case, op0 evaluates the conditional on line 6 of Figure 3 as true and increments pri on line 7, so pri + i0 is even immediately after op0. In the latter case, op0 evaluates the conditional on line 6 as false and returns on line 10 without changing 0 0 pri, so pri + i is still even immediately after op . Claim 3.4.1

Claim 3.4.2. If op is a get(i) operation, then op is prioritized if and only if either i > 2 or pri + i is even immediately before op.

Proof. If i > 2, then op is prioritized by Deﬁnition 3.2. So it suﬃces to show that if i ≤ 2, then op is prioritized if and only if pri + i is even immediately before op. There are two cases: either there is at least one get(1) or get(2) operation that precedes op, or there is no get(1) or get(2) operation that precedes op.

Case 1. There is at least one get(1) or get(2) operation that precedes op. Then let op0 be the latest such operation, i.e., there are no get(1) or get(2) operations between op0 and op. Furthermore, let i0 ∈ {1, 2} be the port of O that op0 is applied on, i.e., op0 is a get(i0) operation. So by Claim 3.4.1, pri + i0 is even immediately after op0. Now consider the operation(s) between op0 and op. Since they precede op, they do not violate either of the two sequential specifications of the n-P&G object (Figure 3 or Definition 3.3). So according to Figure 3, since none of them are get(1) or get(2) operations, none of them change pri. Thus pri + i0 is still even immediately before op. Since i, i0 ∈ {1, 2}, pri + i is even immediately before op if and only if i = i0. Similarly, since there is a get(1) or get(2) operation that precedes op, Definition 3.2 implies that op is prioritized if and only if i = i0. Thus op is prioritized if and only if pri + i is even immediately before op.

Case 2 There is no get(1) or get(2) operation that precedes op. Consider the operation(s) that precede op. Since they precede op, they do not violate either of the two sequential specifications of the n-P&G object (Figure 3 or Definition 3.3). So according to Figure 3, since none of them are get(1) or get(2) operations, none of them change pri. Thus pri still contains its initial value 1 immediately before op. Since there is no get(1) or get(2) operation that precedes op, Definition 3.2 implies that op is prioritized if and only if i = 1, i.e., if and only if pri + i is even immediately before op.

So in both cases, op is prioritized if and only if pri + i is even immediately before op. Claim 3.4.2

There are 4 cases:

• op is a prioritized get(i) operation, and op is preceded by a propose(−, −) operation.

• op is a prioritized get(i) operation, and op is not preceded by a propose(−, −) operation.

• op is an unprioritized get(i) operation. Chapter 3. The Imprecision of the Set Agreement Power Classification 18

• op is a propose(−, −) operation.

In all cases, we will prove that either op violates both sequential specifications of the n-P&G object (Figure 3 and Definition 3.3), or op satisfies both sequential specifications — contradicting our assumption that op violates exactly one of the two sequential specifications of the n-P&G object.

Case 1. op is a prioritized get(i) operation, and op is preceded by a propose(−, −) operation. Then let op0 be the first propose(−, −) operation, and u be the proposal value of op0. Since op is preceded by a propose(−, −) operation, and op0 is the first propose(−, −) operation, op0 precedes op. Consider the operation(s) that precede op0. They also precede op, so they do not violate either of the two sequential specifications of the n-P&G object. Furthermore, since op0 is the first propose(−, −) operation, none of them are propose(−, −) operations. Thus according to Fig- ure 3, none of them changes val. So val still contains its initial value ⊥ immediately before op0.

Now consider the first propose(−, −) operation op0, and recall that u is its proposal value. Since op0 precedes op, it does not violate either of the two sequential specifications of the n-P&G object. Thus according to Figure 3, since val = ⊥ immediately before op0, op0 evaluates the conditional on line 2 as true and then writes its own proposal value u into val on line 3. So val contains u immediately after op0. Then consider the operation(s) between op0 and op. They also precede op, so they do not violate either of the two sequential specifications of the n-P&G object. Thus according to Figure 3, none of them set val back to ⊥. So all propose(−, −) operation(s) between op0 and op evaluate the conditional on line 2 as false and do not change val. Thus val still contains u immediately before op.

Finally, consider the prioritized get(i) operation op. First, consider the sequential specification of the n-P&G object as given by Definition 3.3. Since op is prioritized and op is preceded by a propose(−, −) operation, op should return u, the proposal value of the first propose(−, −) operation. Next, consider the sequential specification of the n-P&G object as given by the automaton of Figure 3. Since op is prioritized, by Claim 3.4.2, either i > 2 or pri + i is even immediately before op. Thus op should evaluate the conditional on line 6 as false, and so should return val on line 10, i.e., op should return u, the value of val immediately before op. Consequently, op either returns a value u0 6= u and violates both sequential specifications of the n-P&G object, or op returns u and satisfies both.

Case 2. op is a prioritized get(i) operation, and op is not preceded by a propose(−, −) operation. Consider the operation(s) that precede op. Since they precede op, they do not violate either of the two sequential speciﬁcations of the n-P&G object. Furthermore, since op is not preceded by a propose(−, −) operation, none of them are propose(−, −) operations. Thus according to Figure 3, none of them changes val. So val still contains its initial value ⊥ immediately before op. Chapter 3. The Imprecision of the Set Agreement Power Classification 19

Algorithm 4 Solving the (n − 1)-consensus problem using an n-P&G object D. + Code executed by each process q ∈ [1..n − 1] with input vq ∈ Z : 1: D.propose(vq) on port q + 1 2: repeat 3: val ← D.get() on port q + 1 4: until val 6= ? 5: decide val

Now consider the get(i) operation op. First, consider the sequential specification of the n-P&G object as given by Definition 3.3. Since op is prioritized and op is not preceded by a propose(−, −) operation, op should return ⊥. Next, consider the sequential specification of the n-P&G object as given by the automaton of Figure 3. Since op is prioritized, by Claim 3.4.2, either i > 2 or pri + i is even immediately before op. Thus op should evaluate the conditional on line 6 as false, and so should return val on line 10, i.e., op should return ⊥, the value of val immediately before op. Consequently, op either returns a value u0 6= ⊥ and violates both sequential specifications of the n-P&G object, or op returns ⊥ and satisfies both.

Case 3. op is an unprioritized get(i) operation. Since op is unprioritized, by Claim 3.4.2, i ≤ 2 and pri + i is odd immediately before op. So according to Figure 3, op should evaluate the conditional on line 6 as true and return ? on line 8. According to Definition 3.3, since op is unprioritized, op should return ?. Consequently, op either returns a value u0 6= ? and violates both sequential specifications of the n-P&G object, or op returns ? and satisfies both.

Case 4. op a propose(−, −) operation. Then according to Figure 3, op should return done on line 4. Also, according to Deﬁnition 3.3, op should return done. Consequently, op either returns a value u0 6= done and violates both

sequential speciﬁcations of the n-P&G object, or op returns done and satisﬁes both. Theorem 3.4

3.3 The Set Agreement Power of the n-P&G Object

We now show that the n-P&G object has the same set agreement power as the (n − 1)-consensus object. It is known that:

Theorem 3.5 (Chaudhuri and Reiners [14]). For all n > 2, the (n − 1)-consensus object has set agreement power (n1, n2, . . . , nk,...) such that, for all k ≥ 1, nk = k(n − 1).

3.3.1 Lower Bound

Theorem 3.6. For all n > 2, the (n − 1)-consensus problem can be solved using an n-P&G object.

Proof. For each process q ∈ [1..n − 1], let vq be the input value of process q. To solve the (n − 1)- consensus problem with an n-P&G object D, each process q executes Algorithm 4. Note that each process q ∈ [1..n−1] performs operations only on port q+1 of the n-P&G object D. Thus, this algorithm satisﬁes Chapter 3. The Imprecision of the Set Agreement Power Classification 20 the requirement that no two operations are applied on the same port of D concurrently. Furthermore, no process performs an operation on port 1 of D. In Algorithm 4, each process performs a propose(−, −) operation on D with its own input value (line 1), and repeatedly performs get(−) operations until it receives a non-? value (lines 2 to 4), which it then decides in line 5. We now show that the Validity, Agreement, and Termination properties of the consensus problem hold. Since each process performs a propose(−, −) operation before performing any get(−) operation, by Lemma 3.1(1) no get(−) operation on D returns ⊥. Thus, if a process decides a value v (in line 5), then v was returned by a get(−) operation in line 3, and v∈ / {⊥, ?}. So, by Lemma 3.1(2), v was proposed by the ﬁrst propose(−, −) operation on D, and therefore Validity and Agreement hold. Finally, since there are no get(1) operations on D, by Lemma 3.1(3), every correct process will eventually exit the loop of lines 2-4, and decide; so Termination holds. Theorem 3.6

It is clear that any solution for the (n − 1)-consensus problem can be used to implement the (n − 1)- consensus object. So the n-P&G object has at least the set agreement power of the (n − 1)-consensus object. Thus, by Theorems 3.5 and 3.6:

Corollary 3.7. For all n > 2, the n-P&G object has set agreement power (n1, n2, . . . , nk,...) such that, for all k ≥ 1, nk ≥ k(n − 1).

3.3.2 Upper Bound

It now suﬃces to show that for all n > 2, the n-P&G object has at most the set agreement power of the (n − 1)-consensus object, i.e., that it has set agreement power (n1, n2, . . . , nk,...) such that, for all k ≥ 1, nk ≤ k(n − 1).

Theorem 3.8. For all n > 2 and k ≥ 1, the (k(n − 1) + 1, k)-SA problem cannot be solved using n-P&G objects and registers.

Proof. Suppose, for contradiction, that there exists a wait-free algorithm AP that solves the (k(n − 1) + 1, k)-SA problem in a system P that has |P | = k(n − 1) + 1 processes, and n-P&G objects and registers. Without loss of generality, we assume that in system P , every process p ∈ [1..|P |] owns a single-writer multi-reader register, and there are no other registers.2

Since AP is wait-free, AP is well-formed, i.e., at every point in time of every history of AP , there is at most one pending operation on each port of each n-P&G object; otherwise multiple pending operations on the same port would violate the semantics of ports and not be required to terminate in a wait-free manner. Without loss of generality, we assume that AP protects each port of each n-P&G object that it accesses with a mutual exclusion algorithm that uses registers [16] (the single-writer multi-reader registers mentioned above). Note that since AP is well-formed, at every point in time of every history of AP , there is at most one process participating in the mutual exclusion algorithm of each port of each n-P&G object. Thus these mutual exclusion algorithms never block any process, and hence do not aﬀect the wait-freedom of AP . Therefore, these mutual exclusion algorithms are unnecessary, they only serve as auxiliary constructs that simplify our proof.

2It is known that arbitrarily many multi-writer multi-reader registers can be implemented from a single array of single- writer multi-reader registers, one for each process [33, 45]. Chapter 3. The Imprecision of the Set Agreement Power Classification 21

For convenience, we also assume without loss of generality that in AP , correct processes never halt.

Thus in an inﬁnite history of AP , all correct processes take inﬁnitely many steps.

Using the algorithm AP , we construct a wait-free algorithm AQ that solves the (k +1, k)-SA problem in a system Q that has |Q| = k + 1 processes and only registers — which is known to be impossible [7, 8,

28, 44]. Intuitively, in the algorithm AQ, the k + 1 real processes of the system Q simulate an execution of AP by the k(n − 1) + 1 virtual processes of the system P . In this simulated execution the input value of each virtual process is the input value of a real process. Each correct real process adopts the decision of some virtual process in the simulated execution of AP . The algorithm AQ is a novel extension of the Borowsky-Gafni (BG) simulation algorithm introduced in [7, 8]: the original BG simulation algorithm showed how to simulate the algorithm AP if AP only uses registers; since AP also uses n-P&G objects, the extension adds code to also simulate n-P&G objects.

The Simulation Algorithm AQ.

Intuitively, each real process q ∈ [1..|Q|] simulates an execution of the algorithm AP by the virtual processes in P . More precisely, for each virtual process p ∈ [1..|P |], each real process q has a thread qp that simulates the actions of virtual process p in AP .

To ensure that all real processes simulate the same execution of AP , they use safe agreement objects [7, 8], which are consensus-like objects that can be implemented using registers but are not wait-free. Safe agreement objects support propose(v) operations that return done, and decide() operations that return the agreed upon value. As long as each (real) process invokes a decide() operation only if it previously invoked a propose(−) operation, and each (real) process invokes at most one propose(−) operation, the following properties hold:

Agreement: If two invocations of decide() return v1 and v2 then v1 = v2.

Validity: If some invocation of decide() returns v then some process previously invoked propose(v).

Termination (a): Every propose(−) operation invoked by a correct process eventually completes.

Termination (b): If no process crashes during a propose(−) operation, then every decide() operation invoked by a correct process eventually completes.

The threads of the real processes that simulate (the execution of the algorithm AP by) a virtual process p use safe agreement objects to agree on the actions of p. Reaching this agreement, however, comes at a cost: by Termination (b), if a real process q crashes while one of its threads qp is executing 0 0 a propose(−) operation on a safe agreement object, the thread qp of every other real process q that simulates p could be stuck executing a non-terminating decide() operation on that object, thereby preventing progress in the simulation of virtual process p by all real processes. To prevent the crash of a real process q from blocking the progress of too many virtual processes, the threads of q use a local lock lockq to ensure that at most one of them is within a propose(−) operation on a safe agreement object at any time.3

Algorithm 5 shows an implementation of a safe agreement object Osafe that uses a {0, 1, 2}-level mechanism [7, 8]. This implementation uses a snapshot object, which is a linearizable object that can be implemented using registers [1]. Roughly speaking, a snapshot object is composed of an array of registers

3 Since lockq is local to process q, it does not require shared memory objects for its implementation. Chapter 3. The Imprecision of the Set Agreement Power Classification 22

Algorithm 5 Implementation of a safe agreement object Osafe by the real processes of system Q. Shared Memory: V [1..|Q|]: A snapshot object; all ﬁelds initially (⊥, 0).

A real process q ∈ [1..|Q|] performs propose(v) and decide() operations on the safe agreement object Osafe as follows: 1: procedure propose(v) 2: V [q].write((v, 1)) 3: snap ← V.snapshot() 4: if snap does not contain (−, 2) then 5: V [q].write((v, 2)) 6: else 7: V [q].write((v, 0)) 8: return done

9: procedure decide() 10: repeat 11: snap ← V.snapshot() 12: until snap does not contain (−, 1) 13: Letv ˜ be the smallest value s.t. (˜v, 2) ∈ snap. 14: return v˜ referred to as the fields of the snapshot object. Each field supports standard write(v) operations that replace the field’s current value with the parameter v, and the object as a whole supports snapshot() operations that return the current value of every field in the array.

In addition to safe agreement objects, the simulation algorithm AQ also requires base objects for simulating the n-P&G objects and registers used by the simulated algorithm AP . To simulate the registers of AP , we use an array of snapshot objects such that for each virtual process p ∈ [1..|P |], MEM[1..|Q|][p] is a snapshot object that simulates the register owned by p. To simulate an n-P&G object O, we use the following base objects:

• A snapshot object VALO[1..|Q|][1..|P |]:

Intuitively, VALO simulates the val ﬁeld of the n-P&G object O. In our simulation algorithm, 4 all ﬁelds of VALO are initially (⊥, 0). For each q ∈ [1..|Q|], the real process q owns (is the only

writer of) the ﬁeld VALO[q][p] for every p ∈ [1..|P |].

• A max register PRIO:

Intuitively, PRIO simulates the pri field of the n-P&G object O. Note that linearizable max registers can be implemented using registers [6]. Roughly speaking, max registers are modified registers that support readMax() and writeMax(v) operations instead of standard read() and write(v) operations of registers. readMax() is identical to read(), but writeMax(v) differs from write(v) in that the value of the register is replaced with v if and only if v is larger than the current value; otherwise the value of the register remains the same. Thus the value of a max register is simply the maximum of all values that have been written into it, including its initial

value. In our simulation algorithm, PRIO has initial value 1.

Since the snapshot and max register base objects are linearizable, we assume that they are atomic (i.e., operations on them are instantaneous).

4 + We consider ⊥ < v, for all v ∈ Z ∪ {0}. Chapter 3. The Imprecision of the Set Agreement Power Classification 23

Algorithm 6 The Algorithm AQ executed by the real processes of system Q to simulate the execution of the Algorithm AP by the virtual processes of system P . Shared Memory: MEM[1..|Q|][1..|P |]: For each p ∈ [1..|P |], MEM[1..|Q|][p] is a snapshot object, all ﬁelds initially (up, 0) where up is the initial value of the register owned by p in AP . SAFE[1..|P |][0..∞][1..∞]: An array of safe agreement objects, all initially in the fresh state. Max registers and snapshot objects for simulating each n-P&G object O, as listed in Algorithm 7. Local Variables for each real process q ∈ [1..|Q|]: For each virtual process p ∈ [1..|Q|], statep: state of p according to q. lockq: a lock that is held by at most one of the |P | threads of q at any time. + Code executed by each real process q ∈ [1..|Q|] with input vq ∈ Z : 1: Create local lock lockq. 2: For every virtual process p ∈ [1..|P |], create a thread qp. 3: for each p ∈ [1..|P |] do in parallel {* Thread qp: simulates q’s execution of virtual process p *} 4: lockq.acquire() 5: SAFE[p][0][1].propose(vq) 6: lockq.release() 7: vp ← SAFE[p][0][1].decide() 8: statep ← initial state of p in AP with input value vp. 9: for x ← 1 to ∞ do {* Simulate the x-th action of virtual process p in algorithm AP *} 10: Let op be the next action of p in statep according to algorithm AP . 11: case op is a write(u) operation on the register of p. 12: MEM[q][p].write((u, x)) 13: ret ← done 14: case op is a read() operation on the register of some p0 ∈ [1..|P |]. 15: snap ← MEM[1..|Q|][p0].snapshot() 16: y ← x0, where x0 is largest (−, x0) ∈ snap. 17: u ← v˜, such that (˜v, y) ∈ snap. 18: lockq.acquire() 19: SAFE[p][x][1].propose(u) 20: lockq.release() 21: ret ← SAFE[p][x][1].decide() 22: case op is a propose(v) or get() operation on some port i of some n-P&G object O. 23: Simulate the execution of op on port i of O as described by Algorithm 7. 24: ret ← response of O to op by this simulation. 25: case op is a decide(u) 26: if q has not decided then 27: decide u 28: ret ← done 29: statep ← the new state of p in AP after receiving ret from its x-th action

Algorithms 6 and 7 present the code for this extended simulation algorithm AQ, with accesses of the

local lock lockq highlighted in red. Algorithm 6 also displays in black the code of the BG simulation algorithm (as given in [37]), with our additional code for simulating n-P&G objects shown in blue and in Algorithm 7. For comparison, Algorithm 8 presents the code for the original BG simulation algorithm as given in [37].

For each real process q ∈ [1..|Q|] and each virtual process p ∈ [1..|P |], thread qp executes algorithm AQ as follows. First, thread qp acquires lockq to propose vq, the input value of the real process q, to a 0 safe agreement object (lines 4 to 8); this is to agree with every other thread qp on the input value vp of virtual process p. Then thread qp enters an unbounded loop where in each loop iteration x ≥ 1, thread qp simulates the x-th action of virtual process p according to algorithm AP . This action is simulated as follows.

• If the action of p simulated by qp is to write a value u to the register owned by p, thread qp writes the pair (u, x) into the ﬁeld MEM[q][p] of the snapshot object MEM[1..|Q|][p] (line 12). Chapter 3. The Imprecision of the Set Agreement Power Classification 24

Algorithm 7 Code executed to simulate a n-P&G object O. Shared Memory: VALO[1..|Q|][1..|P |]: A snapshot object; all ﬁelds initially (⊥, 0). PRIO: A max register; initially 1. Code executed by the thread qp of real process q to simulate the x-th action of virtual process p when this action is a propose(v) or get() operation on port i of O: + 30: procedure propose(v) applied to port i ∈ [1..n] { v ∈ Z } {* Denoted propose(v, i)*} 31: snap ← VALO.snapshot() 32: if all ﬁelds of snap are (−, 0) then 33: lockq.acquire() 34: VALO[q][p].write((v, 1)) 35: snap ← VALO.snapshot() 36: if snap does not contain (−, 2) then 37: VALO[q][p].write((v, 2)) 38: else 39: VALO[q][p].write((v, 0)) 40: lockq.release() 41: return done

42: procedure get() applied to port i ∈ [1..n] { i ∈ [1..n] } {* Denoted get(i)*} 43: for j ← 1 to ∞ do 44: val ← ? 45: snap ← VALO.snapshot() 46: if all ﬁelds of snap are (−, 0) then 47: val ← ⊥ 48: else if snap contains (−, 2) but not (−, 1) then 49: Letv ˜ be the smallest value s.t. (˜v, 2) ∈ snap 50: val ← v˜ 51: pri ← PRIO.readMax() 52: lockq.acquire() 53: SAFE[p][x][j].propose((val, pri)) 54: lockq.release() 55:( val, pri) ← SAFE[p][x][j].decide() 56: if i ≤ 2 and pri + i is odd then 57: PRIO.writeMax(pri + 1) 58: return ? 59: if val 6= ? then 60: return val

• If the action of p simulated by qp is to read a value from the register owned by a virtual process 0 p , thread qp ﬁrst ﬁnds the value u paired with the largest action number in the snapshot object 0 MEM[1..|Q|][p ] (lines 15 to 17). Thread qp then acquires lockq to propose u to a safe agreement object to ensure that all simulations of this read action of p read the same value (lines 18 to 21).

• If the action of p simulated by qp is to decide a value u, and the real process q has not yet decided, q also decides u (lines 26 to 27).

• If the action of p simulated by qp is to perform a propose(v, i) operation op on an n-P&G object

O (Algorithm 7, lines 30 to 41), then intuitively, thread qp competes with other threads to set the

val ﬁeld of O, which is simulated by the snapshot object VALO (lines 31 to 39). The mechanism used to win this competition is essentially the {0,1,2}-level mechanism used by safe agreement objects (as shown in Algorithm 5). More precisely:

1. Thread qp checks whether VALO already contains level 1 or 2 proposals (lines 31 to 32); if so,

thread qp simply returns done (line 41) without changing VALO.

2. Otherwise, thread qp acquires lockq (line 33). Chapter 3. The Imprecision of the Set Agreement Power Classification 25

Algorithm 8 The original BG simulation algorithm. Shared Memory: MEM[1..|Q|][1..|P |]: For each p ∈ [1..|P |], MEM[1..|Q|][p] is a snapshot object, all ﬁelds initially (up, 0) where up is the initial value of the register owned by p in AP . SAFE[1..|P |][0..∞]: An array of safe agreement objects, all initially in the fresh state.

Local Variables for each real process q ∈ [1..|Q|]: For each virtual process p ∈ [1..|Q|], statep: state of p according to q. lockq: a lock that is held by at most one of the |P | threads of q at any time.

+ Code executed by each real process q ∈ [1..|Q|] with input value vq ∈ Z : 1: Create local lock lockq. 2: For every virtual process p ∈ [1..|P |], create a thread qp. 3: for each p ∈ [1..|P |] do in parallel {* Thread qp: simulates q’s execution of virtual process p *} 4: lockq.acquire() 5: SAFE[p][0].propose(vq) 6: lockq.release() 7: vp ← SAFE[p][0].decide() 8: statep ← initial state of p in AP with input value vp 9: for x ← 1 to ∞ do {* Simulate the x-th action of virtual process p in algorithm AP *} 10: Let op be the next action of p in statep according to algorithm AP 11: case op is a write(u) operation on the register of p 12: MEM[q][p].write((u, x)) 13: ret ← done 14: case op is a read() operation on the register of some p0 ∈ [1..|P |] 15: snap ← MEM[1..|Q|][p0].snapshot() 16: y ← x0, where x0 is largest (−, x0) ∈ snap 17: u ← v˜, such that (˜v, y) ∈ snap 18: lockq.acquire() 19: SAFE[p][x].propose(u) 20: lockq.release() 21: ret ← SAFE[p][x].decide() 22: case op is a decide(u) 23: if q has not decided then 24: decide u 25: ret ← done 26: statep ← the new state of p in AP after receiving ret from its x-th action Chapter 3. The Imprecision of the Set Agreement Power Classification 26

3. Thread qp elevates its proposal v from level 0 to level 1 by writing (v, 1) into VALO[q][p] (line 34).

4. Thread qp checks whether VALO already contains level 2 proposals (lines 35 to 36). If not,

thread qp elevates its proposal v from level 1 to level 2 by writing (v, 2) into VALO[q][p]

(line 37); otherwise, thread qp reduces its proposal v from level 1 to level 0 by writing (v, 0)

into VALO[q][p] (line 39).

5. Thread qp releases lockq (line 40), and then returns done (line 41).

• If the action of p simulated by qp is to perform a get(i) operation on an n-P&G object O, thread

qp repeatedly attempts to obtain the state (val, pri) of O so that it can return the correct value.

To obtain the value of the pri ﬁeld of O from the max register PRIO, thread qp performs a

readMax() operation on PRIO (line 51).

The mechanism used to obtain the value of the val field of O from VALO is a modification of the {0,1,2}-level mechanism used by safe agreement objects, as we now explain. With a safe agreement object, a process attempting to obtain the value of the object by performing a decide() operation on that object must loop until there are no level 1 proposals. Thus if a proposal remains at level 1 forever, then every process performing a decide() operation may loop forever, so all decide() operations may never return. In our case, we take advantage of the fact that a get(i) operation that is not prioritized, i.e., one such that i ≤ 2 and pri + i is odd, does not need to find out the value of the val, because by the sequential specification of the n-P&G object, such a get(i)

operation simply returns ?. Thus in our modiﬁed {0,1,2}-level mechanism, the thread qp that simulates a get(i) operation by virtual process p only loops until either (a) there are no level 1 proposals, as in the original safe agreement mechanism, or (b) it ﬁnds that the get(i) operation is not prioritized (in which case it returns ?). So even if a proposal remains at level 1 forever, the get(i) operation may still be able to return. More precisely, to simulate a get(i) operation that is the x-th action of a virtual process p, thread

qp enters a loop where in each loop iteration j ≥ 1 (Algorithm 7, lines 42 to 60):

1. Thread qp attempts to determine the value of the val ﬁeld of Ofrom VALO as follows (lines 44 to 50): if there are only level 0 proposals, it sets val to ⊥ (line 47), indicating that the val ﬁeld of O still has its initial value ⊥; if there are level 2 proposals but no level 1 proposals, it sets val to the smallest value u such that u is a level 2 proposal (line 50); otherwise, val

remains ? (as initially set in line 44), indicating that so far thread qp has failed to ﬁnd the value of the val ﬁeld of O.

2. Thread qp determines the value of the pri ﬁeld of O by performing a readMax() operation

on PRIO and assigning the response to pri (line 51).

3. Thread qp acquires lockq to propose (val, pri) to a safe agreement object to ensure that all of the threads that execute the j-th loop iteration while simulating this get(i) operation agree on the (val, pri) state of O to use for this j-th loop iteration (lines 52 to 55).

4. Thread qp uses the agreed (val, pri) state of O for this j-th loop iteration to determine if this get(i) operation is prioritized (line 56).

– If it is not prioritized, thread qp increments the pri ﬁeld of O by performing a writeMax(pri+

1) operation on PRIO (line 57), and then returns ? (line 58). Chapter 3. The Imprecision of the Set Agreement Power Classification 27

– If it is prioritized and val 6= ? (line 59), thread qp returns val (line 60).

5. If thread qp does not return in the j-th loop iteration (line 58 or 60), it goes to the j + 1-th loop iteration.

The Simulated History HP of System P

We assume that in the system Q, there is at least one real process that is correct. Let HQ be an arbitrary inﬁnite history of AQ (Algorithm 6) by the k + 1 real processes in the system Q.

Roughly speaking, in the history HQ, because of the use of safe agreement objects, the simulations of each virtual process p by diﬀerent real processes cannot diverge: if two real processes simulate the x-th action of p, then that action is the application of the same operation on the same object and with the same return value. So intuitively, we can construct from HQ a well-deﬁned simulated history HP of the k(n − 1) + 1 virtual processes in the system P such that for each operation op of virtual process p:

• The execution interval of op begins at the earliest time that thread qp of any real process q begins simulating op.

• The execution interval of op ends at the earliest time that thread qp of any real process q completes

simulating op. (If no real process ever completes simulating op, then op is incomplete in HP .)

• If op is complete, then op returns the value returned by every complete simulation of op.

More formally, to deﬁne the simulated history of the system P , we must ﬁrst show that the safe agreement objects accessed by the the simulation algorithm AQ satisfy their Validity, Agreement, Termination (a), and Termination (b) properties.

Claim 3.8.1. For all p ∈ [1..|P |], x ∈ [0..∞], and j ∈ [1..∞], the safe agreement object SAFE[p][x][j] used in algorithm AQ (Algorithm 6) satisﬁes the Validity, Agreement, Termination (a), and Termina- tion (b) properties.

Proof. Given any p ∈ [1..|P |], x ∈ [0..∞], and j ∈ [1..∞], consider the safe agreement object SAFE[p][x][j]. By the speciﬁcation of safe agreement objects, it suﬃces to show that:

For each real process q ∈ [1..|Q|], each x ∈ [0..∞], and each j ∈ [1..∞], q performs a propose(−) operation on SAFE[p][x][j] before it performs a decide() operation on (3.1) SAFE[p][x][j], and q performs at most one propose(−) operation on SAFE[p][x][j].

First, note that SAFE[p][x][j] is accessed by real process q only via thread qp. There are two cases: either x = 0 or x ≥ 1.

Case 1. x = 0.

Then in algorithm AQ:

• Thread qp performs a propose(−) operation on SAFE[p][0][j] only on line 5.

• Thread qp performs a decide() operation on SAFE[p][0][j] only on line 7.

So it is clear that (3.1) is satisﬁed. Chapter 3. The Imprecision of the Set Agreement Power Classification 28

Case 2. x ≥ 1.

Then in algorithm AQ, thread qp accesses SAFE[p][x][j] only within the x-th loop iteration of

the for loop on lines 9 to 29. Consider the value that thread qp sets op to on line 10 of this loop iteration. There are three subcases: either op is a read() operation on the register of some p0 ∈ [1..|P |], op is a get(−) operation on some n-P&G object O, or op is neither.

Case 2(a). op is a read() operation on the register of some p0 ∈ [1..|P |]. First suppose j = 1. Then within the x-th loop iteration of the for loop on lines 9 to 29:

• Thread qp performs a propose(−) operation on SAFE[p][x][j] only on line 19.

• Thread qp performs a decide() operation on SAFE[p][x][j] only on line 21. Thus it is clear that if j = 1, (3.1) is satisﬁed.

Now suppose instead that j > 1. Then thread qp never accesses SAFE[p][x][j]. So it is vacuously true that (3.1) is satisﬁed.

Case 2(b). op is a get(−) operation on some n-P&G object O. Then within the x-th loop iteration of the for loop on lines 9 to 29:

• Thread qp performs a propose(−) operation on SAFE[p][x][j] only on line 53 within the j-th loop iteration of the for loop on lines 43 to 60.

• Thread qp performs a decide() operation on SAFE[p][x][j] only on line 55 within the j-th loop iteration of the for loop on lines 43 to 60. So it is clear that (3.1) is satisﬁed.

Case 2(c). op is neither a read() operation on the register of some p0 ∈ [1..|P |], nor a get(−) operation on some n-P&G object O.

Then thread qp never accesses SAFE[p][x][j]. So it is vacuously true that (3.1) is satisﬁed.

Claim 3.8.1

We now deﬁne what it means for a real process q to begin and to complete the simulation of an action of a virtual process p as follows:

Deﬁnition 3.8.2. Let q ∈ [1..|Q|] be a real process in system Q, p ∈ [1..|P |] be a virtual process in system P , x ≥ 1 be an integer such that thread qp of the real process q reaches the x-th loop iteration of x the for-loop on lines 9 to 29 of algorithm AQ. Consider this loop iteration. Let statep denote the value of statep at the beginning of this loop iteration for thread qp, and let op be the next action of virtual process x p in state statep according to algorithm AP .

• If thread qp executes line 10 of Algorithm 6 within this loop iteration, we say that thread qp begins its simulation of op, the x-th action of virtual process p, when it executes this line.

• If thread qp executes line 29 of Algorithm 6 within this loop iteration, and u is the value of ret

when thread qp executes this line, we say that thread qp completes its simulation of op, the x-th action of virtual process p, with return value u, when it executes this line.

Note that by this deﬁnition, whenever a thread begins its simulation of an action op, it has to complete its simulation of op before it can begin its simulation of another action. Chapter 3. The Imprecision of the Set Agreement Power Classification 29

Definition 3.8.3. For each real process q ∈ [1..|Q|] and each virtual process p ∈ [1..|P |], we define σq,p as the finite or infinite sequence of actions that the real process q (partially or completely) simulates for q,p the virtual process p in HQ. Thus for all x ≥ 1, σ [x] (if it exists) denotes the x-th action of virtual process p as simulated by real process q.

We now show that if a pair of real processes simulate some action x ≥ 1 of some virtual process p ∈ [1..|P |], they simulate the same action, and if their simulations eventually complete, they have the same return value. This will allow us to consider the sequence of actions performed by virtual process p independently from the real processes that simulate them.

Claim 3.8.4. Let q and q0 be real processes in [1..|Q|], p ∈ [1..|P |] be a virtual process, and x ≥ 1 be an 0 0 integer such that |σq,p| ≥ x and |σq ,p| ≥ x. Then the actions of σq,p[x] and σq ,p[x] are the same, and 0 0 if the simulations of σq,p[x] and σq ,p[x] are both complete, the return values of σq,p[x] and σq ,p[x] are the same.

Proof. Since virtual processes are deterministic in algorithm AP , for all q ∈ [1..|Q|], p ∈ [1..|P |], and x ≥ 1, the action of σq,p[x] is uniquely determined by

• The initial state that thread qp assigns for the virtual process p.

• The return values of the previous actions σq,p[1..x − 1].

Let q and q0 be arbitrary real processes in [1..|Q|]. Suppose, for contradiction, that for some integer 0 0 x ≥ 1 such that |σq,p| ≥ x and |σq ,p| ≥ x, either the actions of σq,p[x] and σq ,p[x] differ, or the 0 0 simulations of σq,p[x] and σq ,p[x] are both complete, but the return values of σq,p[x] and σq ,p[x] differ. 0 Letx ˜ be the smallest such positive integer, i.e., for all x ∈ [1..x˜ − 1], the actions of σq,p[x] and σq ,p[x] 0 are the same, and the return values of σq,p[x] and σq ,p[x] are the same. 0 By using a safe agreement object (lines 5 and 7), both threads qp and qp assign the same initial state 0 for the virtual process p. Thus the actions of σq,p[˜x] and σq ,p[˜x] are the same. So by our choice ofx ˜, the 0 0 simulations of σq,p[˜x] and σq ,p[˜x] are both complete, but the return values of σq,p[˜x] and σq ,p[˜x] differ. 0 Consider the actions of σq,p[˜x] and σq ,p[˜x]. There are 3 cases:

0 Case 1. The actions of σq,p[˜x] and σq ,p[˜x] are either both decide(−) actions, both write(−) operations on a register, or both propose(−, −) operations on an n-P&G object. q,p Then observe that according to line 28, 13, or 41 of algorithm AQ, the return values of σ [˜x] 0 and σq ,p[˜x] are both done — contradicting our assumption that the return values of σq,p[˜x] and 0 σq ,p[˜x] diﬀer.

0 Case 2. The actions of σq,p[˜x] and σq ,p[˜x] are read() operations on a register. q,p q0,p Then observe that according to line 21 of algorithm AQ, the return values of σ [˜x] and σ [˜x] are both returned by decide() operations on the safe agreement object SAFE[p][˜x][1]. By Claim 3.8.1, the Agreement property holds for SAFE[p][˜x][1], so these decide() operations on SAFE[p][˜x][1] 0 return the same value. Thus the return values of σq,p[˜x] and σq ,p[˜x] are the same — contradicting 0 our assumption that the return values of σq,p[˜x] and σq ,p[˜x] diﬀer.

0 Case 3. The actions of σq,p[˜x] and σq ,p[˜x] are get(−) operations on an n-P&G object O. Then let i ∈ [1..n] be the port of O on which these operations are applied. i.e., the actions of q,p q0,p q,p σ [˜x] and σ [˜x] are get(i) operations on O. Since thread qp completes its simulation of σ [˜x], Chapter 3. The Imprecision of the Set Agreement Power Classification 30

it executes either line 58 or line 60 of algorithm AQ in some loop iteration j of its simulation of q,p 0 the action of σ [˜x]. Similarly, thread qp executes either line 58 or line 60 of algorithm AQ in 0 some loop iteration j0 of its simulation of the action of σq ,p[˜x]. Without loss of generality, suppose j ≤ j0.

q,p q0,p Consider the j-th loop iteration of the simulations of the actions of σ [˜x] and σ [˜x] by threads qp 0 and qp respectively. On line 55, both threads perform a decide() operation on SAFE[p][˜x][j], and assign the response to (val, pri). By Claim 3.8.1, the Agreement property holds for SAFE[p][˜x][j],

so both threads assign the same values to (val, pri) on line 55. There are two cases: thread qp

executes either line 58 or line 60 of algorithm AQ.

Case 3(a). Thread qp executes line 58 of algorithm AQ.

So thread qp evaluates the conditional on line 56 as true, i.e., it ﬁnds that i ≤ 2 and pri + i is 0 odd. Then, since thread qp has the same values of i and pri, it also evaluates the conditional 0 q,p on line 56 as true. So thread qp also executes line 58. Thus the return values of σ [˜x] 0 and σq ,p[˜x] are both ? — contradicting our assumption that the return values of σq,p[˜x] and 0 σq ,p[˜x] diﬀer.

Case 3(b). Thread qp executes line 60 of algorithm AQ.

So thread qp evaluates the conditional on line 56 as false and the conditional on line 59 as 0 true, i.e., it ﬁnds that val 6= ?, and either i > 2 or pri + i is even. Then, since thread qp has the same values of val, i, and pri, it also evaluates the conditional on line 56 as false and the 0 conditional on line 59 as true. So thread qp also executes line 60. Thus the return values of q,p q0,p 0 σ [˜x] and σ [˜x] are both the value that both threads qp and qp assigned to val on line 55. 0 So the return values of σq,p[˜x] and σq ,p[˜x] are the same — contradicting our assumption that q,p q0,p the return values of σ [˜x] and σ [˜x] diﬀer. Claim 3.8.4

Definition 3.8.5. For each virtual process p ∈ [1..|P |], we define σp = σq,p where q is the real process such that σq,p is of maximum length. This is well-defined by Claim 3.8.4.

Deﬁnition 3.8.6. We construct a simulated history HP of the virtual processes in P executing AP from the real history HQ of the real processes in Q executing AQ as follows. For every virtual process p ∈ [1..|P |] and every positive integer x ≤ |σp|, if the x-th action of σp is an operation op on any object, we add op to HP such that:

• The execution interval of op begins at the earliest time when the thread qp of any real process q,p p q ∈ [1..|Q|] begins simulating σ [x] (line 10 of algorithm AQ). Note that since |σ | ≥ x, by Deﬁnition 3.8.2 and Deﬁnition 3.8.3, such a time exists.

• The execution interval of op ends at the earliest time when the thread qp of any real process q,p q ∈ [1..|Q|] completes simulating σ [x] (line 29 of algorithm AQ). Note that if no such time exists, then op is an incomplete operation whose execution interval has no end.

• If op is complete, then op returns the value that is returned by all complete simulations of op.

In addition, for every virtual process p ∈ [1..|P |]:

• If there is a complete decide() operation on SAFE[p][0][1] (line 7), the input value of p in HP is set to the response value of the decide() operation on SAFE[p][0][1] that completes earliest.

Otherwise, the input value of p in HP is set to the input value of an arbitrary real process. Chapter 3. The Imprecision of the Set Agreement Power Classification 31

p • If there is a decide(−) action in σ , then the output value of p in HP is set to the decided value p of the ﬁrst decide(−) action in σ . Otherwise, the output value of p in HP is left blank.

p • If |σ | is not inﬁnite, we add a crash step for p to HP at the earliest time after which p takes no steps.

The following is a consequence of the fact that at most one step is taken at any time:

Claim 3.8.7. Given any time t:

• No operation in the history HP is both invoked and completed at time t.

• There is at most one operation op in the history HP such that op is either invoked or completed at time t.

Proof. We ﬁrst prove that no operation in the history HP is both invoked and completed at time t.

Letop ˆ be an operation in HP that completes at time t; if no such operation exists, we are done. By p Deﬁnition 3.8.6, sinceop ˆ is in HP , there is a virtual process p and a positive integer x ≤ |σ | such p that the x-th action of σ isop ˆ . Furthermore, at time t, the thread qp of some real process q ∈ [1..|Q|] q,p 0 completes simulating σ [x] (line 29 of algorithm AQ). So at some time t < t, this thread qp has to q,p begin simulating σ [x] (line 10 of algorithm AQ). By Deﬁnition 3.8.6,op ˆ is invoked at the earliest time 0 0 q0,p when the thread qp of any real process q ∈ [1..|Q|] begins simulating σ [x] (line 10 of algorithm AQ). 0 Thusop ˆ is invoked at some time tˆ≤ t < t. Soop ˆ is not invoked at time t. Thus no operation in HP is both invoked and completed at time t.

We now prove that there is at most one operation op in the history HP such that op is either invoked or completed at time t. Suppose, for contradiction, that there are two distinct operations op and op0 0 such that each of op and op is either invoked or completed at time t in HP .

First consider op. Since op is either invoked or completed at time t in HP , by Deﬁnition 3.8.6, some real process executes some step (namely line 10 or line 29) of algorithm AQ in its simulation of op at time 0 t. By a symmetric argument, since op is either invoked or completed at time t in HP , some real process 0 0 takes a step (i.e., executes a line of algorithm AQ) in its simulation of op at time t. Since op 6= op , the steps taken in the simulation of op are distinct from the steps taken in the simulation of op0. Thus there are at least two distinct steps taken at time t — contradicting the fact that at most one step is taken at any time. Thus there is at most one operation op in the history HP such that op is either invoked or completed at time t. Claim 3.8.7

Claim 3.8.8. HP is well-formed, i.e., at any time t, there is at most one pending operation on each port of each n-P&G object.

Proof. By deﬁnition, an operation is pending at time t if it is invoked at or before time t and not completed before time t. Recall that AP protects each port i ∈ [1..n] of each n-P&G object O with a unique lock (implemented using registers) that each virtual process p in the system P acquires immediately before performing an operation on port i of O, and releases immediately after. Since the original BG simulation algorithm correctly simulates registers [7, 8, 37], our extension AQ also correctly simulates the registers used to implement these locks. So accesses to these locks by the virtual processes in system P are correctly simulated by the threads of the real processes in system Q. Thus at any time t, there is at most one pending operation on each port of each n-P&G object. Claim 3.8.8 Chapter 3. The Imprecision of the Set Agreement Power Classification 32

Recall that AP is a wait-free algorithm that solves the k-set agreement problem. Thus, to prove that the algorithm AQ, which simulates AP , solves the k-set agreement problem, we will prove the following properties of AQ:

(a) Safety: HP is a history of AP in which the input value of each virtual process p is the input of

some real process q in HQ (Claim 3.8.53).

(b) Liveness: If HQ is inﬁnite then every correct real process eventually decides in HQ,

As we will see (a) implies that AQ satisfies the Agreement and Validity properties of the k-set agreement problem, and (b) implies that AQ satisfies the Termination property of the k-set agreement problem (Claim 3.8.54). O For (a) we must show that, for each register and n-P&G object O used by AP , the history HP of operations on O in HP is linearizable with respect to the sequential specification of O — i.e., that O is correctly simulated in HP . This is true for the registers used by AP , since AQ simulates them as in the original BG simulation [7, 8, 37]. We will show that it is also true for the n-P&G objects used by AP (Claim 3.8.52).

For (b) we show that, if HQ is inﬁnite, every correct real process q simulates an inﬁnite number of steps by some virutal process p in HP (Claim 3.8.27). Then, by (a), p eventually decides in HP and so q decides in HQ. Our proof of these facts requires a long sequence of intermediate steps.

Properties of n-P&G Objects

We now prove some basic properties about the n-P&G objects accessed by the virtual processes in HP .

Claim 3.8.9. Let q ∈ [1..|Q|] be a real process, p ∈ [1..|P |] be a virtual process, O be an n-P&G object, 0 and v be a value. If at some time t, VALO[q][p] contains (v, 2), then at any time t ≥ t, VALO[q][p] contains (v, 2).

Proof. Initially, VALO[q][p] = (⊥, 0). Thus (v, 2) is written into VALO[q][p] before or at time t. In Algo- rithm 7, this only happens when thread qp executes line 37 within its simulation of some propose(−, −) operation op on O. Thread qp does not subsequently perform any more writes on VALO[q][p].

In Algorithm 7, only thread qp writes to VALO[q][p], and only within its simulation of propose(−, −) 0 operations on O. So consider each propose(−, −) operation op on O that thread qp simulates after 0 op, in chronological order. Thread qp begins simulating op by taking a snapshot of VALO (line 31). 0 Since thread qp has not written to VALO[q][p] within its simulation of op , this snapshot contains the

(v, 2) that is still in VALO[q][p]. Thus thread qp evaluates the conditional on line 32 to be false, and 0 simply returns done (line 41) without writing to VALO[q][p]. So at all times t ≥ t, VALO[q][p] contains

(v, 2). Claim 3.8.9

Deﬁnition 3.8.10. For every n-P&G object O, tO is the earliest time when VALO contains (−, 1); or ∞ if no such time exists.

Note that for each n-P&G object O, VALO initially contains only (−, 0), and (−, 1) is written into

VALO only when a simulation of an operation on O executes line 34. Thus: Chapter 3. The Imprecision of the Set Agreement Power Classification 33

Observation 3.8.11. For every n-P&G object O, tO is the earliest time when a simulation of an operation on O executes line 34; or ∞ if no such time exists.

Claim 3.8.12. For each n-P&G object O, all ﬁelds of VALO are (⊥, 0) at any time t < tO.

Proof. Initially, all ﬁelds of VALO are (⊥, 0). In Algorithm 7, only simulations of propose(−, −) operations on O perform write(−) operations on VALO (lines 34, 37, and 39). Furthermore, (−, 1) is written to VALO (line 34) before any other value in any simulation of any propose(−, −) operation on

O. By Deﬁnition 3.8.10, tO denotes the earliest time when (−, 1) is written to VALO (line 34); or ∞ if no such time exists. Thus every write(−) operation on VALO happens at or after tO. So at any time before tO, all ﬁelds of VALO are (⊥, 0). Claim 3.8.12

Claim 3.8.13. For each n-P&G object O, at least one ﬁeld of VALO does not contain (−, 0) at any time t ≥ tO.

Proof. If tO = ∞, then the claim is vacuously true. So suppose tO 6= ∞. Then by Deﬁnition 3.8.10,

VALO contains (−, 1) at tO. Since all ﬁelds of VALO initially contain (⊥, 0), there exist q ∈ [1..|Q|] and p ∈ [1..|P |] such that thread qp writes (−, 1) to VALO[q][p] at time tO. In Algorithm 7, only thread qp performs write operations on VALO[q][p]. Thus if thread qp never writes (−, 0) to VALO[q][p] after tO, then VALO[q][p] contains (−, 1) instead of (−, 0) at all times after tO.

It remains to consider the case where thread qp writes (−, 0) to VALO[q][p] after tO. Consider the

ﬁrst time it does so after tO. In Algorithm 7, this only happens when thread qp executes line 39 within its simulation of some propose(−, −) operation op on O. Earlier in its simulation of op, thread qp must have done the following:

1. Evaluated the conditional on line 36 to be false, so snap contains at least one (−, 2) when thread

qp evaluates the conditional on line 36.

2. Before doing so, qp took a snapshot of VALO and assigned it to snap on line 35, so VALO contains

at a least one (−, 2) when thread qp evaluates the conditional on line 35.

0 Let t > tO be the time when qp executes line 35 within its simulation of op. So VALO contains (−, 2) 0 0 0 at time t , as well as at all times after t by Claim 3.8.9. Furthermore, between time tO and time t ,

VALO[q][p] does not contain (−, 0) since thread qp has not yet written (−, 0) to VALO[q][p]. Thus at any time t ≥ tO, at least one ﬁeld of VALO does not contain (−, 0). Claim 3.8.13

Claim 3.8.14. For every n-P&G object O, every simulation of a propose(−, −) operation op on O that is invoked at time t ≥ tO does not write anything into VALO.

Proof. If tO = ∞, then the claim is vacuously true. So suppose tO 6= ∞, and let op be a propose(−, −) operation on O that is invoked at time t ≥ tO. Let p be the virtual process that invokes op on O. For each real process q ∈ [1..|Q|], consider the simulation of op by thread qp.

Thread qp begins the simulation of op by taking a snapshot of VALO (line 31). Since op is invoked at time t ≥ tO, by Claim 3.8.13 this snapshot contains at least one ﬁeld that is not (−, 0). Thus qp evaluates the conditional on line 32 as false, and simply returns done (line 41) without writing anything into VALO. Claim 3.8.14

We now prove the liveness and safety of our simulation algorithm. Chapter 3. The Imprecision of the Set Agreement Power Classification 34

Liveness of the Simulation Algorithm

We now show that every correct real process in the history HQ of the simulation algorithm AQ simulates infinitely many steps of some virtual process in the history HP of the simulated algorithm AP . We first outline the proof of this liveness of the simulation algorithm and then present it in detail. Outline. Let q be an arbitrary correct real process, and recall that it has |P | = k(n − 1) + 1 threads, each simulating a distinct virtual process. We define the critical parts of the simulation as the lines of the simulation code that are protected by a local lock, namely, lines 5, 19, 53, and lines 34 to 39. First, suppose no real process crashes within a critical part of the simulation. Then it is easy to show that for each virtual process p, thread qp simulates infinitely many steps of p. Roughly speaking, this is because in the simulation algorithm AQ, thread qp can only become stuck in two ways, both of which require some real process q0 to crash within a critical part of the simulation:

• For some integers x ≥ 0 and j ≥ 1, thread qp could invoke a decide() operation that never completes on the safe agreement object SAFE[p][x][j] (in line 7, 21, or 55). This however, requires a real process to crash while performing a propose(−) operation on SAFE[p][x][j] (in line 5, 19, or 53).

• Within the simulation of some get(−) operation invoked by p on some n-P&G object O, thread

qp could execute inﬁnitely many loop iterations of the for loop on lines 43 to 60. This however,

requires a real process to crash while it has a level 1 proposal in VALO (within lines 34 to 39).

Otherwise, thread qp will eventually ﬁnd the non-? value of the val ﬁeld of O, and return it on line 60.

Our goal is to show that even if all |Q| − 1 = k real processes other than q crash, at least one of the |P | = k(n − 1) + 1 threads of q is not blocked, i.e., at least one of the threads of q simulates infinitely many steps of some virtual process. So intuitively, it suffices to show that each real process crash blocks at most n − 1 threads of q. 0 In the simulation algorithm AQ, every real process q uses its local lock lockq0 to ensure that at most one of its threads is within a critical part of the simulation at any time. Thus, if a real process q0 crashes, it does so within at most one critical part of the simulation. There are two cases: either (1) this critical part involves a safe agreement object (line 5, 19, or 53), or (2) this critical part is within the simulation of a propose(−, −) operation on an n-P&G object O (lines 34 to 39). First consider case (1); i.e., suppose that a real process q0 crashes within a critical part involving a safe agreement object SAFE[p][x][j] for some virtual process p and some integers x ≥ 0 and j ≥ 1 (line 5, 19, or 53). This crash may leave an incomplete propose(−) operation that could cause a decide() operation invoked by thread qp on SAFE[p][x][j] to never complete; so this crash would block thread 0 qp. However, it would not affect any other thread qp0 of q, since thread qp0 for p 6= p does not access SAFE[p][x][j]. So if a real process crashes within a critical part involving a safe agreement object, at most one thread of q is blocked. Next, consider case (2); i.e., suppose that a real process q0 crashes within the critical part of the simulation of a propose(−, −) operation on an n-P&G object O (lines 34 to 39). This crash may leave a level 1 proposal in VALO forever. We claim that this may block at most n − 1 threads of q. Suppose, for contradiction, that n threads of q are blocked. Each of these n threads of q is stuck in a simulation of a distinct get(−) operation on O: it is executing infinitely many loop iterations of the for loop on Chapter 3. The Imprecision of the Set Agreement Power Classification 35

lines 43 to 60 because, in every iteration, the level 1 proposal left in VALO prevents it from ﬁnding the value of the val ﬁeld of O. By Claim 3.8.8, these n get(−) operations are on distinct ports of O. Let opi be the get(−) operation on port i ∈ [1..n].

We claim that no simulation of opi by any real process completes. This is because if a thread of some real process qb 6= q completes the simulation of opi in some loop iteration j, the use of the safe agreement object on lines 53 to 55 ensures that the thread of q simulating opi will also complete in loop iteration j (contradicting our assumption that this thread of q is blocked). Thus opi is incomplete in HP .

Since every port i ∈ [1..n] of O has an incomplete operation opi, by Claim 3.8.8, there is a time after which no other operation is invoked on O. So there is a time after which the pri field of O stops changing; let r be the final value of pri. Without loss of generality, suppose r is even (a symmetric argument holds for the case where r is odd). Since r + i is odd for i = 1, there is a time after which the thread of q simulating op1 (which loops forever in lines 43 to 60) will find that the pri field of O contains r, will evaluate the conditional on line 56 as true, and will return ? on line 58 — contradicting our earlier claim that no simulation of op1 completes. So if a real process crashes within the critical part of the simulation of a propose(−, −) operation on an n-P&G object O, at most n − 1 threads of q are blocked. Since the crash of a real process blocks at most n − 1 threads of q, even if k out of the |Q| = k + 1 real processes of Q (all real processes except q) crash, at least one of the |P | = k(n − 1) + 1 threads of q is not blocked, i.e., this thread simulates infinitely many steps of a virtual process in P . Detailed proof. We now present the detailed proof that for each real process q ∈ [1..|Q|], if q is q,p correct in HQ, then there is a virtual process p ∈ [1..|P |] such that σ is infinite, i.e., thread qp simulates infinitely many steps of the virtual process p in HP . To do so, we first consider the n-P&G objects that are accessed in the simulated history HP .

Deﬁnition 3.8.15. Let q ∈ [1..|Q|] be a real process of the real system Q. We say that q propose-blocks an n-P&G object O if there is a virtual processs p ∈ [1..|P |] such that q crashes while thread qp is simulating a propose(−, −) operation op on O and holding its local lock lockq, i.e., q crashes while thread qp is within lines 34 to 39 of its simulation of op.

Claim 3.8.16. If no real process propose-blocks an n-P&G object O, then there is a time after which

VALO does not contain any (−, 1).

Proof. Suppose no real process propose-blocks O. All ﬁelds of VALO are initially (⊥, 0), so if VALO contains any (−, 1), it must be written by some thread.

Let q ∈ [1..|Q|], p ∈ [1..|P |], and t be a time such that at time t, thread qp writes (−, 1) to

VALO[q][p]. In Algorithm 7, this can only happen when thread qp executes line 34 within its simulation of a propose(−, −) operation op on O. Furthermore, since line 34 is between lines 33 and 40, this can only happen while thread qp holds lockq. Since no real process propose-blocks O, thread qp must eventually continue its simulation of op until it releases lockq on line 40. Thus qp eventually executes either line 37 or line 39 within its simulation of op, setting VALO[q][p] to not contain (−, 1). So whenever

(−, 1) is written into VALO[q][p] eventually VALO[q][p] is set to not contain (−, 1).

In Algorithm 7, the ﬁelds of VALO are only written by simulations of propose(−, −) operations on O. Furthermore, by Claim 3.8.14, every simulation of every propose(−, −) operation on O that is invoked after tO does not perform any write operations. Since each simulation of each propose(−, −) operation performs at most two write operations on VALO according to Algorithm 7, there is a time Chapter 3. The Imprecision of the Set Agreement Power Classification 36

after which no process writes (−, 1) to VALO. Thus there is a time after which VALO does not contain any (−, 1). Claim 3.8.16

Deﬁnition 3.8.17. Let q ∈ [1..|Q|] be a real process of the real system Q, p ∈ [1..|P |] be a virtual processs of the system P , op be a get(−) operation invoked by p on an n-P&G object, and j be a positive integer.

We say that q get-blocks the j-th loop iteration of op if q crashes while thread qp is executing the j-th loop iteration of op and holding its local lock lockq, i.e., q crashes while thread qp is in line 53 within the j-th loop iteration of its simulation of op.

Claim 3.8.18. Let p ∈ [1..|P |] be a virtual process of the system P , and let x ≥ 1 be a positive integer such that the x-th action of virtual process p is a get(−) operation op on an n-P&G object. Let j ≥ 1 be a positive integer. If no real process get-blocks the j-th loop iteration of op, then every decide() operation on the safe agreement object SAFE[p][x][j] (line 55) performed by a correct real process eventually completes.

Proof. Suppose no real process get-blocks the j-th loop iteration of op. According to Algorithm 7, the safe agreement object SAFE[p][x][j] is accessed only by simulations of op within their j-th loop iteration. Thus by Deﬁnition 3.8.17, since no real process get-blocks the j-th loop iteration of op, no process crashes during a propose(−) operation on SAFE[p][x][j] (line 53). By Claim 3.8.1, the Termination (b) property holds for the safe agreement object SAFE[p][x][j]. Consequently, every decide() operation on SAFE[p][x][j] performed by a correct real process eventually completes. Claim 3.8.18

Deﬁnition 3.8.19. Let p ∈ [1..|P |] be a virtual process of the system P , and let op be an operation that is invoked by p. We say that op is a blocking operation if and only if there is a correct real process q ∈ [1..|Q|] such that thread qp begins simulating op but never completes simulating op.

Claim 3.8.20. Every propose(−, −) operation on every n-P&G object is not blocking.

Proof. Let p ∈ [1..|P |] be a virtual process, and let op be a propose(−, −) operation invoked by the virtual process p on an n-P&G object. In Algorithm 7, the code for simulating a propose(−, −) operation on an n-P&G object contains no loops. Thus for each correct real process q ∈ [1..|Q|], if thread qp of the correct real process q begins simulating op, it eventually completes simulating op. So by Deﬁnition 3.8.19, op is not blocking. Claim 3.8.20

Deﬁnition 3.8.21. We say that an n-P&G object O is propose-blocked if some real process propose- blocks O. We say that a get(−) operation op is get-blocked if some real process get-blocks some iteration of op.

Claim 3.8.22. Let p ∈ [1..|P |] be a virtual process of the system P , O be an n-P&G object, and x ≥ 1 be a positive integer such that the x-th action of virtual process p is a get(−) operation op on O. If op is blocking and not get-blocked, then there is a correct real process q ∈ [1..|Q|] such that thread qp completes inﬁnitely many loop iterations in its simulation of op.

Proof. Since op is blocking, by deﬁnition there is a correct real process q ∈ [1..|Q|] such that thread qp begins simulating op, but never completes simulating op. For each integer j ≥ 1, suppose thread qp reaches the j-th loop iteration of its simulation of op. Since op is not get-blocked, by Claim 3.8.18, every decide() operation on the safe agreement object SAFE[p][x][j] (line 55) performed by a correct real Chapter 3. The Imprecision of the Set Agreement Power Classification 37

process eventually completes. Therefore, thread qp eventually completes its decide() operation on the safe agreement object SAFE[p][x][j] (line 55), and thus eventually completes the j-th loop iteration of its simulation of op. Furthermore, since thread qp never completes its simulation of op, it reaches the (j +1)- th loop iteration after completing the j-th loop iteration of its simulation of op. Consequently, by simple induction, thread qp completes inﬁnitely many loop iterations in its simulation of op. Claim 3.8.22

Claim 3.8.23. Let op be a get(−) operation that is invoked on some n-P&G object O by some virtual process p. If O is not propose-blocked and op is not get-blocked, then op is not blocking.

Proof. Let x ≥ 1 be an integer such that p invokes op on O as its x-th action. Suppose, for contradiction, that O is not propose-blocked and op is not get-blocked, but op is blocking, i.e., some correct real process begins simulating op but never completes simulating op. By Claim 3.8.22, since op is blocking and op is not get-blocked, there is a correct real process q ∈ [1..|Q|] such that thread qp completes inﬁnitely many loop iterations in its simulation of op.

Since O is not propose-blocked, by Claim 3.8.16 there exists a time t after which VALO does not contain (−, 1). Let j be the smallest positive integer such that every thread that begins the j-th loop iteration of its simulation of op does so after this time t (such a j exists because thread qp completes inﬁnitely many loop iterations in its simulation of op).

Consider the j-th loop iteration of the simulation of op by thread qp. Thread qp completes this loop iteration without executing line 58 or line 60 (otherwise it would not complete inﬁnitely many loop iterations in its simulation of op). Therefore, thread qp evaluates the conditionals on lines 56 and

59 as false. So immediately before line 56, val = ?. Thus on line 55, thread qp performs a decide() operation that returns (?, −) on the safe agreement object SAFE[p][x][j], and so qp assigns ? to val. By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so before this decide() operation completes, a propose((?, −)) operation is invoked on SAFE[p][x][j]. Since op is the x-th action of virtual process p, according to Algorithm 7, SAFE[p][x][j] is accessed only by simulations of op in their j-th loop iteration. So this propose((?, −)) operation is invoked by 0 0 thread qp of some real process q ∈ [1..|Q|] on line 53 in the j-th loop iteration of its simulation of op. 0 0 Consider the j-th loop iteration of the simulation of op by thread qp. On line 45, thread qp takes a snapshot of VALO and assigns it to snap. Recall that every thread that begins the j-th loop iteration of its simulation of op does so after time t, where t is a time after which VALO does not contain (−, 1). 0 So the snapshot that thread qp assigns to snap does not contain (−, 1).

In Algorithm 7, the fields of VALO are initially (⊥, 0), and the only writes on the fields of VALO 0 are (−, 0) (line 39), (−, 1) (line 34), and (−, 2) (line 37). Thus after thread qp executes line 45, either 0 all fields of snap are (−, 0), or snap contains (−, 2). So thread qp evaluates one of the conditionals on 0 lines 46 and 48 as true. Thus thread qp either assigns ⊥ to val on line 47, or assigns some valuev ˜ to val on line 50 such that snap contains (˜v, 2) (line 49). Note that VALO never contains (?, 2), so the valuev ˜ 0 is not ?. Consequently, when thread qp executes line 53, val 6= ? — contradicting the fact that thread 0 qp invokes a propose((?, −)) operation on line 53. Claim 3.8.23

Claim 3.8.24. Let op be a get(−) operation that is invoked on some n-P&G object O by some virtual process p such that op is blocking but not get-blocked. Then op is incomplete in the simulated history

HP . Chapter 3. The Imprecision of the Set Agreement Power Classification 38

Proof. By Claim 3.8.22, since op is blocking but not get-blocked, there is a correct real process q ∈ [1..|Q|] such that thread qp completes infinitely many loop iterations in its simulation of op. 0 Suppose, for contradiction, that op is complete in HP . Thus there exists a real process q ∈ [1..|Q|] 0 and an integer j ≥ 1 such that thread qp completes its simulation of op in the j-th loop iteration, i.e., 0 thread qp returns (line 58 or 60) within the j-th loop iteration of its simulation of op. 0 Consider the j-th loop iterations of the simulations of op by threads qp and qp respectively. Note that both of these loop iterations are eventually completed by their respective threads, since thread qp 0 completes infinitely many loop iterations in its simulation of op, while thread qp returns within this loop iteration. On line 55, both threads perform a decide() operation on SAFE[p][x][j], and assign the response to (val, pri). By Claim 3.8.1, the Agreement property holds for SAFE[p][x][j], so both threads assign the same values to (val, pri) on line 55. Then observe that the execution of lines 56 to 60 by both threads depends solely on the values they assigned to (val, pri) on line 55 and the port of the n-P&G 0 0 object that the operations op and op are applied on. Thus, since thread qp returns on either line 58 or line 60, thread qp also returns on the same line. So thread qp completes its simulation of op in the j-th loop iteration — contradicting the fact that thread qp completes infinitely many loop iterations in its simulation of op. Claim 3.8.24

Deﬁnition 3.8.25. Let q ∈ [1..|Q|] be a real process of the real system Q, and O be an n-P&G object.

We say that q obstructs O if there is a virtual process p ∈ [1..|P |] such that q crashes while its thread qp is holding its local lock lockq and executing an operation on O.

Thus q obstructs an n-P&G object O if and only if either q propose-blocks O, or there is a get(−) operation op on O and an integer j ≥ 1 such that q get-blocks the j-th loop iteration of op.

Claim 3.8.26. For every integer f ≥ 0, if at most f real processes obstruct an n-P&G object O, then at most f(n − 1) operations on O are blocking.

Proof. Suppose at most f real processes obstruct an n-P&G object O. Then there are two cases: either O is not propose-blocked or O is propose-blocked.

Case 1. O is not propose-blocked.

Then let S be the set of all get(−) operations on O that are get-blocked. By Deﬁnition 3.8.25, since there are at most f real processes that obstruct O, and by the local lock mechanism each real process can get-block at most one loop iteration of one get(−) operation, there are at most f get(−) operations on O that are get-blocked. So |S| ≤ f.

Since O is not propose-blocked, and every get(−) operation op 6∈ S on O is not get-blocked, by Claim 3.8.23, every get(−) operation op 6∈ S on O is not blocking. Furthermore, by Claim 3.8.20, every propose(−, −) operation on O is not blocking. Thus all operations on O that are not in S are not blocking.

Finally, recall that n > 2. So there are at most |S| = f ≤ f(n − 1) blocking operations on O.

Case 2. O is propose-blocked.

Then by Deﬁnition 3.8.25, f ≥ 1. Let S be the set of blocking operations on O. If |S| ≤ f(n − 1), we are done. Chapter 3. The Imprecision of the Set Agreement Power Classification 39

So suppose, for contradiction, that there are more than f(n − 1) blocking operations on O, i.e., |S| ≥ f(n − 1) + 1. By Claim 3.8.20, every propose(−, −) operation on O is not blocking, so this set S contains only get(−) operations on O. Since O is propose-blocked, by Deﬁnition 3.8.25, at least one of the at most f real processes that obstruct O is propose-blocking O instead of get-blocking any loop iteration of any get(−) operation on O. Thus at most f − 1 real processes get-block loop iterations of get(−) operations on O.

Consider the set S0 of all get(−) operations in S that are not get-blocked. Since at most f − 1 real processes get-block loop iterations of get(−) operations on O, S0 contains at least f(n − 1) + 1 − (f − 1) = f(n − 2) + 2 operations on O. Note that since f ≥ 1, S0 contains at least f(n − 2) + 2 ≥ (n − 2) + 2 = n operations on O. So S0 is a set of at least n get(−) operations on O such that for each operation op ∈ S0 on O, op is blocking but not get-blocked. Thus by 0 Claim 3.8.24, all operations in S are incomplete in the simulated history HP .

0 Claim 3.8.26.1. For all i ∈ [1..n], there is exactly one operation opi ∈ S that is applied on port i of O.

Proof. Suppose, for contradiction, that not every port of O has exactly one operation in S0 applied to it. Then one of the n ports of O has at least two operations in S0 applied to it (otherwise, the total number of operations in S0 would be less than n).

0 Recall that all operations in S are incomplete in the simulated history HP . So there is a time after which there are at least two pending operations in S0 on the same port i of the n-P&G object O at the same time. This contradicts Claim 3.8.8.

For all i ∈ [1..n], let opi be as deﬁned in Claim 3.8.26.1. So there is a time after which for 0 every integer i ∈ [1..n], there is a pending incomplete get(i) operation opi ∈ S on port i of the n-P&G object O. No further operations are invoked on O after this time; otherwise there would be multiple pending operations on the same port of O at the same time — contradicting Claim 3.8.8. Consequently, there are only ﬁnitely many get(−) operations on O.

Consider the max register PRIO. In Algorithm 7, writeMax(−) operations on PRIO are performed only by simulations of get(−) operations on O (line 57). Furthermore, each simulation

that performs a writeMax(−) operation on PRIO either takes no more steps or completes immediately after (line 58). Thus the fact that there are at most |Q| = k + 1 simulations of each

operation on O implies that there are at most k +1 writeMax(−) operations performed on PRIO for each get(−) operation on O. Then, since there are only ﬁnitely many get(−) operations on

O, there are only ﬁnitely many writeMax(−) operations are performed on PRIO. So there is a

time t after which the value stored in PRIO does not change.

Let r be the value stored in PRIO after time t. There are two cases: either r is odd, or r is even.

Case 2(a). r is odd.

Consider the incomplete get(2) operation op2. Let p ∈ [1..|P |] be a virtual process and x ≥ 1

be an integer such that op2 is invoked by p as its x-th action. Chapter 3. The Imprecision of the Set Agreement Power Classification 40

0 Since op2 is in S , op2 is blocking, and not get-blocked. Thus by Claim 3.8.22, since op2 is

blocking and op2 is not get-blocked, there is a correct real process q ∈ [1..|Q|] such that thread

qp completes inﬁnitely many loop iterations in its simulation of op2. Let j be the smallest positive integer such that every thread that begins the j-th loop iteration

of its simulation of op2 does so after time t. Consider the j-th loop iteration of the simulation

of op2 by thread qp. Thread qp completes this loop iteration without executing line 58 or

line 60 (otherwise qp would not complete inﬁnitely many iterations in its simulation of op2).

Thus thread qp evaluates the conditionals on lines 56 and 59 as false. So immediately before

line 56, val = ?, and either i > 2 or pri + i is even. Then, since i = 2 for op2, pri + i is even, 0 and so pri is some even number r . Thus on line 55, thread qp performs a decide() operation 0 0 that returns (−, r ) on the safe agreement object SAFE[p][x][j], and so qp assigns r to pri. By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so before this decide() operation completes, a propose((−, r0)) operation is invoked on SAFE[p][x][j].

Since op2 is the x-th action of virtual process p, according to Algorithm 7, SAFE[p][x][j] is 0 accessed only by simulations of op2 in their j-th loop iteration. So this propose((−, r )) 0 0 operation is invoked by thread qp of some real process q ∈ [1..|Q|] on line 53 in the j-th 0 loop iteration of its simulation of op2. Immediately before invoking this propose((−, r )) 0 operation, thread qp performs a readMax() operation on PRIO, and assigns the response to pri (line 51).

Recall that every thread that begins the j-th loop iteration of its simulation of op2 does

so after time t, where t is the time after which PRIO contains the odd number r. So this 0 readMax() operation on PRIO returns r, and so thread qp assigns r to pri on line 51. Thus 0 0 on line 53, thread qp invokes a propose((−, r)) operation on SAFE[p][x][j]. So r = r — contradicting the fact that r is odd while r0 is even. Case 2(b). r is even.

We derive a contradiction via symmetric arguments on op1. Claim 3.8.26

Claim 3.8.27. For each correct real process q ∈ [1..|Q|], there is a virtual process p ∈ [1..|P |] such that |σq,p| = ∞.

Proof. Let f1 denote the number of real processes that crash while one of their threads is holding its local lock and initializing a virtual process (line 5). Then let S1 denote the set of virtual processes such that for each virtual process p ∈ S1, there is a correct real process that does not complete its initialization of p. In Algorithm 6, a correct real process does not complete its initialization of a virtual process p only if it never completes its decide() operation on the safe agreement object SAFE[p][0][1] on line 7. By Claim 3.8.1 and the Termination (b) property of safe agreement objects, this occurs only if a real process crashes during a propose(−) operation on SAFE[p][0][1] (line 5), i.e., while one of its threads is holding its local lock and initializing a virtual process. Thus |S1| ≤ f1.

Let f2 denote the number of real processes that crash while one of their threads is holding its local lock and simulating an operation on a register (line 19). Then let S2 denote the set of virtual processes that invoke blocking operations on registers in HP . In Algorithm 6, an operation on register in HP is blocking only if a correct real process never completes its decide() operation on the safe agreement object on line 21. By Claim 3.8.1 and the Termination (b) property of safe agreement objects, this only occurs if a real process crashes during a propose(−) operation on the same safe agreement object Chapter 3. The Imprecision of the Set Agreement Power Classification 41

(line 19), i.e., while one of its threads is holding its local lock and simulating an operation on a register.

Thus |S2| ≤ f2.

Let f3 denote the number of real processes that crash while one of their threads is holding its local lock and accessing an n-P&G object (lines 34 to 39 or line 53). Then let S3 denote the set of virtual processes that invoke blocking operations on n-P&G objects in HP . By Deﬁnition 3.8.25, f3 real processes obstruct n-P&G objects. So by Claim 3.8.26, there are at most f3(n − 1) blocking operations on n-P&G objects in HP . Then, since there are at most f3(n−1) blocking operations on n-P&G objects in HP , |S3| ≤ f3(n − 1).

Recall that there are |Q| = k + 1 real processes, and at most k of them can crash in HQ. Thus f1 + f2 + f3 ≤ k.

Let S = S1 ∪ S2 ∪ S3. Since |S1| ≤ f1, |S2| ≤ f2, and |S3| ≤ f3(n − 1), |S| ≤ f1 + f2 + f3(n − 1).

Then, since f1 + f2 + f3 ≤ k and n > 2, |S| ≤ k(n − 1). Now recall that |P | = k(n − 1) + 1. Thus there is a virtual process p ∈ [1..|P |] such that p 6∈ S.

Let q ∈ [1..|Q|] be any correct real process in HQ, and consider the simulation of virtual process p by thread qp. Since p 6∈ S1, thread qp completes its initialization of p. Since p 6∈ S2 and p 6∈ S3, every operation invoked by virtual process p on any register or n-P&G object is not blocking. So thread qp begins and completes its simulation of every operation invoked by virtual process p. Consequently, q,p σ = ∞. Claim 3.8.27

Safety of the Simulation Algorithm

We now show that the real processes that execute our extended BG simulation algorithm AQ in system

Q (which has only registers) correctly simulate an execution of the algorithm AP by the virtual processes of system P (which has registers and n-P&G objects). If the algorithm AP is such that no process in

P ever accesses n-P&G objects, then the correctness of AQ follows directly from the correctness of the original BG simulation algorithm [7, 8, 37]. This is because, in this case, no real process would ever execute the code of AQ that we added to the original BG simulation to simulate accesses to n-P&G objects by the virtual processes of P (lines 22 to 24 and lines 30 to 60). So to prove the correctness of our extended BG simulation algorithm AQ, it suﬃces to show that this additional code correctly simulates accesses to n-P&G objects by the virtual processes of P .

From Deﬁnition 3.8.6, recall that HP is the history of the algorithm AP simulated by the algorithm

AQ.

Deﬁnition 3.8.28. For the remainder of this section:

• O is a n-P&G object that is accessed in HP , and

O • HP is the history of the operations on O in HP .

O Since HP is a subsequence of HP , from Claim 3.8.7 we have:

Observation 3.8.29. Given any time t:

O • No operation in the history HP is both invoked and completed at time t.

O • There is at most one operation op in the history HP such that op is either invoked or completed at time t. Chapter 3. The Imprecision of the Set Agreement Power Classification 42

O In the following we show that history HP is linearizable with respect to the sequential specification of the n-P&G object. We first outline the proof and then present it in detail. O Outline. In this outline, we explain how to linearize HP assuming that it contains at least one get(−) operation on O that returns a value other than ? or ⊥.5 It turns out that all such get(−) operations win return the same value, denoted vO ; this is because they obtain the value of the val field of O from VALO via a modification of the {0,1,2}-level mechanism used by safe agreement objects. Recall that: (i) the state of the n-P&G object O consists of two fields val and pri (as defined in

Figure 3), and (ii) the val ﬁeld of O is simulated by the snapshot object VALO, while the pri ﬁeld of O is simulated by the max register PRIO. Intuitively, at any time t:

• If all ﬁelds of the snapshot object VALO still contain (−, 0), i.e., there are no level 1 or 2 proposals, win then the val ﬁeld of O contains ⊥; otherwise it contains vO .

• The value of the max register PRIO is the value of the pri ﬁeld of O.

0O O We first construct a completion HP of the history HP of the operations on O: For each incomplete operation op, if op changes the state of O, we complete op immediately after it does so; otherwise we remove op. 0O We then linearize the operations of HP as follows: We first linearize the propose(−, −) operations on O, and then we linearize the get(−) operations on O. In both cases, we begin by linearizing the operations that change the state of O, and then we linearize the operations that only read the state of O, as described below. win win win Let opO be the propose(vO , −) operation with the earliest invocation time among all propose(vO , −) O win operations on O in HP . Intuitively, opO is the first propose(−, −) operation to take effect, by chang- win win ing the val field of O from ⊥ to vO . Thus, we linearize opO at the earliest time t when VALO is changed, i.e., when a (−, 1) is first written into VALO (line 34). This time t is within the execution win interval of opO :

win win • Time t cannot be after opO is completed, because every simulation of opO either ﬁnds that VALO has been changed from its initial value (lines 31 to 32), or it changes VALO (line 34).

win win • If time t is before opO is invoked, then every simulation of opO would evaluate the conditional on win line 32 as false and return without changing VALO. The same would hold for all propose(vO , −) win O win operations invoked after opO in HP , so vO would never be written into VALO — contradicting win our deﬁnition of vO as the agreed upon value determined by the {0, 1, 2}-level mechanism on VALO.

O win Let op be any propose(−, −) operation in HP that is not opO . Intuitively, op should be linearized win after opO (and it should return done without changing the state of O). So we linearize op at its completion time. Note that the completion time of op is after the earliest time VALO is changed (the win linearization time of opO ), because every simulation of op either ﬁnds that VALO has been changed win from its initial value (lines 31 to 32), or it changes VALO (line 34). Thus op is linearized after opO is linearized, as wanted. O Next, let op be any get(−) operation in HP that performs a writeMax(−) operation on PRIO (line 57). Intuitively, op is an unprioritized get(−) operation, which changes the pri ﬁeld of O. Roughly

5 O If this assumption does not hold, the linearization of HP is simpler; intuitively, this is because in this case the values proposed to O are not “visible”. Chapter 3. The Imprecision of the Set Agreement Power Classification 43

speaking, since safe agreement objects are used to agree on the values read from PRIO (lines 51 to 55), all simulations of op that perform writeMax(−) operations on PRIO (line 57) write the same value.

Since PRIO is a max register that contains the largest value ever written to it, only the earliest of these writeMax(−) operations can change the state of PRIO. Thus we linearize op at the earliest time t when a simulation of op performs a writeMax(−) operation on PRIO (line 57). Clearly, this time t is within the execution interval of op. O Finally, let op be any get(−) operation in HP that does not perform a writeMax(−) operation on PRIO (line 57). Intuitively, op is a prioritized get(−) operation, which returns the current value u 6= ? of the val field of O without changing the state of O. Thus we linearize op at the earliest time within its execution interval when (a) the val field of O contains u, and (b) the pri field of O contains an integer r such that op is prioritized. We now explain why such a time exists. First observe that before op returns u 6= ?, some simulation of op reads u from the val field of O (lines 45 to 50), and then reads an integer r indicating that op is prioritized from the pri field of O 0 (line 51). Let t be the time when this simulation of op takes a snapshot of VALO (line 45), and t be the time when this simulation of op performs a readMax() operation on PRIO (line 51). Note that the interval [t, t0] is within the execution interval of op: op returns u after time t0, and op is invoked before time t, because some simulation of op executes line 45 at time t. Clearly, (a) is true at time t, while (b) is true at time t0. We claim that (b) is also true at time t, as wanted. Suppose, for contradiction, that (b) is not true at time t. This implies that op is on a port i ≤ 2; otherwise, (b) would be true at all times. Without loss of generality, assume that op is on port 1 (a symmetric argument applies in the case where op is on port 2). Since op is a get(1) operation, the integer r that indicates that op is prioritized is odd. From lines 56-57, it clear that only get(1) operations can write odd integers to PRIO. Since (b) is not true at time t, PRIO does not contain r at time t. Thus 0 0 (a thread simulating) some get(1) operation op writes r into PRIO at some time between t and t , i.e., within the execution interval of op. Since PRIO is a max register, and PRIO does not contain r at time 0 0 t, it is clear that no thread simulating op writes r into PRIO before time t. Thus op completes after 0 0 0 time t. Furthermore, since a thread simulating op writes r into PRIO before time t , op is invoked 0 0 O before time t . So op and op are concurrent operations in HP , and they are both on port 1 of O,— contradicting Claim 3.8.8. Thus, (b) is true at time t; and so our linearization of op is well-defined. O Detailed proof. We now give the detailed proof that the history HP of the operations on O in the simulated history HP is indeed linearizable with respect to the sequential specification of the n-P&G object.

From Deﬁnition 3.8.10, recall that tO is the earliest time when (−, 1) is written into VALO; or ∞ if

(−, 1) is never written into VALO.

win Deﬁnition 3.8.30. We deﬁne vO as follows:

win • If there is a value v such that (v, 2) is eventually written into VALO (line 37), then vO is the smallest such value.

win • Else if tO 6= ∞, then vO is the value v such that (v, 1) is written into VALO (line 34) at time tO.

win • Else vO = nil.

Note that VALO never contains (nil, −), and by Claim 3.8.12, (−, 2) is never written to VALO if tO = ∞. Thus: Chapter 3. The Imprecision of the Set Agreement Power Classification 44

win Observation 3.8.31. vO = nil if and only if tO = ∞.

win Intuitively, vO is the value of the first propose(−, −) operation on O to take effect, or nil if no O propose(−, −) operation on O takes effect in HP . We will later show that if any get(−) operation op win on O returns a value other than ? or ⊥, then it returns the value vO in Definition 3.8.30 (matching the win definition of vO in the proof outline).

win Claim 3.8.32. For all times t, if VALO contains (−, 2) but not (−, 1) at time t, then vO 6= nil and win vO is the smallest value v such that VALO contains (v, 2) at time t.

win Proof. Suppose, for contradiction, that at some time t, VALO contains (−, 2) but not (−, 1), and vO is not the smallest value v such that VALO contains (v, 2). Since VALO contains (−, 2) at time t, by win win Deﬁnition 3.8.30, vO 6= nil, and vO is the smallest value v such that (v, 2) is written into VALO. win win Thus observe that VALO does not contain (vO , 2) at time t; otherwise vO would be the smallest value v such that VALO contains (v, 2) at time t. win win Then, by Claim 3.8.9, VALO does not contain (vO , 2) before time t. Thus (vO , 2) is written into VALO after time t.

In Algorithm 7, (−, 2) can only be written into VALO by a simulation of a propose(−, −) operation on the n-P&G object O, and only when the simulation executes line 37. So let q ∈ [1..|Q|] be a real process, p ∈ [1..|P |] be a virtual process, and op be a propose(−, −) operation on O invoked by p such win that thread qp writes (vO , 2) into VALO at a time after t, when it executes line 37 within its simulation of op. Then earlier in the simulation of op, thread qp did the following in reverse chronological order:

1. Evaluated the conditional on line 36 as true, i.e., snap does not contain (−, 2) when thread qp executes this line.

2. Took a snapshot of VALO and assigned it to snap on line 35, i.e., VALO does not contain (−, 2)

when thread qp executes this line.

3. Wrote (−, 1) into VALO[q][p] on line 34.

Note that since VALO contains (−, 2) at time t, by Claim 3.8.9, VALO always contains (−, 2) after time t. Therefore, since VALO does not contain (−, 2) when thread qp executes line 35 within its simulation of op, thread qp executes line 35 within its simulation of op before time t. Thus thread qp also executes line 34 within its simulation of op before time t. Then, since only thread qp writes to

VALO[q][p] in Algorithm 7, the (−, 1) that thread qp wrote into VALO[q][p] when it executed line 34 within its simulation of op is still in VALO[q][p] at time t — contradicting our assumption that VALO does not contain (−, 1) at time t. Claim 3.8.32

win win By Deﬁnition 3.8.30, if vO 6= nil, then (vO , −) is eventually written into VALO. In Algorithm 7, win win (vO , −) is only written into VALO by simulations of propose(vO , −) operations on O. Thus:

win win O Observation 3.8.33. If vO 6= nil, then there is a propose(vO , −) operation in HP .

win Deﬁnition 3.8.34. We deﬁne opO as follows:

win win win O • If vO 6= nil, then opO is the earliest invoked propose(vO , −) operation in HP .

win win • If vO = nil, then opO = nil. Chapter 3. The Imprecision of the Set Agreement Power Classification 45

win Note that if opO 6= nil, by Observation 3.8.29 it is indeed unique. From Observation 3.8.31, it is clear that:

win Observation 3.8.35. opO = nil if and only if tO = ∞.

win win Claim 3.8.36. If opO 6= nil, then opO is invoked at some time t < tO.

win win Proof. Suppose, for contradiction, opO 6= nil and opO is invoked at some time t ≥ tO. By Observa- win tion 3.8.35, since opO 6= nil, tO 6= ∞. win win O Furthermore, by Definition 3.8.34, opO is the earliest invoked propose(vO , −) operation in HP . win O 0 Thus each propose(vO , −) operation in HP is invoked at some time t ≥ tO. So by Claim 3.8.14, win O every simulation of every propose(vO , −) operation in HP does not write anything into VALO. In win win Algorithm 7, (vO , −) is only written to VALO by simulations of propose(vO , −) operations on O, win and all fields of VALO are initially (⊥, 0). Thus VALO never contains (vO , −). This contradicts win win Definition 3.8.30, which implies that since vO 6= nil, there is a time when VALO contains (vO , −). Claim 3.8.36

O Completion of the History HP .

0O O Deﬁnition 3.8.37. We construct a completion HP of the history HP of the operations on O in the O simulated history HP as follows. For each incomplete operation op in HP :

Case 1. op is a propose(−, −) operation.

win If op = opO , then complete op with return value done at time tO; otherwise remove op.

Case 2. op is a get(−) operation.

If some simulation of op executes line 57 of algorithm AQ, then complete op with return value ? at

the earliest time when a simulation of op executes line 57 of algorithm AQ; otherwise remove op.

0O Claim 3.8.38. Let op be an operation on the n-P&G object O in HP . Then the time when op completes exists and is after op is invoked.

O Proof. If op is complete in HP , then it is easy to see that the claim holds from Deﬁnition 3.8.6, O Claim 3.8.7, and Deﬁnition 3.8.28. So suppose op is incomplete in HP . There are two cases: either op is a propose(−, −) operation, or op is a get(−) operation.

Case 1. op is a propose(−, −) operation.

0O win Then since op is in HP , by Deﬁnition 3.8.37, op = opO , and in the construction of the completion 0O O win HP of HP , op is completed at time tO. By Observation 3.8.35, since opO 6= nil, tO 6= ∞. Thus 0O win the time when op is completed in HP exists. Furthermore, by Claim 3.8.36, op = opO is invoked at some time t < tO. Thus the time when op completes is after op is invoked.

Case 2. op is a get(−) operation.

0O Then since op is in HP , by Deﬁnition 3.8.37, there exists a time t such that t is the earliest time 0O when a simulation of op executes line 57 of algorithm AQ, and op is completed at time t in HP . So the time when op completes exists. Since some simulation of op executes line 57 of algorithm

AQ at time t, by Deﬁnition 3.8.6 and Deﬁnition 3.8.28, op is invoked before time t. Thus the time

t when op completes is after op is invoked. Claim 3.8.38 Chapter 3. The Imprecision of the Set Agreement Power Classification 46

Claim 3.8.39. Given any time t:

0O • No operation in the history HP is both invoked and completed at time t.

0O • There is at most one operation op in the history HP such that op is either invoked or completed at time t.

0O Proof. By Observation 3.8.29 and Claim 3.8.38, no operation in the history HP is both invoked and 0O completed at time t. Thus it suﬃces to show that there is at most one operation op in the history HP such that op is either invoked or completed at time t. 0 0O Let op and op be two arbitrary distinct operations in the history HP . There are two cases: either at 0 O least one of op and op is a propose(−, −) operation that is incomplete in HP and completed at time t 0O 0 O in HP , or neither of op and op is a propose(−, −) operation that is incomplete in HP and completed 0O at time t in HP .

0 O Case 1. At least one of op and op is a propose(−, −) operation that is incomplete in HP and completed 0O at time t in HP . O 0O By Definition 3.8.37, an incomplete propose(−, −) operation in HP is completed in HP only if it win win 0 win is opO ; otherwise it is removed. Since opO is unique, op and op cannot both be opO . Without win 0 win win win loss of generality let op = opO and op 6= opO . Note that since op = opO , opO 6= nil. win Thus by Observation 3.8.35, tO 6= ∞. By Definition 3.8.37, since op = opO is an incomplete O 0O propose(−, −) operation in HP , it is completed at time tO in HP . So t = tO. Thus it suffices to 0 show that op is neither invoked nor completed at time t = tO.

0O O Recall that the history HP is a completion of HP , which by Definition 3.8.28, is the history of 0O operations on O in HP . Thus every operation in HP is also in HP , and each operation is invoked 0O 0O at the same time in HP as in HP . Furthermore, every operation in HP that is complete in HP 0O completes at the same time in HP as in HP . So by Definition 3.8.6, some real process executes 0 0 line 10 of algorithm AQ at the invocation time of op , and if op is complete in HP , then some real 0 0 process executes line 29 of algorithm AQ at the completion time of op . If op is incomplete in HP O 0 win 0O (and hence HP by Definition 3.8.28), then by Definition 3.8.37, since op 6= opO is in HP , some real process executes line 57 at the completion time of op0. Thus at the each of the invocation time and the completion time of op0, some real process executes one of lines 10, 29, and 57 of algorithm

AQ.

In contrast, by Observation 3.8.11, since tO 6= ∞, some real process executes line 34 of algorithm 0 AQ at time tO. Since at most one step is taken at any time, op is neither invoked nor completed

at time tO.

0 O Case 2. Neither of op and op is a propose(−, −) operation that is incomplete in HP and completed 0O at time t in HP . It suﬃces to show that at least one of op and op0 is neither invoked nor completed at time t. 0 0O Suppose, for contradiction, that each of op and op is either invoked or completed at time t in HP . 0O First consider op. Since op is either invoked or completed at time t in HP , one of the following is true:

0O • op is invoked at time t in HP . Chapter 3. The Imprecision of the Set Agreement Power Classification 47

0O O • op is completed at time t in HP and op is complete in HP . 0O O • op is completed at time t in HP and op is incomplete in HP .

0O Case 2(a). op is invoked at time t in HP . 0O O Recall that the history HP is a completion of HP , which by Deﬁnition 3.8.28, is the history 0O of operations on O in HP . Thus every operation in HP is also in HP , and each operation in 0O 0O HP is invoked at the same time in HP as in HP . So op is invoked at time t in HP . Thus by Deﬁnition 3.8.6, some real process executes line 10 of algorithm AQ in its simulation of op at time t.

0O O Case 2(b). op is completed at time t in HP and op is complete in HP . O O Then op is completed at time t in HP . By Deﬁnition 3.8.28, HP is the history of operations O O on O in HP . Thus every operation in HP is also in HP , and each complete operation in HP O is completed at the same time in HP as in HP . So op is completed at time t in HP . Thus by Deﬁnition 3.8.6, some real process executes line 29 of algorithm AQ in its simulation of op at time t.

0O O Case 2(c). op is completed at time t in HP and op is incomplete in HP . O Note that by the assumption of Case 2, op cannot be a propose(−, −) operation in HP . O 0O Thus op is a get(−) operation that is incomplete in HP and completed at time t in HP . So by Deﬁnition 3.8.37, some real process executes line 57 of algorithm AQ in its simulation of op at time t.

So in all cases, some real process takes a step (i.e., executes a line of algorithm AQ) in its simulation of op at time t.

0 0O By symmetric arguments, since op is either invoked or completed at time t in HP , some real 0 process takes a step (i.e., executes a line of algorithm AQ) in its simulation of op at time t. Since op 6= op0, it is easy to see that the steps taken in the simulation of op are distinct from the steps taken in the simulation of op0. Thus there are at least two distinct steps taken at time t —

contradicting the fact that at most one step is taken at any time. Claim 3.8.39

0O Claim 3.8.40. If op is a propose(−, −) operation in HP , then op is completed at some time t ≥ tO (and so tO 6= ∞).

0O Proof. Let p ∈ [1..|P |] be a virtual process and op be a propose(−, −) operation invoked by p in HP . Suppose, for contradiction, that op is completed at some time t < tO. There are two cases: either op is O O complete in HP , or op is incomplete in HP .

O Case 1. op is complete in HP .

Then let q ∈ [1..|Q|] be a real process such that thread qp is the ﬁrst thread to complete its

simulation of op, i.e., the end of the execution interval of op is when thread qp completes its

simulation of op. So thread qp completes its simulation of op at time t < tO.

Before thread qp does so, it takes a snapshot of VALO and assigns it to snap (line 31 of algorithm 0 0 AQ) at some time t < t. Since t < tO, t < tO. So by Claim 3.8.12, all ﬁelds of this snapshot

of VALO that thread qp assigns to snap are (−, 0). Thus thread qp evaluates the conditional on Chapter 3. The Imprecision of the Set Agreement Power Classification 48

line 32 of algorithm AQ as true. So thread qp writes (−, 1) to VALO[q][p] (line 34 of algorithm AQ) at some time tˆ before completing its simulation of op at time t.

By Deﬁnition 3.8.10, tO denotes the earliest time when VALO contains (−, 1); or ∞ if no such time

exists. Thus tO ≤ tˆ. Then, since tˆ < t, tO < t — contradicting our assumption that t < tO.

O Case 2. op is incomplete in HP . 0O O Then since the propose(−, −) operation op is in the completion HP of HP , by Deﬁnition 3.8.37, win op = opO and op is completed at time tO, contradicting our assumption that op is completed at time t < tO. Claim 3.8.40

0O Claim 3.8.41. For each get(−) operation op on the n-P&G object O in HP , one of the following is true:

• The return value of op is ?.

• The return value of op is ⊥.

win win • vO 6= nil and the return value of op is vO .

0O Proof. Let op be a get(−) operation on the n-P&G object O in HP such that the return value of op win win is a value u that is neither ? nor ⊥. It suﬃces to show that vO 6= nil and u = vO . Let p ∈ [1..|P |] be the virtual process that invokes op, and let x ≥ 1 be an integer such that op is the x-th action of p. First, note that by Deﬁnition 3.8.37, since u 6= ?, the get(−) operation op is complete O in HP . So let q ∈ [1..|Q|] be a real process such that thread qp eventually completes its simulation of op with return value u. Since thread qp eventually completes its simulation of op, there is an integer j ≥ 1 such that thread qp executes line 58 or 60 of algorithm AQ within the j-th loop iteration of its simulation of op.

Consider this j-th loop iteration of the simulation of op by thread qp. Since u 6= ?, thread qp does not execute line 58. So thread qp executes line 60, and val = u when it does. Thus on line 55, thread qp performs a decide() operation that returns (u, −) on the safe agreement object SAFE[p][x][j], and so assigns u to val. By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so before this decide() operation completes, a propose((u, −)) operation is invoked on SAFE[p][x][j]. Since op is the x-th action of virtual process p, according to Algorithm 7, SAFE[p][x][j] is accessed only by simulations of op in their j-th loop iteration. So this propose((u, −)) operation is invoked by 0 0 thread qp of some real process q ∈ [1..|Q|] on line 53 in the j-th loop iteration of its simulation of op. 0 0 Consider this j-th loop iteration of the simulation of op by thread qp. Since thread qp invokes a 0 propose((u, −)) operation on line 53, val = u when thread qp executes line 53. Note that earlier in the 0 0 loop iteration, thread qp set val to ? on line 44. Thus, since u 6= ? and u 6= ⊥, thread qp executes line 50 before it executes line 53. Furthermore, u is the smallest value such that snap contains (u, 2) on line 49. 0 In addition, since thread qp executes line 50, it evaluates the conditional on line 48 as true, i.e., snap contains (−, 2) but not (−, 1). 0 Consequently, when thread qp takes a snapshot of VALO and assigns it to snap on line 45, VALO contains (−, 2) but not (−, 1), and u is the smallest value such that VALO contains (u, 2). Thus by win win Claim 3.8.32, vO 6= nil and u = vO . Claim 3.8.41

0O Linearization of the Completed History HP . Chapter 3. The Imprecision of the Set Agreement Power Classification 49

0O O Deﬁnition 3.8.42. We construct a linearization L of the completion HP of the history HP of the 0O operations on O in the simulated history HP as follows. For each operation op in HP :

Case 1. op is a propose(−, −) operation.

win If op = opO , then linearize op at tO; otherwise linearize op at the end of its execution interval, i.e., at the earliest time when a simulation of op executes line 29.

Case 2. op is a get(−) operation. Let u denote the return value of op.

• If u = ?, then linearize op at the earliest time when a simulation of op executes line 57.

• If u = ⊥, then linearize op at the earliest time t < tO when a simulation of op executes line 45.

• If u 6= ? and u 6= ⊥, then linearize op at the earliest time t ≥ tO when a simulation of op executes line 45.

We next show that the above linearization is well-deﬁned.

0O Claim 3.8.43. Let op be an operation on the n-P&G object O in HP . Then the linearization time of op in the linearization L (Deﬁnition 3.8.42) exists and is within the execution interval of op.

Proof. Let p ∈ [1..|P |] be the virtual process that invokes op, i ∈ [1..n] be the port of O that op is applied on, and x ≥ 1 be an integer such that op is the x-th action of p. There are 4 cases:

win Case 1. op is a propose(−, −) operation and op = opO . 0O win Then by Deﬁnition 3.8.42, op is linearized at time tO in the linearization L of HP . Since op = opO , win win opO 6= nil. By Observation 3.8.35, since opO 6= nil, tO 6= ∞. Thus the linearization time of op 0O in the linearization L of HP exists. win Next, by Claim 3.8.36, opO is invoked before time tO. Furthermore, by Claim 3.8.40, since op is a propose(−, −) operation on O, op is completed at some time t ≥ tO. So tO, the linearization 0O time of op in the linearization L of HP , is within the execution interval of op.

win Case 2. op is a propose(−, −) operation and op 6= opO . 0O Then by Deﬁnition 3.8.42, in the linearization L of HP , op is linearized at the end of its execution interval, which exists and is within the execution interval of op by Claim 3.8.38.

Case 3. op is a get(−) operation and op returns ?.

O O There are two cases: either op is complete in HP , or op is incomplete in HP .

O Case 3(a). op is complete in HP .

Then let q ∈ [1..|Q|] be the real process such that thread qp completes its simulation of op the earliest. Note that by Deﬁnition 3.8.6, this implies that the end of the execution interval

of op is when thread qp completes its simulation of op. Let tend denote this time when op completes.

Since op returns ?, thread qp completes its simulation of op with return value ?. From Algo- rithm 7 it is clear that the conditional on line 59 prevents simulations of get(−) operations Chapter 3. The Imprecision of the Set Agreement Power Classification 50

from returning ? via line 60. So thread qp completes its simulation of op with return value ? via line 58.

Thus before thread qp completes its simulation of op, it executes line 57 of algorithm AQ at 0O some time tpri < tend. By Deﬁnition 3.8.42, in the linearization L of HP , op is linearized at the earliest time t when a simulation of op executes line 57 of algorithm AQ. Since thread qp

executes line 57 of algorithm AQ within its simulation of op at time tpri, t exists and t ≤ tpri.

Furthermore, since t ≤ tpri and tpri < tend, t < tend, so the linearization time t of op in 0O the linearization L of HP is before the end of the execution interval of op. Finally, since some simulation of op executes line 57 of algorithm AQ at time t, time t is clearly after op 0O is invoked. Thus the linearization time t of op in the linearization L of HP is within the execution interval of op.

O Case 3(b). op is incomplete in HP . 0O Then since the get(−) operation op is in HP , by Deﬁnition 3.8.37 and Claim 3.8.38, there exists a time top such that top is the earliest time when a simulation of op executes line 57 0O of algorithm AQ, and op is completed at time top in HP . By Deﬁnition 3.8.42, in the 0O linearization L of HP , op is linearized at the earliest time t when a simulation of op executes line 57 of algorithm AQ. So op is linearized at time t = top. Since this is the time when op is 0O completed in HP , op is linearized within its execution interval.

Case 4. op is a get(−) operation and op returns a value u 6= ?.

0O O By Deﬁnition 3.8.37, in the construction of the completion HP of HP , each incomplete get(−) 0O operation is either removed or completed with return value ?. Thus, since op is in HP and returns O u 6= ?, op is complete in HP .

So let q ∈ [1..|Q|] be the real process such that thread qp completes its simulation of op the earliest,

i.e., the end of the execution interval of op is when thread qp completes its simulation of op. Note

that since thread qp completes its simulation of op and has return value u 6= ?, there is an integer j

such that thread qp executes line 60 of algorithm AQ within the j-th loop iteration of its simulation of op.

Consider the j-th loop iteration of the simulation of op by thread qp. Since thread qp completes

its simulation of op with return value u, val = u when thread qp executes line 60. So earlier in this

loop iteration, on line 55 of algorithm AQ, thread qp receives (u, −) from a decide() operation on the safe agreement object SAFE[p][x][j] and assigns u to val.

By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so a propose((u, −)) operation is in-

voked on SAFE[p][x][j] before thread qp receives (u, −) from a decide() operation on SAFE[p][x][j]. Since op is a get(−) operation on the n-P&G object O, according to Algorithm 7, SAFE[p][x][j] is only accessed by simulations of op within the j-th loop iteration (line 53 and 55 of algorithm 0 0 AQ). So let q ∈ [1..|Q|] be a real process such that thread qp invokes a propose((u, −)) operation

on SAFE[p][x][j] (line 53 of algorithm AQ) within the j-th loop iteration of its simulation of op

before thread qp receives (u, −) from a decide() operation on SAFE[p][x][j].

0 0 Now consider the j-th loop iteration of the simulation of op by thread qp. Since thread qp invokes 0 a propose((u, −)) operation on SAFE[p][x][j] on line 53, val = u when thread qp executes line 53. Chapter 3. The Imprecision of the Set Agreement Power Classification 51

0 This implies that qp previously set val to u 6= ? in either line 47 or line 50 of this loop iteration. There are two cases: either u = ⊥, or u 6= ⊥.

Case 4(a). u = ⊥.

Note that in Algorithm 7, (⊥, 2) is never written into VALO. Thus no snapshot of VALO contains (⊥, 2), and so ⊥ is never assigned to val when a thread executes line 50. Therefore, 0 since u = ⊥, thread qp assigns ⊥ to val on line 47. Thus all ﬁelds of snap are (−, 0) when 0 0 thread qp evaluates the conditional on line 46. Thus when thread qp executes line 45, it takes

a snapshot of VALO that contains only (−, 0) and assigns it to snap. Let t be the time when this happens, and note that t is before the end of the execution interval of op, since t is before 0 thread qp invokes a propose((u, −)) operation on SAFE[p][x][j], which is before thread qp

completes a decide() operation on SAFE[p][x][j], which is before thread qp completes its simulation of op, which is the end of the execution interval of op. Furthermore, since all ﬁelds

of VALO are (−, 0) at time t, by Claim 3.8.13, t < tO. 0O By Deﬁnition 3.8.42, in the linearization L of HP , since u = ⊥, op is linearized at the earliest 0 0 time t < tO when a simulation of op executes line 45. Since thread qp executes line 45 within 0 0 its simulation of op at a time t such that t < tO, t exists and t ≤ t. Then, since t is before the end of the execution interval of op, t0 ≤ t is also before the end of the execution interval of op. Furthermore, since a simulation of op executes line 45 at time t0, t0 is after the start of the execution interval of op. Thus the linearization time t0 of op in the linearization L of 0O HP is within its execution interval. Case 4(b). u 6= ⊥. 0 Then, since u 6= ? and u 6= ⊥, thread qp assigns u to val on line 50. Thus snap contains (−, 2) 0 0 but not (−, 1) when thread qp evaluates the conditional on line 48. Thus when thread qp

executes line 45, it takes a snapshot of VALO that contains (−, 2) but not (−, 1) and assigns it to snap. Let t be the time when this happens, and note that t is before the end of the 0 execution interval of op, since t is before thread qp invokes a propose((u, −)) operation on

SAFE[p][x][j], which is before thread qp completes a decide() operation on SAFE[p][x][j],

which is before thread qp completes its simulation of op, which is the end of the execution

interval of op. Furthermore, since VALO contains (−, 2) at time t, by Claim 3.8.12, t ≥ tO. 0O By Deﬁnition 3.8.42, in the linearization L of HP , since u 6= ⊥, op is linearized at the earliest 0 0 time t ≥ tO when a simulation of op executes line 45. Since thread qp executes line 45 of 0 0 0 algorithm AQ within its simulation of op at time t ≥ tO, t exists and t ≤ t. So t is before the end of the execution interval of op. Furthermore, since a simulation of op executes line 45 at time t0, t0 is after the start of the execution interval of op. Thus the linearization time t0 0O of op in the linearization L of HP is within its execution interval. Claim 3.8.43

0O Claim 3.8.44. The linearization L of HP is a strict total ordering of all the operations on the n-P&G 0O object O in HP .

Proof. It suﬃces to show that the linearization times t and t0 of any two distinct operations op and op0 0O 0 0 win in the linearization L of HP are such that t 6= t . There are two cases: either one of op and op is opO , 0 win or neither of op and op is opO .

0 win Case 1. One of op and op is opO . Chapter 3. The Imprecision of the Set Agreement Power Classification 52

win 0 win win Since opO is unique, op and op cannot both be opO . Without loss of generality let op = opO 0 win win win win and op 6= opO . Note that since op = opO , opO 6= nil. Since opO 6= nil, by Observa- win tion 3.8.35, tO 6= ∞. By Deﬁnition 3.8.42, opO is linearized at time tO in the linearization L 0O of HP . So t = tO. By Observation 3.8.11, since tO 6= ∞, some real process executes line 34 of algorithm AQ at time t = tO.

0 win 0 0 Since op 6= opO , by Deﬁnition 3.8.42, at the linearization time t of op , some real process executes 0 one of lines 29, 45, or 57 of algorithm AQ. Since at most one step is taken at any time, t 6= t .

0 win Case 2. Neither of op and op is opO . win Since op 6= opO , by Deﬁnition 3.8.42, at the linearization time t of op, some real process takes 0 win a step (i.e., executes a line of algorithm AQ) in its simulation of op. Since op 6= opO , by Deﬁnition 3.8.42, at the linearization time t0 of op0, some real process takes a step (i.e., executes a 0 0 line of algorithm AQ) in its simulation of op . Since op 6= op , it is easy to see that the steps taken at 0 0 times t and t are distinct. Thus, since at most one step is taken at any time, t 6= t . Claim 3.8.44

0O win win Claim 3.8.45. If there is any propose(−, −) operation in HP , then opO 6= nil and opO is the ﬁrst 0O propose(−, −) operation on O in the linearization L of HP .

0O Proof. Suppose there is a propose(−, −) operation in HP . Then by Claim 3.8.40, tO 6= ∞, and so win win opO 6= nil (Observation 3.8.35). It remains to show that opO is the first propose(−, −) operation 0O in the linearization L of HP . win Suppose, for contradiction, that some operation op 6= opO is the first propose(−, −) operation in 0O 0O the linearization L of HP . Let t be the linearization time of op in to the linearization L of HP , which win by Definition 3.8.42, is the end of the execution interval of op since op 6= opO . Then, since t is the time when op completes, by Claim 3.8.40, tO ≤ t. O Next, note that the propose(−, −) operation op is complete in HP ; otherwise, by Definition 3.8.37, 0O win op would be removed in the construction of HP since op 6= opO . Thus by Definition 3.8.6, at time t, the end of the execution interval of op, a real process executes line 29 of algorithm AQ. By Observation 3.8.11, at time tO, a real process executes line 34 of algorithm AQ. Since at most one step is taken at any time, tO 6= t. Thus, since tO ≤ t, tO < t. win win win Now consider opO . Since opO 6= nil, by Definition 3.8.34, opO is a propose(−, −) operation O win in HP . Furthermore, by Definition 3.8.37, opO is not removed in the construction of the completion 0O O 0O win HP of HP . Moreover, by Definition 3.8.42, in the linearization L of HP , opO is linearized at time tO. 0O win Then, since tO < t, where t is the linearization time of op in the linearization L of HP , opO is linearized win before op. Thus opO is a propose(−, −) operation that is linearized before op in the linearization L of 0O HP — contradicting our assumption that op is the first propose(−, −) operation in the linearization L 0O of HP . Claim 3.8.45

O Claim 3.8.46. For every pair of integers i and r, if a simulation of a get(i) operation in HP performs a writeMax(r) operation on PRIO, then i ≤ 2 and r + i is even.

O Proof. Let i and r be integers such that a simulation of a get(i) operation op in HP performs a writeMax(r) operation on PRIO. From Algorithm 7, it is easy to see that this simulation of op performs the writeMax(r) operation on PRIO when it executes line 57 of algorithm AQ. Thus pri + 1 = r when this simulation of op performs the writeMax(r) operation on PRIO (line 57 of algorithm AQ). Chapter 3. The Imprecision of the Set Agreement Power Classification 53

Immediately before doing so, this simulation of op had to evaluate the conditional on line 56 of algorithm

AQ as true, so i ≤ 2 and pri + i is odd. Thus pri + 1 + i = r + i is even. Claim 3.8.46

Claim 3.8.47. Given any time t and any integer i ∈ {1, 2}, if PRIO + i is odd immediately before time 0O t and even at time t, then there is a get(i) operation in HP that returns ? and is linearized at time t in L.

Proof. Let r be the value of PRIO at time t, i.e., r + i is even. Then, since PRIO + i is odd immediately before time t, and PRIO is a monotonically non-decreasing max register:

PRIO is set to r for the ﬁrst time by a writeMax(r) operation at time t. (3.2)

In algorithm AQ, writeMax(−) operations are performed on PRIO only when a simulation of a get(−) operation on O executes line 57 of algorithm AQ. So let p ∈ [1..|P |] be a virtual process, 0 O q ∈ [1..|Q|] be a real process, j ≥ 1 be an integer, and op be a get(i ) operation in HP invoked by 0 p on some port i ∈ [1..n] of O such that thread qp performs a writeMax(r) operation on PRIO at time t when it executes line 57 in the j-th loop iteration of its simulation of op. By Claim 3.8.46, since a 0 O 0 simulation of the get(i ) operation op in HP performs a writeMax(r) operation on PRIO, then i ≤ 2 and r + i0 is even. Then, since i ∈ {1, 2} and r + i is also even, i = i0. Thus op is a get(i) operation in O HP .

Furthermore, since thread qp executes line 57 of algorithm AQ within its simulation of the get(i) 0O O operation op, by Deﬁnition 3.8.37, op is not removed in the construction of the completion HP of HP O 0O even if op is incomplete in HP . So op is also in HP .

0O Claim 3.8.47.1. The return value of op in HP is ?.

Proof. Suppose, for contradiction, that the return value of op is some value u 6= ?. By Deﬁnition 3.8.37, 0O O in the construction of the completion HP of HP , each incomplete get(−) operation is either removed 0O or completed with return value ?. Thus, since the get(−) operation op is in HP and returns u 6= ?, op O 0 0 is complete in HP . So there is a real process q ∈ [1..|Q|] such that thread qp completes its simulation 0 0 of op with return value u 6= ?. Thus there is an integer j ≥ 1 such that thread qp executes line 60 of 0 algorithm AQ within the j -th loop iteration of its simulation of op. There are two cases: either j > j0, or j ≤ j0.

Case 1. j > j0.

0 0 Consider the j -th loop iteration of the simulations of op by threads qp and qp. Note that since 0 j > j , thread qp completes this loop iteration without executing line 58 or 60. On line 55, both threads perform a decide() operation on SAFE[p][x][j], and assign the response to (val, pri). By Claim 3.8.1, the Agreement property holds for SAFE[p][x][j], so both threads assign the same values to (val, pri) on line 55. Let (ˆv, rˆ) be these values.

0 Since thread qp executes line 60 within this loop iteration, it evaluates the conditional on line 59 0 as true. So thread qp ﬁnds thatv ˆ 6= ?.

Since thread qp completes this loop iteration without executing line 58 or 60, it evaluates the

conditional on line 59 as false. So thread qp ﬁnds thatv ˆ = ? — contradicting the ﬁndings of thread 0 0 qp. Thus it cannot be that j > j . Chapter 3. The Imprecision of the Set Agreement Power Classification 54

Case 2. j ≤ j0.

0 Consider the j-th loop iterations of the simulations of op by threads qp and qp. Note that since 0 0 j ≤ j , thread qp completes this loop iteration. On line 55, both threads perform a decide() operation on SAFE[p][x][j], and assign the response to (val, pri). By Claim 3.8.1, the Agreement property holds for SAFE[p][x][j], so both threads assign the same values to (val, pri) on line 55. Let (ˆv, rˆ) be these values.

Since thread qp executes line 57 within this loop iteration, it evaluates the conditional on lines 56

as true. So thread qp ﬁnds that i ≤ 2 andr ˆ + i is odd.

0 Thus thread qp also ﬁnds that i ≤ 2 andr ˆ + i is odd, and also evaluates the conditional on 0 lines 56 as true. So thread qp executes line 58 within this loop iteration, and eventually completes 0 its simulation of op with return value ? — contradicting the fact that thread qp completes its

simulation of op with return value u 6= ?. Claim 3.8.47.1

Claim 3.8.47.2. No simulation of op executes line 57 before time t.

Proof. Suppose, for contradiction, that some simulation of op executes line 57 before time t. Thus there is a real process q0 ∈ [1..|Q|], an integer j0 ≥ 1, and an integer r0 ≥ 1 such that before time t, thread 0 0 qp performs a writeMax(r ) operation on PRIO when it executes line 57 of algorithm AQ within the j0-th loop iteration of its simulation of op. There are two cases: either j > j0, or j ≤ j0.

Case 1. j > j0.

0 Since thread qp executes line 57 within this loop iteration, it evaluates the conditional on line 56 0 as true. So thread qp ﬁnds that i ≤ 2 andr ˆ + i is odd.

Since thread qp completes this loop iteration without executing line 58 or 60, it evaluates the

conditional on line 56 as false. So thread qp ﬁnds that either i > 2 orr ˆ+ i is even — contradicting 0 0 the ﬁndings of thread qp. Thus it cannot be that j > j .

Case 2. j ≤ j0.

Since thread qp executes line 57 within this loop iteration, it evaluates the conditional on line 56

as true. So thread qp ﬁnds that i ≤ 2 andr ˆ + i is odd. Furthermore, since thread qp performs a

writeMax(r) operation on PRIO when it executes line 57 within this loop iteration,r ˆ = r − 1.

0 Thus thread qp also ﬁnds that i ≤ 2 andr ˆ + i is odd, and also evaluates the conditional on 0 lines 56 as true. So thread qp also executes line 57 within this loop iteration, and also performs Chapter 3. The Imprecision of the Set Agreement Power Classification 55

a writeMax(r) operation on PRIO when it executes line 57 within this loop iteration. Then, 0 since thread qp eventually completes its simulation of op without executing line 57 again, this 0 0 writeMax(r) operation on PRIO is a writeMax(r ) operation on PRIO that thread qp performs

before time t. Thus there is a writeMax(r) operation on PRIO before time t. Then, since

PRIO is a max register whose value is monotonically non-decreasing, PRIO ≥ r before time t —

contradicting (3.2). Claim 3.8.47.2

0O By Claim 3.8.47.1, the return value of op is ? in HP . So by Deﬁnition 3.8.42, in L, op is linearized at the earliest time when a simulation of op executes line 57. Thus by Claim 3.8.47.2, op is linearized at time t, when thread qp executes line 57 within its simulation of op. Claim 3.8.47

0O Claim 3.8.48. Let i ≤ 2 be an integer, and op be a get(i) operation in HP that returns u 6= ?. Then PRIO + i is even immediately before op is linearized in L.

Proof. So suppose i ≤ 2. Let p ∈ [1..|P |] be the virtual process that invokes op, u 6= ? be the return value of op, and x ≥ 1 be an integer such that op is the x-th action of p. By Deﬁnition 3.8.37, in 0O O the construction of the completion HP of HP , each incomplete get(−) operation is either removed or 0O completed with return value ?. Thus, since the get(−) operation op is in HP and returns u 6= ?, op O is complete in HP . So let q ∈ [1..|Q|] be a real process such that thread qp completes its simulation of op the earliest, i.e., the end of the execution interval of op is when thread qp completes its simulation of op. Note that since thread qp completes its simulation of op with return value u 6= ?, there is an integer j such that thread qp executes line 60 of algorithm AQ within the j-th loop iteration of its simulation of op.

Consider the j-th loop iteration of the simulation of op by thread qp. Since thread qp completes its simulation of op with return value u, val = u when thread qp executes line 60. Furthermore, since thread qp executes line 60, it does not execute line 58. So thread qp ﬁnds that either i > 2 or pri + i is even when it evaluates the conditional on line 56. Since i ≤ 2, it ﬁnds that pri + i is even. Let r be the value of pri when thread qp evaluates the conditional on line 56, so r + i is even. Thus earlier in this loop iteration, on line 55 of algorithm AQ, thread qp receives (u, r) from a decide() operation on the safe agreement object SAFE[p][x][j] and assigns (u, r) to (val, pri). By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so a propose((u, r)) operation is invoked on SAFE[p][x][j] before thread qp receives (u, r) from a decide() operation on SAFE[p][x][j]. Since op is a get(−) operation on the n-P&G object O, according to Algorithm 7, SAFE[p][x][j] is only accessed by simulations of op within the j-th loop iteration (line 53 and 55 of algorithm AQ). So let 0 0 q ∈ [1..|Q|] be a real process such that thread qp invokes a propose((u, r)) operation on SAFE[p][x][j]

(line 53 of algorithm AQ) within the j-th loop iteration of its simulation of op before thread qp receives (u, r) from a decide() operation on SAFE[p][x][j]. 0 0 Now consider the j-th loop iteration of the simulation of op by thread qp. Since thread qp invokes 0 a propose((u, r)) operation on SAFE[p][x][j] on line 53, (val, pri) = (u, r) when thread qp executes 0 line 53. So on line 51, thread qp receives r from a readMax() operation on PRIO and assigns r to pri. 0 0 Let t be the time when thread qp performs this readMax() operation on PRIO that returns r. So 0 0 at time t , PRIO = r, and since r + i is even, PRIO + i is even. Note that t is within the execution 0 0 0 interval of op, since a simulation of op executes line 51 at time t , and time t is before thread qp invokes a propose((u, r)) operation on SAFE[p][x][j], which is before thread qp receives (u, r) from a decide() Chapter 3. The Imprecision of the Set Agreement Power Classification 56

operation on SAFE[p][x][j], which is before thread qp completes its simulation of op, which is the end of the execution interval of op. 0 0 Next, let t < t be the time when thread qp takes a snapshot of VALO on line 45 within this j-th loop iteration of its simulation of op. Then let tL be the linearization time of op in L. We will show that t ≤ tL. There are two cases: either u = ⊥, or u 6= ⊥.

Case 1. u = ⊥.

Then note that in Algorithm 7, (⊥, 2) is never written into VALO. Thus no snapshot of VALO 0 contains (⊥, 2), and so ⊥ is never assigned to val when a thread executes line 50. So thread qp 0 assigns u = ⊥ to val on line 47. Thus thread qp evaluates the conditional on line 46 as true. So 0 at time t, when thread qp takes a snapshot of VALO on line 45 and assigns it to snap, all ﬁelds of

VALO are (−, 0). Thus by Claim 3.8.13, t < tO.

So a simulation of op executes line 45 at time t < tO. Recall that by Deﬁnition 3.8.42, in L, each

get(−) operationop ˆ that returns ⊥ is linearized at the earliest time t < tO when a simulation of

opˆ executes line 45. Thus, since op returns u = ⊥, op is linearized at time tL ≤ t in L.

Case 2. u 6= ⊥.

0 0 Then, since u 6= ?, thread qp assigns u to val on line 50. Thus thread qp evaluates the conditional 0 on line 48 as true. So at time t, when thread qp takes a snapshot of VALO on line 45 and assigns

it to snap, VALO contains (−, 2) but not (−, 1). Thus by Claim 3.8.12, t ≥ tO.

So a simulation of op executes line 45 at time t ≥ tO. Recall that by Deﬁnition 3.8.42, in L, each

get(−) operationop ˆ that returns neither ? nor ⊥ is linearized at the earliest time t ≥ tO when a simulation ofop ˆ executes line 45. Thus, since op returns a value u such that u 6= ? and u 6= ⊥, op

is linearized at time tL ≤ t in L.

So in both cases, tL ≤ t. Note that by Claim 3.8.43, tL is within the execution interval of op.

Since op is linearized at time tL in L, it suﬃces to show that PRIO + i is even immediately before time tL. Suppose, for contradiction, that PRIO + i is odd immediately before time tL. Recall that 0 0 0 PRIO + i is even at time t . Moreover, since tL ≤ t and t < t , t > tL. Thus there is a time tˆ such 0 that tL ≤ tˆ < t , and PRIO + i is even at time tˆ but odd immediately before time tˆ. Note that tˆ is in 0 the execution interval of op, since both tL and t are in the execution interval of op. Furthermore, by 0 0O ˆ Claim 3.8.47, there is a get(i) operation op on O that returns ? in HP and is linearized at time t in L. Note that since the return value of op0 is ? and the return value of op is u 6= ?, op 6= op0. Moreover, by Claim 3.8.43, since op0 is linearized at time tˆ in L, tˆ is also within the execution interval of op0. Thus at time tˆ, op and op0 are two distinct pending operations on the same port i of the n-P&G object O — contradicting Claim 3.8.8. Claim 3.8.48

0O Claim 3.8.49. For some integer i ≤ 2, let op be a get(i) operation in HP . Then PRIO + i is even at the linearization time of op in L.

Proof. There are two cases: either op returns ?, or op returns a value u 6= ?.

Case 1. op returns ?. Then by Deﬁnition 3.8.42, in L, op is linearized at the earliest time when a simulation of op executes

line 57, i.e., when it performs a writeMax(pri + 1) operation on PRIO. So PRIO = pri + 1 when Chapter 3. The Imprecision of the Set Agreement Power Classification 57

op is linearized. Immediately before doing so, this simulation of op evaluates the conditional on

line 56 as true, i.e., it ﬁnds that pri + i is odd. Thus when op is linearized, PRIO + i = pri + 1 + i is even.

Case 2. op returns a value u 6= ?.

Then by Claim 3.8.48, since i ≤ 2, PRIO + i is even immediately before op is linearized. By Deﬁnition 3.8.42, when op is linearized, a simulation of op executes line 45, which does not access

PRIO. Recall that we assume at most one step is taken at any time. That implies that at most

one thread takes a step at any time, so no other steps occur when op is linearized. Thus PRIO + i

is still even when op is linearized. Claim 3.8.49

0O By Claim 3.8.44, the linearization L of HP is a sequential history of (all the operations on) the 0O n-P&G object O in HP . By Deﬁnition 3.2:

• A get(2) operation op is prioritized in L if and only if it is preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op.

• A get(1) operation op is prioritized in L if and only if it is not preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op.

• Every get(i) operation with i > 2 is prioritized in L.

0O Claim 3.8.50. For each get(−) operation op in HP , op returns ? if and only if op is not prioritized in L.

Proof. Let i ∈ [1..n] be the port of O that op is applied on, i.e., op is a get(i) operation. [If] Suppose, for contradiction, that op is not prioritized in L, but op does not return ?. By Deﬁnition 3.2, since op is not prioritized, one of the following is true:

0O 0 • i = 1 and in the linearization L of HP , op is preceded by a get(2) operation op such that there is no get(1) operation between op0 and op.

0O 0 • i = 2 and in the linearization L of HP , op is not preceded by a get(2) operation op such that there is no get(1) operation between op0 and op.

0O 0 Case 1. i = 1 and in the linearization L of HP , op is preceded by a get(2) operation op such that there is no get(1) operation between op0 and op. Let t and t0 be the linearization times of op and op0 respectively in L. Note that t0 < t, since op0

is linearized before op. At time t, since op is a get(1) operation, by Claim 3.8.49, PRIO + 1 is 0 0 even, and so PRIO is odd. At time t , since op is a get(2) operation, by Claim 3.8.49, PRIO + 2

is even, and so PRIO is even. 0 0 Let tˆ be the earliest time after t when PRIO is odd. Note that tˆ exists and t < tˆ≤ t, since PRIO 0 0 is even at time t and odd at time t. Since tˆ is the earliest time after t when PRIO + 1 is even,

PRIO + 1 is odd immediately before time tˆ. So by Claim 3.8.47, there is a get(1) operationop ˆ in 0O ˆ HP that returns ? and is linearized at time t in L. Note thatop ˆ 6= op, since op does not return ?. Then, since op is linearized at time t andop ˆ 6= op is linearized at time tˆin L, tˆ6= t by Claim 3.8.44. Therefore, since tˆ≤ t, tˆ < t. So in L,op ˆ is a get(1) operation linearized between op0 and op in L — contradicting the fact that there is no such operation. Chapter 3. The Imprecision of the Set Agreement Power Classification 58

0O 0 Case 2. i = 2 and in the linearization L of HP , op is not preceded by a get(2) operation op such that there is no get(1) operation between op0 and op.

0O Consider the linearization L of HP . There are two possible explanations for why op is not preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op:

• There is at least one get(2) operation that precedes op, but for each get(2) operation op0 that precedes op there is a get(1) operation between op0 and op. • op is not preceded by any get(2) operation.

Case 2(a). There is at least one get(2) operation that precedes op, but for each get(2) operation op0 that precedes op there is a get(1) operation between op0 and op. Let op0 be one of the get(2) operations that precede op, and let op∗ be the latest get(1) operation between op0 and op. Then it is straightforward to see that op is preceded by this get(1) operation op∗ such that there is no get(2) operation between op∗ and op. Let t and t∗ be the linearization times of op and op∗ respectively in L. Note that t∗ < t, since op∗ is linearized before op. At time t, since op is a get(2) operation, by Claim 3.8.49, ∗ ∗ PRIO + 2 is even, and so PRIO is even. At time t , since op is a get(1) operation, by

Claim 3.8.49, PRIO + 1 is even, and so PRIO is odd. ∗ ∗ Let tˆ be the earliest time after t when PRIO is even. Note that tˆ exists and t < tˆ≤ t, since ∗ ∗ PRIO is odd at time t and even at time t. Since tˆ is the earliest time after t when PRIO + 2

is even, PRIO + 2 is odd immediately before time tˆ. So by Claim 3.8.47, there is a get(2) 0O ˆ operationop ˆ HP that returns ? and is linearized at time t in L. Note thatop ˆ 6= op, since op does not return ?. Then, since op is linearized at time t andop ˆ 6= op is linearized at time tˆ in L, tˆ6= t by Claim 3.8.44. Therefore, since tˆ≤ t, tˆ < t. Soop ˆ is a get(2) operation linearized between op∗ and op in L — contradicting the fact that there is no such operation. Case 2(b). op is not preceded by any get(2) operation. 0O Let t be the linearization time of op in L. Since op is a get(2) operation in HP , by Claim 3.8.49, PRIO is even at time t. Note that at time 0, PRIO contains the odd initial value 1.

Then let tˆ be the earliest time when PRIO is even. Note that tˆ exists and 0 < tˆ ≤ t, since

PRIO is even at time t and odd at time 0. Since tˆ is the earliest time when PRIO + 2 is even,

PRIO + 2 is odd immediately before time tˆ. So by Claim 3.8.47, there is a get(2) operation 0O ˆ opˆ in HP that returns ? and is linearized at time t in L. Note thatop ˆ 6= op, since op does not return ?. Then, since op is linearized at time t andop ˆ 6= op is linearized at time tˆ in L, tˆ 6= t by Claim 3.8.44. Therefore, since tˆ ≤ t, tˆ < t. Soop ˆ is a get(2) operation linearized before op in L — contradicting the fact that there is no such operation.

[Only If] Suppose, for contradiction, that op is prioritized in L, but op returns ?. By Deﬁnition 3.2, since op is prioritized, one of the following is true:

• i > 2.

• i = 2 and op is preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op. Chapter 3. The Imprecision of the Set Agreement Power Classification 59

• i = 1 and op is not preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op.

Case 1. i > 2. Then every simulation of op always evaluates the conditional on line 56 as false. So no simulation of op executes line 57 or 58. Then by Deﬁnition 3.8.37, the get(−) operation op is complete in O HP . So some simulation of op eventually completes. Consider such a complete simulation of op. Since no simulation of op executes line 58, this complete simulation of op returns a value u by executing line 60. Before doing so, this simulation of op evaluates the conditional on line 59 as true, and so it returns u 6= ? — contradicting our assumption that op returns ?.

Case 2. i = 2 and op is preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op. Then let t and t0 be the linearization times of op and op0 respectively in L. Note that t0 < t, since 0 0 op is linearized before op. Since both op and op are get(2) operations, by Claim 3.8.49, PRIO is even at times t and t0. Next, let p ∈ [1..|P |] be a virtual process and x ≥ 1 be an integer such that op is the x-th action of

p. By Deﬁnition 3.8.42, there is a real process q ∈ [1..|Q|] and an integer j ≥ 1 such that thread qp executes line 57 in the j-th loop iteration of its simulation of op at time t, the linearization time of op in L.

Consider this j-th loop iteration of the simulation of op by thread qp. Since thread qp executes line 57, it ﬁnds that pri + i is odd when it evaluates the conditional on line 56. Since i = 2, it ﬁnds

that pri is an odd integer r. Thus earlier in this loop iteration, on line 55, thread qp receives (−, r) from a decide() operation on the safe agreement object SAFE[p][x][j] and assigns r to pri before time t.

By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so a propose((−, r)) operation is

invoked on SAFE[p][x][j] (line 53) before thread qp receives (−, r) from a decide() operation on SAFE[p][x][j] (line 55). Since op is a get(−) operation on the n-P&G object O, according to Algorithm 7, SAFE[p][x][j] is only accessed by simulations of op within the j-th loop iteration 0 0 (line 53 and 55 of algorithm AQ). So there is a real process q ∈ [1..|Q|] such that thread qp invokes a propose((−, r)) operation on SAFE[p][x][j] (line 53) within the j-th loop iteration of

its simulation of op before thread qp receives (−, r) from a decide() operation on SAFE[p][x][j] (line 55), and so before time t.

0 0 Consider this j-th loop iteration of the simulation of op by thread qp. Since thread qp invokes a 0 propose((−, r)) operation on line 53, pri = r when thread qp executes line 53. Thus earlier in 0 this loop iteration, on line 51, thread qp receives the odd integer r from a readMax() operation

on PRIO and assigns it to pri. Let tread be the time when this happens, so tread < t and PRIO is

odd at time tread.

0 0 There are two cases: either tread ≤ t or tread > t .

0 Case 2(a). tread ≤ t . Chapter 3. The Imprecision of the Set Agreement Power Classification 60

0 So tread ≤ t < t. Then note that since a simulation of op executes line 51 at time tread, tread is after the start of the execution interval of op. Furthermore, by Claim 3.8.43, since t is the linearization time of op in L, t is within the execution interval of op. Thus t0 is also within the execution interval of op. Since op0 precedes op in L, op0 6= op. Thus op and op0 are two distinct pending operations on port 2 of O at time t0 — contradicting Claim 3.8.8.

0 Case 2(b). tread > t . 0 0 Then let tˆ be the earliest time after t when PRIO is odd. Note that tˆ exists and t < tˆ ≤ 0 tread < t, since PRIO is even at time t and odd at time tread. Since tˆ is the earliest time 0 after t when PRIO is odd, PRIO + 1 is odd immediately before time tˆ and even at time tˆ. 0O ˆ So by Claim 3.8.47, there is a get(1) operationop ˆ in HP that is linearized at time t, before op since tˆ < t, and after op0 since t0 < tˆ — contradicting the fact that there is no get(1) operation between op0 and op.

Case 3. i = 1 and op is not preceded by a get(2) operation op0 such that there is no get(1) operation between op0 and op. Then there are two cases: either op is preceded by a get(1) operation op0 such that there is no get(2) operation between op0 and op, or op is not preceded by a get(2) operation. In the former, we derive a contradiction via arguments symmetric to Case 2. So it suﬃces to consider the latter case where op is not preceded by a get(2) operation.

Let t be the linearization time of op in L. Since op is a get(1) operation, by Claim 3.8.49, PRIO

is odd at time t. Note that at time 0, PRIO contains the odd initial value 1. Let p ∈ [1..|P |] be a virtual process and x ≥ 1 be an integer such that op is the x-th action of p.

By Deﬁnition 3.8.42, there is a real process q ∈ [1..|Q|] and an integer j ≥ 1 such that thread qp executes line 57 in the j-th loop iteration of its simulation of op at time t, the linearization time of op in L.

Consider this j-th loop iteration of the simulation of op by thread qp. Since thread qp executes line 57, it ﬁnds that pri + i is odd when it evaluates the conditional on line 56. Since i = 1, it

ﬁnds that pri is an even integer r. Thus earlier in this loop iteration, on line 55, thread qp receives (−, r) from a decide() operation on the safe agreement object SAFE[p][x][j] and assigns r to pri before time t. By Claim 3.8.1, the Validity property holds for SAFE[p][x][j], so a propose((−, r)) operation is

its simulation of op before thread qp receives (−, r) from a decide() operation on SAFE[p][x][j] (line 55), and so before time t.

on PRIO and assigns it to pri. Let tread be the time when this happens, so tread < t and PRIO is

even at time tread.

Then let tˆ be the earliest time when PRIO is even. Note that tˆ exists and 0 < tˆ ≤ tread < t, since

PRIO is odd at time 0 and even at time tread. Since tˆ is the earliest time when PRIO is even,

PRIO + 2 is odd immediately before time tˆ and even at time tˆ. So by Claim 3.8.47, there is a 0O ˆ ˆ get(2) operationop ˆ in HP that is linearized at time t, before op since t < t — contradicting the fact that op is not preceded by a get(2) operation. Claim 3.8.50

Claim 3.8.51. For each prioritized get(−) operation op in L, if op is linearized at some time t < tO, win win then op returns ⊥; otherwise vO 6= nil and op returns vO .

win win Proof. Let u be the return value of op. By Claim 3.8.41, either u = ?, u = ⊥, or u = vO and vO 6= nil. Since op is prioritized in L, by Claim 3.8.50, u 6= ?. 0 0O Now recall that by Deﬁnition 3.8.42, for each get(−) operation op in HP :

• If op0 returns ?, then in L, op0 is linearized at the earliest time when a simulation of op0 executes line 57.

0 0 0 0 • If op returns ⊥, then in L, op is linearized at the earliest time t < tO when a simulation of op executes line 45.

0 0 0 • If op returns neither ? nor ⊥, then in L, op is linearized at the earliest time t ≥ tO when a simulation of op0 executes line 45.

So, since u 6= ?, u = ⊥ if and only if op is linearized before tO. Thus op returns ⊥ if it is linearized win win before tO; otherwise it returns vO and vO 6= nil. Claim 3.8.51

0O Claim 3.8.52. The linearization L of HP respects the sequential speciﬁcation of the n-P&G object (Deﬁnition 3.3).

Proof. There are four cases:

Case 1. op is a prioritized get(−) operation and op is preceded by a propose(−, −) operation in L.

win We will show that in this case op returns vO , consistent with Definition 3.3. Since op is preceded 0O by a propose(−, −) operation, there is at least one propose(−, −) operation in HP . So by win win Claim 3.8.45, opO 6= nil and opO is the first propose(−, −) operation on O in L. Thus by win win Definition 3.8.34, vO 6= nil and vO is the proposal value of the first propose(−, −) operation win win on O, namely opO . So it suffices to show that op returns vO . win First, since opO is the first propose(−, −) operation, and op is preceded by a propose(−, −) win win operation, opO precedes op in L. By Definition 3.8.42, opO is linearized at time tO in L. Thus win op is linearized at some time t > tO. So by Claim 3.8.51, op returns vO .

Case 2. op is a prioritized get(−) operation and op is not preceded by a propose(−, −) operation in L.

Let t be the time when op is linearized. There are two subcases: either t < tO, or t ≥ tO. In the former, by Claim 3.8.51, op returns ⊥, consistent with Deﬁnition 3.3. We now show that the latter

subcase, where t ≥ tO, is impossible. Chapter 3. The Imprecision of the Set Agreement Power Classification 62

win Since t ≥ tO, tO 6= ∞. So by Observation 3.8.35, opO 6= nil, and thus by Definition 3.8.34, win O win opO is a propose(−, −) operation in HP . By Definition 3.8.37, opO is not removed in the 0O O win 0O win construction of the completion HP of HP , so opO is in HP . By Definition 3.8.42, opO is linearized at time tO in L.

win win Since op is a get(−) operation and opO is a propose(−, −) operation, op 6= opO . By win Claim 3.8.44, no two operations are linearized at the same time in L. Thus, since opO is linearized win at time tO, and op 6= opO is linearized at time t, t 6= tO. Then, since t ≥ tO and t 6= tO, t > tO. win So, since op is linearized at time t > tO and opO is linearized at time tO in L, the propose(−, −) win operation opO precedes op — contradicting the fact that op is not preceded by a propose(−, −) operation.

Case 3. op is an unprioritized get(−) operation in L. Then by Claim 3.8.50, op returns ?, consistent with Deﬁnition 3.3.

Case 4. op is a propose(−, −) operation in L.

O If op is complete in HP , then every complete simulation of op returns done on line 41, so by O 0O Definition 3.8.6, op returns done in HP (and so in HP as well). If the propose(−, −) operation O 0O op is incomplete in HP , then by Definition 3.8.37, op is completed with return value done in HP . Either way, op returns done, consistent with Definition 3.3. Claim 3.8.52

Claim 3.8.53. The k + 1 real processes that execute the simulation algorithm AQ in system Q correctly simulate an execution of the simulated algorithm AP by the virtual processes of system P where every virtual process begins the execution with an input of some real process. In other words, HP is the history of an execution of algorithm AP in system P where the input value of each virtual process is the input of some real process.

Proof. Recall that: (1) AQ is an extension of the original BG simulation algorithm, and (2) this extension consists only of code added to simulate accesses to n-P&G objects by the virtual processes that execute algorithm AP in system P . In the case that AP never accesses n-P&G objects (i.e., when AP accesses only registers), the original BG simulation algorithm correctly simulates an execution of algorithm AP by the virtual processes of system P where every virtual process begins the execution with an input of some real process. Thus to show that our extended BG simulation AQ correctly simulates the algorithm AP

(even if it accesses n-P&G objects), it suffices to show that AQ correctly simulates every n-P&G object accessed by the virtual processes that execute AP in system P . More precisely, it suffices to show that, O for every n-P&G object O, the history HP of operations on O in the simulated history HP (of the algorithm AP in system P ) is linearizable with respect to the sequential specification of the n-P&G object. This was shown in Claim 3.8.52. Claim 3.8.53

Claim 3.8.54. The simulation algorithm AQ is a wait-free algorithm that solves the k-set agreement problem among k + 1 real processes.

Proof. Recall that AP is a wait-free algorithm that solves the k-set agreement problem. By Claim 3.8.53,

AQ correctly simulates the execution of algorithm AP by the virtual processes of system P , where every virtual process begins the execution with an input of some real process. Let q be an arbitrary correct real process in Q. By Claim 3.8.27, there is a virtual process p ∈ [1..|P |] such that |σq,p| = ∞, i.e., q Chapter 3. The Imprecision of the Set Agreement Power Classification 63

Algorithm 9 Solving the n-SLC problem using an n-P&G object D

Code executed by each process q ∈ [1..n] with input vq: 1: D.propose(vq) on port q 2: repeat 3: val ← D.get() on port q 4: until val 6= ? 5: decide val

simulates inﬁnitely many steps of p. Since AP is a wait-free algorithm that solves the k-set agreement problem, and AQ correctly simulates AP , the inﬁnite sequence of steps of virtual process p simulated by q must contain a decide(−) action. So each correct real process q ∈ Q eventually simulates a virtual process p ∈ P deciding some value u (line 25), and decides u itself (line 27). Thus AQ is a wait-free algorithm that also solves the k-set agreement problem among k + 1 real processes. Claim 3.8.54

By Claim 3.8.54, the simulation algorithm AQ is a wait-free algorithm that solves the k-set agreement problem among k+1 real processes even though it only uses registers, which is known to be impossible [7, 28, 44]. This contradiction concludes the proof that, for n > 2 and k ≥ 1, there is no wait-free algorithm that solves the (k(n − 1) + 1, k)-SA problem using only n-P&G objects and registers. Theorem 3.8

This completes our proof of Theorem 3.8. By Corollary 3.7, Theorem 3.8, and Theorem 3.5:

Corollary 3.9. For all n > 2, the n-P&G object has the same set agreement power as the (n − 1)- consensus object.

3.4 (n − 1)-Consensus Cannot Implement the n-P&G Object

In this section, we show that the n-P&G and (n − 1)-consensus objects are not equivalent. To do so, for all n > 2, we ﬁrst introduce a variation of the n-consensus problem that we call n-SLC (for “Semi-Live Consensus”). The n-SLC problem is identical to the wait-free n-consensus problem, except that the processes q ∈ {1, 2} are only required to be obstruction-free [25], instead of wait-free. More precisely, an algorithm solves the n-SLC problem if every execution of the algorithm satisﬁes the following properties:

• Agreement: No two processes decide diﬀerent values.

• Validity: If a process decides v, then v is the input of some process.

• Termination:

(a) If any process q ∈ {1, 2} takes inﬁnitely many steps solo (i.e., steps that are not interleaved with steps of other processes), then q eventually decides. (b) If any process q ∈ [3..n] takes inﬁnitely many steps, then q eventually decides.

Theorem 3.10. For all n > 2, the n-SLC problem can be solved using an n-P&G object.

Proof. For each process q ∈ [1..n], let vq be the input value of process q. To solve the n-SLC problem with an n-P&G object D, each process q executes Algorithm 9. Note that each process q ∈ [1..n] only Chapter 3. The Imprecision of the Set Agreement Power Classification 64 performs operations on port q of the n-P&G object D. Thus this algorithm satisfies the requirement that no two operations are applied on the same port of an object concurrently. In Algorithm 9, each process performs a propose(−, −) operation on D with its own input value (line 1), and repeatedly performs get(−) operations until it receives a non-? value (lines 2 to 4), which it then decides in line 5. We now show that the properties of the n-SLC problem, namely, Validity, Agreement, Termination (a), and Termination (b), hold. Since each process performs a propose(−, −) operation before performing any get(−) operation, by Lemma 3.1(1) no get(−) operation on D returns ⊥. Thus, if a process decides a value v (in line 5), then v was returned by a get(−) operation in line 3, and v∈ / {⊥, ?}. So, by Lemma 3.1(2), v was proposed by the first propose(−, −) operation on D, and therefore Validity and Agreement hold. Next, suppose a process q ∈ {1, 2} takes infinitely many steps solo. Then, since process q only performs operations on port q of D, there is a time after which there are no get(i) operations on D for some i ∈ {1, 2}. Thus by Lemma 3.1(3), q will eventually exit the loop of lines 2-4, and will decide; so Termination (a) holds. Finally, suppose a process q ∈ [3..n] takes infinitely many steps. Since q ∈ [3..n], by Lemma 3.1(4), the get(q) operation that process q performs on D does not return ?. Thus q will eventually exit the loop of lines 2-4, and decide; so Termination (b) holds. Theorem 3.10

Theorem 3.11. For all n > 2, the n-SLC problem cannot be solved using only (n − 1)-consensus objects and registers.

Proof. Consider the n-DAC problem introduced by Hadzilacos and Toueg [22]. This problem is identical to the obstruction-free consensus problem, except that there is a distinguished process p that is wait-free, but may abort instead of deciding if any other process takes a step. Without loss of generality, let the distinguished process p be process n. Then an algorithm solves the n-DAC problem if every execution of the algorithm satisﬁes the following properties:

• Agreement: If a process q decides v and a process q0 decides v0, then v = v0.

• Validity: If a process decides v, then v is the input of some process that does not abort.

• Termination:

(a) If any process q ∈ [1..n − 1] takes inﬁnitely many steps solo (i.e., not interleaved with steps of other processes), then q eventually decides. (b) If the process p = n takes inﬁnitely many steps, then q eventually decides or aborts.

• Nontriviality: If the process p = n aborts, then some process q < n took at least one step.

We will show that for all n > 2, the n-SLC problem is at least as hard as the n-DAC problem. To do so, we ﬁrst strengthen the n-DAC problem by removing the option to abort. The resulting problem, denoted n-DAC 0, satisﬁes the following properties:

• Agreement: If a process q decides v and a process q0 decides v0, then v = v0.

• Validity: If a process decides v, then v is the input of some process.

• Termination: Chapter 3. The Imprecision of the Set Agreement Power Classification 65

It is now clear that, for n > 2, the n-SLC problem is at least as hard as the n-DAC0 problem: They have the same Agreement and Validity properties, and (for n > 2) the Termination requirement of n-SLC implies that of n-DAC0 problem. Thus, the n-SLC problem is at least as hard as the n-DAC problem. The theorem now follows from the fact that the n-DAC problem cannot be solved using only (n − 1)- 6 consensus objects and registers [22]. Theorem 3.11

Theorem 3.12. For all n > 2, the n-P&G object cannot be implemented using only (n − 1)-consensus objects and registers.

Proof. Let n > 2. Suppose, for contradiction, that there exists a wait-free implementation I of the n-P&G object that uses only (n − 1)-consensus objects and registers. Without loss of generality, we assume that in this implementation I, every complete operation on the implemented n-P&G object performs at least one operation on a base object of I. This assumption is without loss of generality because we can modify any wait-free implementation of an n-P&G object D0 by adding a base register that every operation on the implemented object D0 reads once and then ignores. In the proof of Theorem 3.10, we showed that Algorithm 9 solves the n-SLC problem using an n-P&G object D. Let A denote this algorithm. Then let A0 be a modiﬁcation of A where the n-P&G object D used by A is replaced with the n-P&G object D0 implemented by the wait-free implementation I. Since I uses only (n − 1)-consensus objects and registers, the algorithm A0 also uses only (n − 1)-consensus objects and registers.

Claim 3.12.1. Every history H0 of A0 corresponds to a history H of A such that:

(a) Each process has the same input value in H as in H0.

(b) Each process decides the same value in H as in H0.

(d) If a process p takes only ﬁnitely many steps in H, then p also takes only ﬁnitely many steps in H0.

(e) If a process p takes inﬁnitely many steps in H, then p also takes inﬁnitely many steps in H0.

Proof. Parts (a), (b), and (c) are obvious. Suppose a process p takes only finitely many steps in the history H of A. From the algorithm A (Algorithm 9), it is clear that p completes only finitely many operations on the n-P&G object D in the history H of A. Thus p completes only finitely many operations on the implemented n-P&G object D0 in the corresponding history H0 of A0. Since the implementation I of D0 is wait-free, every operation on the implemented object D0 takes only finitely many steps in H0. Thus p takes only finitely many steps in H0. So part (d) holds. Suppose a process p takes infinitely many steps in the history H of A. From the algorithm A (Algorithm 9), it is clear that p completes infinitely many operations on the n-P&G object D in the history H of A. Thus p completes infinitely many operations on the implemented n-P&G object D0

6A proof of a slightly stronger result is also given in [11] (Theorem 4.2). Chapter 3. The Imprecision of the Set Agreement Power Classification 66 in the corresponding history H0 of A0. By assumption on the implementation I of D0, each complete operation on D0 performs at least one operation on a base object of I. Thus p takes inﬁnitely many 0 steps in H . So part (e) holds. Claim 3.12.1

Since A0 uses only (n−1)-consensus objects and registers, by Theorem 3.11, A0 cannot solve the n-SLC problem. In other words, there exists a history H0 of A0 that violates one of the properties of the n-SLC problem. Let H be the history of A that corresponds to this history H0 of A0. There are four cases:

Case 1. H0 violates the Agreement property of the n-SLC problem. This implies that in H0, some process p decides a value v, and some process p0 decides a value v0 6= v. By Claim 3.12.1(b), in H, p also decides v, and p0 also decides v0 6= v. Thus the history H of A violates the Agreement property of the n-SLC problem.

Case 2. H0 violates the Validity property of the n-SLC problem. This implies that in H0, some process p decides a value v that is not the input of any process. By Claim 3.12.1(a) and (b), in H, p also decides a value v that is not the input of any process. Thus the history H of A violates the Validity property of the n-SLC problem.

Case 3. H0 violates the Termination (a) property of the n-SLC problem. This implies that in H0, some process q ∈ {1, 2} takes infinitely many steps solo but never decides. So in H0, q takes infinitely many steps, whereas every process p 6= q takes only finitely many steps. Thus:

• By Claim 3.12.1(c), q also never decides in H. • By Claim 3.12.1(d), q also takes inﬁnitely many steps in H. • By Claim 3.12.1(e), every process p 6= q also takes only ﬁnitely many steps in H.

So in H, the process q also takes inﬁnitely many steps solo but never decides. Thus the history H of A violates the Termination (a) property of the n-SLC problem.

Case 4. H0 violates the Termination (b) property of the n-SLC problem. This implies that in H0, some process q ∈ [3..n] takes infinitely many steps but never decides. By Claim 3.12.1(c), q also never decides in H. By Claim 3.12.1(d), q also takes infinitely many steps in H. So in H, the process q ∈ [3..n] also takes infinitely many steps but never decides. Thus the history H of A violates violating the Termination (b) property of the n-SLC problem.

In all cases, the history H of A violates some property of the n-SLC problem — contradicting the fact that A solves the n-SLC problem. Theorem 3.12

Corollary 3.9 and Theorem 3.12 imply:

Corollary 3.13. For all n > 2, level n − 1 of Herlihy’s consensus hierarchy contains two deterministic objects, namely the n-P&G object and the (n − 1)-consensus object, that have the same set agreement power but are not equivalent.

Thus the set agreement power classiﬁcation of deterministic objects is not precise. Chapter 3. The Imprecision of the Set Agreement Power Classification 67

3.5 Objects Versus Tasks

In distributed computing, a task [29] is a problem that is speciﬁed by a function that takes a vector as input and outputs a set of vectors. Intuitively, an algorithm solves a task T if in every execution of the algorithm, the vector I~ of the inputs of each process and the vector O~ of the outputs of each process are such that O~ ∈ T (I~). Wait-free objects and tasks [29] are fundamental concepts of distributed computing, and a natural question is whether objects and tasks are equivalent. By combining our result with a result of Gafni and Kuztnetsov [19], we now show that, in general, the answer is no: more precisely, we prove that in systems with m > 2 processes, there is a deterministic object that is not equivalent to any task.

Theorem 3.14. For systems with m > 2 processes, the deterministic wait-free object m-P&G is not equivalent to any wait-free task.

Proof. Suppose, for contradiction, that in a system with m > 2 processes, there is a task T that is equivalent to the m-P&G object, i.e., (a) T can be solved in a wait-free manner using m-P&G objects and registers, and (b) the m-P&G object can be implemented in a wait-free manner using any wait-free solution for T and registers. By Corollary 3.9, there is no algorithm that solves the consensus problem for m processes using only m-P&G objects and registers. Gafni and Kuztnetsov proved the following result [19]: “Let A be a deterministic object such that no protocol solves consensus among n + 1 processes using copies of A and read-write registers. If a task T is wait-free solvable by n + 1 processes using read-write shared-memory and copies of A, then T is also wait-free solvable when copies of A are replaced with n-consensus objects.” Let n + 1 = m and A be our deterministic object m-P&G. By the above result and (a), T can be solved in a wait-free manner using (m−1)-consensus objects and registers. So by (b), the m-P&G object can be implemented in a wait-free manner using (m − 1)-consensus objects and registers — contradicting

Theorem 3.12. Theorem 3.14

3.6 BG Simulation and Extensions

The original BG simulation algorithm introduced by Borowsky and Gafni [7] was designed to simulate algorithms that use registers to solve colorless problems (intuitively, in a colorless problem, each process can adopt the input of any process to use as its own input, and adopt the output of any process to use as its own output).7 In that simulation, each real process that crashes can block at most one virtual process. Chaudhuri and Reiners [14] extended the BG simulation algorithm to simulate algorithms that use (n, k)-set agreement objects in addition to registers, and Imbs and Raynal [32] presented a simpler extension for the special case where k = 1, i.e., for simulating n-consensus objects. In both [14, 32], however, the simulation of n-consensus objects comes at a cost: each real process crash may now block up to n virtual processes. In this thesis, we extended the BG simulation algorithm to simulate the access to n-P&G objects, which are slightly weaker than n-consensus objects. Our simulation was carefully crafted to take advantage of this weakness to reduce the number of processes that may be blocked: in our simulation, each real process crash can block at most n − 1 virtual processes. Roughly speaking, this reduction in the number of blocked processes was achieved as follows.

7The BG simulation algorithm was later generalized in [18, 31] to algorithms that use registers to solve arbitrary problems, colorless or colored. Chapter 3. The Imprecision of the Set Agreement Power Classification 68

When a virtual process accesses an n-consensus object C, it can never return without getting the agreed upon proposal value. So if the crash of a real process prevents this agreed upon proposal value from being determined, every virtual process that accesses C may be forever prevented from returning, i.e., may be blocked. In fact, if C is simulated by a safe-agreement object Osafe (as in [14, 32]), and a real process crashes while proposing a value to Osafe, all operations on C may loop forever, unable to return because they cannot determine the agreed upon proposal value. In contrast to an n-consensus object, a virtual process that performs a get(-) operation on an n-P&G object returns ? without getting the agreed upon proposal value, if this get(-) operation is not prioritized. So we simulated each n-P&G object by adding an escape clause to the loop of the safe agreement algorithm: in each loop iteration, a real process also checks the pri field of the n-P&G object to determine whether the get(−) operation it is simulating is prioritized. If it is not, the operation returns ?, and thus the process escapes the unbounded loop. Note that whenever n virtual processes perform concurrent get(-) operations on (distinct ports of) an n-P&G object, either the operation on port 1 or the operation on port 2 is not prioritized: it can thus return ? without getting the agreed upon proposal value. So the crash of a real process that prevents this agreed upon proposal value from being determined, can block at most n − 1 virtual processes. It is worth noting, however, that the pri field of the n-P&G object, which is critical to the escape clause, makes the linearization of the operations of an n-P&G object significantly more complex than the linearization of the operations of an n-consensus object.

3.7 Conclusion

Since the early days of the shared memory model for distributed computing, researchers have sought a simple way to classify objects in a precise manner, i.e., such that two objects are in the same class if and only if each can be implemented by instances of the other (and registers) in a wait-free manner. It is known that the classification of objects given by Herlihy’s wait-free consensus hierarchy [24] is not precise [42], even when it is restricted to deterministic objects [2]. In earlier work, not included in this thesis as it is largely superceded by the result presented in this chapter, we proved that the classification of linearizable objects based on their set agreement power is not precise. In particular, we introduced a deterministic oblivious object called n-PAC, and proved that for all n > 2, there is a nondeterministic object that has the same set agreement power as the n-PAC object but is not equivalent to it. In this thesis, we focused on the subset of linearizable objects that are deterministic, and we proved that their classification by set agreement power is also not precise. It remains open whether the set agreement power classification is precise for the subset of deterministic objects that are oblivious. Chapter 4

The Linearizability of Set Agreement

+ + In this chapter we show that for all n ∈ Z∞ and k ∈ Z∞, there is a linearizable object An,k that is equivalent to the (n, k)-set agreement problem. This is not obvious because the An,k object is not behaviourally equivalent to the (n, k)-set agreement problem. Indeed, any linearizable object for the (n, k)-set agreement problem imposes restrictions that are not inherent to this problem [41, 10]. To see this, suppose that all the proposal values are distinct, and two processes propose concurrently. With the (n, k)-set agreement problem, each of these two processes could decide the proposal value of the other (if k ≥ 2). But a linearizable (n, k)-set agreement object does not allow this behaviour: whichever of the two processes is linearized first cannot decide the proposal value of the other process. Since the set of behaviours allowed by linearizable (n, k)-set agreement objects is a proper subset of those allowed by the corresponding problem, it is conceivable that such objects inherently have greater computational power than the problem. With that in mind, Neiger [41] introduced a generalization of linearizability called set-linearizability. Intuitively, a set-linearizable object allows sets of operations to take effect at the same time. The state transition relation of a set-linearizable object specifies the result of a set of operations taking effect at the same time, and notably, it may differ from the result of the same set of operations taking effect sequentially. Thus a set-linearizable (n, k)-set agreement object can be defined such that two processes apply operations that are linearized simultaneously, and each process decides the proposal value of the other process. However, Castaneda et al. [10] observed that even a set-linearizable object for the (n, k)-set agreement problem imposes restrictions that are not inherent to this problem. To see this, consider the following execution of a solution to the (n, k)-set agreement problem where n ≥ 3 and k ≥ 2:

1. Process 1 begins with proposal value 1.

2. Process 2 then begins with proposal value 2.

3. Process 1 then sees that 2 has been proposed, and decides 2.

4. Process 3 then begins with proposal value 3.

5. Process 2 then sees that 3 has been proposed, and decides 3.

69 Chapter 4. The Linearizability of Set Agreement 70

6. Process 3 then decides 3.

This behaviour is allowed by the set agreement problem, but would not be allowed by a set-linearizable object for the set agreement problem. This is because process 2, which decides 3, cannot be linearized before process 3 begins, and process 1, which decides 2, cannot be linearized before process 2. Thus process 1 must be linearized after process 3 begins, contradicting the fact that process 1 decides before process 3 begins. So as before, since the set of behaviours allowed by set-linearizable (n, k)-set agreement objects is a proper subset of those allowed by the corresponding problem, it is conceivable that such objects inherently have greater computational power than the problem. With that in mind, Castaneda et al. [10] introduced a further generalization of linearizability called interval-linearizability. Intuitively, operations on an interval-linearizable object may take effect over an interval of time called its linearization interval instead of taking effect instantaneously at a linearization point. Finally, Castaneda et al. [10] proved that the set of behaviours allowed by interval-linearizable (n, k)-set agreement objects is not a proper subset of those allowed by the corresponding problem. Surprisingly, despite the difficulty of defining objects that are behaviourally equivalent to set agreement problems, our result about the An,k objects shows that computational equivalence can be achieved with linearizable objects.

The organization of this chapter is as follows. In Section 4.1, we introduce the An,k object for all + + + + n ∈ Z∞ and k ∈ Z∞. In Section 4.2, we show that for all n ∈ Z∞ and k ∈ Z∞, the An,k object is equivalent to the (n, k)-set agreement problem. In Section 4.3, we extend our results to show that for any set C of set agreement problems, there exists a linearizable object O that is equivalent to C.

4.1 The An,k Object

+ For all n, k ∈ Z∞, we now deﬁne the An,k object, a simple linearizable object that we will prove is equivalent to the (n, k)-set agreement problem in the following sense: the (n, k)-set agreement problem can be solved using the An,k object, and the An,k object can be implemented using any solution to the (n, k)-set agreement problem (and registers).

The behaviour of the An,k object when it is accessed sequentially is given by its sequential speciﬁca- tion, described below. The behaviour of An,k when it is accessed concurrently (in a well-formed manner) is linearizable [30].

The sequential speciﬁcation of An,k is given by Algorithm 10. An,k has n ports: each process can apply a propose(v) operation for any value v to any port i ∈ [1..n]. The state of the An,k object consists of:

• The set Vin of all values proposed to An,k; Vin is initially empty.

• The set Vout of all values returned by An,k; Vout is initially empty.

Algorithm 10 Sequential speciﬁcation of the An,k object. 1: procedure propose(v) applied to port i ∈ [1..n] 2: Vin ← Vin ∪ {v} 0 0 3: v ← arbitrary value in Vin such that |Vout ∪ {v }| ≤ k. 0 4: Vout ← Vout ∪ {v } 5: return v0 Chapter 4. The Linearizability of Set Agreement 71

The sequential speciﬁcation of An,k can be formally given in terms of a set of states, a set of operations, a set of responses, and a state transition relation. For brevity, we omit this formal deﬁnition here.

From the above deﬁnition of An,k, we have:

+ Observation 4.1. For all n, k ∈ Z∞, if the An,k object is accessed in a well-formed manner, it satisﬁes: • k-agreement: There are at most k distinct return values.

• Validity: If an operation op returns a value v, then v was proposed by op or by an operation linearized before op.

4.2 The Equivalence of the An,k Object and the (n, k)-Set Agree- ment Problem

By Observation 4.1, it is easy to see that:

+ Observation 4.2. For all n, k ∈ Z∞, the (n, k)-SA problem can be solved using an An,k object. We now show that the converse also holds:

+ Theorem 4.3. For all n, k ∈ Z∞, the linearizable An,k object can be implemented using registers and any solution to the (n, k)-SA problem.

Algorithm 11 Implementing the linearizable object An,k using any algorithm P[1..n] that solves the (n, k)-SA problem. 1: procedure propose(v) applied to port i ∈ [1..n] 2: (b, r) ← R[i].read() 3: if b = true then 4: return r 5: V [i].write(v) 6: j ← P[i].execute(i) 7: X[j].write(1) 8: if X[i].read() = 1 then 9: R[i].write((true, v)) 10: return v 11: else 12: u ← V [j].read() 13: R[i].write((true, u)) 14: return u

Proof. Algorithm 11 is a linearizable implementation of an An,k object that uses:

•P [1..n]: any algorithm that solves the (n, k)-SA problem, where for all i ∈ [1..n], P[i] is the protocol of process i.

• V [1..n]: an array of registers, initially all nil. For all i ∈ [1..n], V [i] stores the proposal value of the ﬁrst operation performed on port i.

• X[1..n]: an array of registers, initially all 0. For all i ∈ [1..n], X[i] stores 1 if for some i0 ∈ [1..n], P[i0] has decided i. Chapter 4. The Linearizability of Set Agreement 72

• R[1..n]: an array of registers, initially all (false, nil). For all i ∈ [1..n], R[i] stores a tuple (b, r), where b is a Boolean indicating whether any operation has been performed on port i, and r is the return value of the ﬁrst such operation (R[i] is used to make all operations on port i return the same value).

To perform an operation propose(v) on port i, a process executes the following steps:

1. It reads R[i] (line 2) to check whether there are previous operation(s) on port i (line 3). If so, it returns the same value as the ﬁrst operation on port i (line 4).

2. It writes its proposal value v to V [i] (line 5), letting the other processes know that v is the proposal value of the ﬁrst operation on port i.

3. It executes protocol P[i] with proposal value i ∈ [1..n], and receives a decision value j (line 6).

4. It writes 1 to X[j] (line 7), letting the other processes know that j is the decision value of some protocol of P[1..n].

5. It reads X[i] (line 8) to check whether i was decided by any protocol of P[1..n]. If so, it writes (true, v) to R[i] (line 9) and returns v (line 10); otherwise it reads a value u from V [j] (line 12), writes (true, u) to R[i] (line 13) and returns u (line 14).

Let H be any history of Algorithm 11 where accesses to ports are well-formed. Hence, the notion of “the ﬁrst operation applied on port i” is unambiguous. Note that, for every i ∈ [1..n], protocol P[i] is only executed by the ﬁrst operation on port i, and thus it is executed at most once. Also note that in line 6 of Algorithm 11, j is set to an integer in [1..n]. This is because all calls to the given solution P[1..n] of the (n, k)-SA problem are made with inputs i ∈ [1..n], and so the decision values of P[1..n] are also integers in [1..n].

We now prove that H is linearizable with respect to the sequential speciﬁcation of the An,k object. We ﬁrst establish some facts about H.

Claim 4.3.1. For each i ∈ [1..n], the ﬁrst operation on port i in H does not return on line 4.

Proof. In Algorithm 11, R[i] is written only by operations on port i (lines 9 and 13). Thus when the ﬁrst operation on port i in H reads R[i] (line 2), R[i] still contains its initial value (false, nil), and the operation does not return on line 4. Claim 4.3.1

Claim 4.3.2. For each i ∈ [1..n], operations on port i in H other than the ﬁrst one do not execute any of lines 5 to 14.

Proof. Consider an operation op on port i ∈ [1..n] other than the ﬁrst one. By Claim 4.3.1, the ﬁrst operation on port i in H does not return on line 4. Thus it returns on either line 10 or line 14. Immediately before doing so, it writes (true, −) to R[i] (line 9 or line 13). In Algorithm 11, (false, −) is never written into R[i]. Thus when op subsequently reads R[i] (line 2), R[i] still contains (true, −). So op evaluates the conditional on line 3 as true and returns on line 4, without executing any of lines 5 to 14. Claim 4.3.2

Claim 4.3.3. For each i ∈ [1..n], all complete operations on port i in H return the same value. Chapter 4. The Linearizability of Set Agreement 73

Proof. If there is at most one complete operation on port i in H, we are done. So suppose there are at least two complete operations on port i in H. By Claim 4.3.1, the ﬁrst operation on port i in H does not return on line 4. Thus it returns on either line 10 or line 14. Immediately before doing so, it writes (true, r) to R[i] (line 9 or line 13), where r is the value that it returns. In Algorithm 11, R[i] is written only by operations on port i, and only when they execute either line 9 or line 13. Furthermore, by Claim 4.3.2, all subsequent operations on port i in H do not execute either line 9 or line 13, and so do not write to R[i]. Thus all subsequent complete operations on port i in H will read (true, r) from R[i] (line 2) and return r on line 4 without ever writing to R[i]. Consequently, all complete operations on port i in H return the same value r. Claim 4.3.3

Claim 4.3.4. For all j ∈ [1..n], every read of V [j] (line 12) returns the proposal value of the ﬁrst operation on port j in H.

Proof. Let j be an integer in [1..n]. In Algorithm 11, an operation op reads V [j] (line 12) only after it completes executing a protocol of P[1..n] with decision value j (line 6). Since P[1..n] is a solution for the (n, k)-SA problem, there exists an operation op0 that begins executing a protocol of P[1..n] with proposal value j before or at the same time as op completes executing a protocol of P[1..n] with decision value j. In Algorithm 11, only operations on port j can execute a protocol of P[1..n] with proposal value j (line 6). Moreover, by Claim 4.3.2, operations on port j other than the first do not execute any protocol of P[1..n] (line 6). Thus op0 is the first operation on port j in H. Furthermore, op0 writes its own proposal value to V [j] (line 5) before executing any protocol of P[1..n] (line 6), and so before op reads V [j]. In Algorithm 11, V [j] is only written when an operation on port j executes line 5, and by Claim 4.3.2, only op0 does so. Thus when op reads V [j], V [j] still contains 0 the proposal value of op , the first operation on port j in H. Claim 4.3.4

Claim 4.3.5. If an operation in H returns a value v, then there is an integer j ∈ [1..n] such that: (a) some protocol of P[1..n] decides j, and (b) v is the proposal value of the ﬁrst operation on port j in H.

Proof. Let op be an operation in H that returns a value v. Let i ∈ [1..n] the port on which op is performed. There are three cases depending on where op returns (line 10, 14, or 4).

Case 1. op returns on line 10. Thus v is the proposal value of op, and when op reads X[i] on line 8, X[i] contains 1. Since X[i] is initially 0, there exists an operation op0 (possibly op) that previously writes 1 to X[i] (line 7). Before doing so, this operation op0 must have completed an execution of a protocol of P[1..n] with decision value i (line 6). Thus letting j = i, we have (a) some protocol of P[1..n] decides j, and (b) v is the proposal value of op, which executed line 10, and therefore (by Claim 4.3.2) is the ﬁrst operation on port j in H.

Case 2. op returns on line 14. Thus op returns the value that it read from V [j] (line 12), where j ∈ [1..n] is the decision value of the protocol P[i] that op executed on line 6. By Claim 4.3.4, op returns the proposal value of the Chapter 4. The Linearizability of Set Agreement 74

ﬁrst operation on port j in H. Thus we have (a) some protocol of P[1..n], namely P[i], decides j, and (b) v is the proposal value of the ﬁrst operation on port j in H.

Case 3. op returns on line 4. Then by Claim 4.3.1, op is not the ﬁrst operation on port i in H. So let op0 be the ﬁrst operation on port i in H. By Claim 4.3.3, op0 also returns v, and by Claim 4.3.1, op0 returns on either line 10 or line 14. Thus from Cases 1 and 2, there is an integer j ∈ [1..n] such that: (a) some protocol of

P[1..n] decides j, and (b) v is the proposal value of the ﬁrst operation on port j in H. Claim 4.3.5

Claim 4.3.6. In H, there are at most k distinct return values.

Proof. By Claim 4.3.5, if an operation in H returns a value v, then there is an integer j ∈ [1..n] such that: (a) some protocol of P[1..n] decides j, and (b) v is the proposal value of the ﬁrst operation on port j in H. Since P[1..n] solves the (n, k)-SA problem, it decides at most k distinct values. Thus there are at most k distinct return values in H. Claim 4.3.6

We now construct a completion H0 of H as follows: for each port i ∈ [1..n] with an incomplete operation op, if op is the ﬁrst operation on port i and some complete operation op0 in H returns the value proposed by op, then complete op immediately after it is invoked by returning its own proposal value (i.e., the same value that op0 returns); otherwise remove op. Next, we construct a linearization L of H0 as follows:

1. Linearize every operation that returns its own proposal value at the point it is invoked.

2. Linearize every operation that returns on line 4 at the point it is invoked.

3. Linearize every remaining operation at the point when it reads a register of X[1..n] (line 8).

From the construction of the completion H0 of H, all operations in H0 that do not return their own proposal values are complete in H. Thus all operations in H0 that do not return their own proposal values and do not return on line 4 read a register of X[1..n] (line 8). So in this linearization L, every operation in H0 is linearized at some point during its execution interval.

Claim 4.3.7. In H0, if an operation op returns a value v, then v was proposed by op or by an operation linearized before op in the linearization L of H0.

Proof. Suppose an operation op on port i proposes v0 and returns v. If v0 = v, the claim holds. Now suppose v0 6= v. Recall that when constructing the completion H0 of H, incomplete operations are only completed by returning their own proposal values. Thus op is a complete operation in H. There are two cases:

Case 1: op is the ﬁrst operation on port i. Then, by Claim 4.3.1, op does not return on line 4. Furthermore, since op proposes v0 and returns v 6= v0, op does not return on line 10. Thus op returns on line 14. So from the code of Algorithm 11 and the way we linearize operations, it is clear that:

(1) At some time t1, op completed executing protocol P[i] of P[1..n] with some decision value j ∈ [1..n] on line 6. Chapter 4. The Linearizability of Set Agreement 75

(2) At some time t2 > t1, op wrote 1 into X[j] on line 7.

(3) At some time t3 > t2, op read a non-1 value from X[i] on line 8. Note that op is linearized at 0 this time t3 in the linearization L of H .

(4) At some time t4 > t3, op read a value u from V [j] on line 12. Note that op eventually returns this value u on line 14, so u = v.

Since P[1..n] solves the (n, k)-SA problem, some operation op0 in H begins executing a protocol

of P[1..n] with proposal value j before or at the same time t1 when op completes executing P[i] with decision value j. In Algorithm 11, only operations on port j can execute a protocol of P[1..n] with proposal value j (line 6). Moreover, by Claim 4.3.2, operations on port j that are not the first cannot execute any protocol of P[1..n] (line 6). Thus op0 is the first operation on port j in H. Then by Claim 4.3.4, since op reads u = v from V [j], v is the proposal value of op0, the first operation on port j in H. Thus the return value v of op is proposed by op0. We now show that op0 is linearized before op in the linearization L of H0, and therefore v was proposed by an operation linearized before op. There are three subcases:

Case 1(a): op0 is incomplete in H. First, recall that when constructing the completion H0 of H, an incomplete operation in H is completed with its own proposal value if it is the ﬁrst operation on its port, and its proposal value is decided by some complete operation in H. Then recall that op0 is the ﬁrst operation on port j in H, and a complete operation in H, namely op, returns the value v that is proposed by op0. So op0 is completed in H0 by returning v immediately after it is invoked. Then, since op0 returns its own proposal value, op0 is linearized in L at the moment it is invoked in H0. Clearly, op0 is invoked before op0 begins executing a protocol of P[1..n] with proposal value 0 j, and so before op completes executing P[i] with decision value j at time t1. Thus op is

linearized before the linearization time t3 > t1 of op. Case 1(b): op0 is complete in H and it returns the value v that it proposed. Then op0 is linearized in L at the moment op0 is invoked in H0. Clearly, op0 is invoked before op0 begins executing a protocol of P[1..n] with proposal value j, and so before op completes 0 executing P[i] with decision value j at time t1. Thus op is linearized before the linearization

time t3 > t1 of op. Case 1(c): op0 is complete in H and it returns a value diﬀerent from the value v that it proposed. Then op0 is linearized in L at the point when it reads X[j] (line 8). Note that X[j] does not contain 1 at this linearization time of op0, otherwise op0 would return its own proposal value 0 v on line 10. Thus op is linearized before the time t2 when op writes 1 into X[j]; otherwise, since 1 is the only value ever written into X[1..n] in Algorithm 11, X[j] would still contain 1 0 0 at the linearization time of op . Then, since t2 < t3 and t3 is the linearization time of op, op is linearized before op in the linearization L of H0.

Case 2: op is not the first operation on port i. Then op is a complete operation in H, since in the construction of the completion H0 of H, every incomplete operation that is not the first on its port is removed. By Claim 4.3.3, the first Chapter 4. The Linearizability of Set Agreement 76

operation op0 on port i also returns the same value v. Thus from Case 1, v was proposed by op0 or by an operation linearized before op0. Since H is well-formed, op0 is linearized before op, so v

was proposed by an operation linearized before op. Claim 4.3.7

0 Let HL be the sequential history obtained by ordering all the operations in the complete history H 0 0 by their linearization points in L. Since the completion of H to H , and the linearization of H to HL does not introduce new return values, from Claim 4.3.6 we have:

Observation 4.3.8. (k-agreement) In HL, there are at most k distinct return values.

By the deﬁnition of HL and Claim 4.3.7, we have:

Observation 4.3.9. (Validity) In HL, if an operation op returns a value v, then v was proposed by op or by an operation before op.

To prove that Algorithm 11 implements the linearizable object An,k, it suffices to show that HL satisfies the sequential specification of An,k.

Suppose, for contradiction, that HL violates the sequential specification of An,k, and let op be the 0 first operation in HL that does so. Let v be the value proposed by op, and v be the value returned by 0 0 op. According to the sequential specification of An,k, v should be such that (i) v is in Vin ∪ {v}, and 0 0 (ii) |Vout ∪ {v }| ≤ k. Thus, since op violates the sequential specification of An,k, either (i) v is not in 0 Vin ∪ {v}, or (ii) |Vout ∪ {v }| > k. We consider these two cases below:

0 Case 1: v is not in Vin ∪ {v}.

By the sequential speciﬁcation of An,k, Vin ∪ {v} is the set of all values proposed by op and the 0 operations before op in HL. Thus op returns a value v that was neither proposed by op nor by an

operation that is before op in HL, contrary to Observation 4.3.9.

0 Case 2: |Vout ∪ {v }| > k.

0 By the sequential speciﬁcation of An,k, Vout ∪ {v } is the set of all values returned by op and

the operations before op in HL. Thus the operations in HL return more than k distinct values,

contrary to Observation 4.3.8. Theorem 4.3

+ Theorem 4.4. For all n, k ∈ Z , the linearizable object An,k is equivalent to the (n, k)-SA problem.

Proof. Part (a) follows from Observation 4.2 and Part (b) from Theorem 4.3. Theorem 4.4

4.3 The Linearizability of Sets of Set Agreement

Theorem 4.5. Given any set C of set agreement problems, there exists a linearizable object O that is equivalent to C.

+ + Proof. Let CA be the set of An,k objects where for all n ∈ Z∞ and k ∈ Z∞, CA contains the An,k object if and only if C contains the (n, k)-SA problem. Thus by Theorem 4.4,

Observation 4.5.1. CA and C are equivalent. Chapter 4. The Linearizability of Set Agreement 77

Next, note that since set agreement problems are deﬁned by two parameters n and k that are both + in the countable set Z∞, there are only countably many distinct set agreement problems. Thus the problems in C (and hence the objects in CA) can be enumerated. For all x ∈ [1..|CA|], let CA[x] denote the x-th object in CA, in some enumeration of CA, and let nx and kx be such that CA[x] is the Anx,kx object.

We now deﬁne an object OC that combines all of objects in CA as follows: for each x ∈ [1..|CA|], each port i ∈ [1..nx] of OC supports propose(v, x) operations, which behave identically to propose(v) operations applied on port i of CA[x] (other ports support propose(v, x) operations which simply return their own proposal values). The state of the OC object is a tuple (Vin[1..|CA|],Vout[1..|CA|]), where for all x ∈ [1..|CA|], Vin[x] is the set of values proposed by propose(−, x) operations, and Vout[x] is the set of values returned by propose(−, x) operations. Initially, for all x ∈ [1..|CA|], Vin[x] = {} and Vout[x] = {}.

The sequential behaviour of the OC object is given in Figure 12.

Figure 12 Sequential speciﬁcation of the OC object.

1: procedure propose(v, x) applied to port i ∈ [1..∞) { x ∈ [1..|CA|] } 2: if i > nx then return v 3: Vin[x] ← Vin[x] ∪ {v} 0 0 4: v ← arbitrary value in Vin[x] such that |Vout[x] ∪ {v }| ≤ kx 0 5: Vout[x] ← Vout ∪ {v } 6: return v0

From the above, it is clear that the sequential speciﬁcation of the OC object can be given in terms of a set of states, a set of operations, a set of responses, and a state transition relation. The OC object is linearizable, i.e., the propose(−, −) operation appears to be atomic [30]. Thus, we consider only sequential histories of this object.

Claim 4.5.2. The OC object is equivalent to CA in the sense that:

(a) OC can be used to implement every object in CA.

(b) OC can be implemented from the objects in CA.

Algorithm 13 Using OC to implement any object CA[x] in CA.

1: procedure propose(v) applied to port i ∈ [1..nx] 2: return OC.propose(v, x) on port i

Algorithm 14 Implementing OC using the objects in CA.

1: procedure propose(v, x) applied to port i ∈ [1..∞) { x ∈ [1..|CA|] } 2: if i > nx then return v 3: return CA[x].propose(v) on port i

Proof. (a): Algorithm 13 shows how OC can be used to implement any object CA[x] in CA. To perform an operation propose(v) on a port i of a CA[x] object, a process simply performs a propose(v, x) operation on port i of OC and returns the return value. The correctness of this trivial implementation is straightforward. Chapter 4. The Linearizability of Set Agreement 78

(b): Algorithm 14 shows how the objects in CA can be used to implement the OC object. To perform an operation propose(v, x) on a port i of OC, a process simply returns v if i > nx; otherwise it simply performs a propose(v) operation on port i of CA[x] and returns the return value. The correctness of this direct implementation is also straightforward. Claim 4.5.2

Claim 4.5.2 and Observation 4.5.1 immediately imply Theorem 4.5. Theorem 4.5 Chapter 5

On the Robustness of Object Classiﬁcations

We say that an object O is equivalent to its set agreement power if O is equivalent to the set of set S∞ agreement problems k=1{(nk, k)-set agreement}, where (n1, n2, . . . , nk,...) is the set agreement power of O. Thus observe that an object O is equivalent to its set agreement power if and only if O is equivalent to the set of all set agreement problems that can be solved using O and registers. Let USA be the set of every object that is equivalent to its set agreement power. The organization of this chapter is as follows. In Section 5.1, we describe an important result of Delporte et al. [15] that will be used in subsequent proofs. In Section 5.2, we prove that Herlihy’s [24] consensus hierarchy (and other set agreement classiﬁcations) is robust when restricted to contain only objects in USA. In Section 5.3, we use the previous results to show an alternative proof that the set agreement power classiﬁcation is imprecise, albeit for nondeterministic objects.

5.1 The Extended Set Agreement Partial Order Theorem

Let C be any set of set agreement problems. Note that C is enumerable, since there are only countably many distinct set agreement problems. So for all x ∈ [1..|C|], let C[x] = (nx, kx)-set agreement be the + + x-th set agreement problem in C, where nx ∈ Z∞ and kx ∈ Z∞.

Then given any integers b, a1..a|C| ∈ [0..∞), it is straightforward to show that arbitrary solutions to |C| |C| all of the problems in C can solve the (m, `)-SA problem where m ≤ b+Σx=1axnx and ` ≥ b+Σx=1axkx. To do so, it suffices to partition the m processes of the (m, `)-SA problem as follows: first create a group of at most b processes, then for every x ∈ [1..|C|], if ax > 0, create ax groups of at most nx processes each. In the first group of b processes, each process simply decides its own proposal value. For every x ∈ [1..|C|], each of the ax groups of at most nx processes independently uses the arbitrary solution to the (nx, kx)-SA problem of C to decide at most kx of its nx processes’ proposal values. In this way, the |C| m processes return at most b + Σx=1axkx distinct values, each proposed by some process. By extending a result of Chaudhuri and Reiners [14], Delporte et al. [15] proved that the ability to partition the m processes in such a manner is also a necessary condition to solve the (m, `)-SA problem. The following theorem follows directly from Theorem 2 of [15]:

79 Chapter 5. On the Robustness of Object Classifications 80

Theorem 5.1 (Extended Set Agreement Partial Order Theorem). Let C be any set of set agreement problems, and for all x ∈ [1..|C|], let C[x] = (nx, kx)-set agreement be the x-th set agreement problem in + + + + C, where nx ∈ Z∞ and kx ∈ Z∞. Then for all m ∈ Z∞ and ` ∈ Z∞, registers and arbitrary solutions to all of the problems in C can solve the (m, `)-SA problem if and only if there exists a non-negative integer b and a sequence of non-negative integers a1..a|C| such that:

|C| • b + Σx=1axnx ≥ m.

|C| • b + Σx=1axkx ≤ `.

5.2 The Robustness of Objects Equivalent to Their Set Agree- ment Power

Consider the set USA of all objects that are equivalent to their set agreement power. If we restrict

Herlihy’s consensus hierarchy [24] to USA, we can prove the resulting hierarchy is robust [34] in the + following sense: in USA, for all n ∈ Z , any set of objects with consensus number at most n cannot be used to implement any object with consensus number n0 > n. In fact, we can prove a more general result, as we now describe. To describe our robustness result, we first define what it means for a sequence to dominate another. 0 0 0 0 + Given any pair of sequences ~v = (v1, v2, . . . , v`) and ~v = (v1, v2, . . . , v`) of the same length ` ∈ Z∞, we 0 0 0 say that v dominates v , denoted ~v ≥ ~v , if for all k ∈ [1..`], vk ≥ vk; similarly, we say that v strictly dominates, denoted ~v> ~v0, if ~v ≥ ~v0 and ~v 6= ~v0. + Then for any ` ∈ Z∞, we can partition the set USA into equivalence classes such two objects are in the same class if and only if the first ` components of their set agreement powers are the same; we call this an `-partition of USA, and denote it P` (note that the 1-partition of USA is simply Herlihy’s consensus hierarchy restricted to the objects in USA). Let C be any equivalence class of P`. By definition, the first

` components of the set agreement power of every object in C is some sequence ~v = (v1, v2, . . . , v`); this 0 0 sequence is the label of C. If C and C are equivalence classes of P` with labels ~v and ~v respectively, we say that C dominates C0 if ~v ≥ ~v0, and C strictly dominates C0 if ~v> ~v0. Our generalized robustness result can now be stated as follows:

+ Theorem 5.2. Given any ` ∈ Z∞, consider the `-partition P` of USA, and let C be any equivalence 0 class of P`. If O is a set of objects that belong to equivalence classes of P` dominated by C and O is an object that belongs to an equivalence class that strictly dominates C, then O0 cannot be implemented from instances of objects in O and registers.

0 0 Proof. Let C and C be any pair of equivalence classes of P` such that C strictly dominates C. Then let O be any set of objects in USA that are in equivalence classes of P` that are dominated by C. Finally, let O0 be an object in C0. We must show that O0 cannot be implemented using instances of objects in O and registers. 0 0 0 0 0 0 Let ~v = (v1, v2, . . . , v`) and ~v = (v1, v2, . . . , v`) be the labels of C and C , respectively. Since C 0 0 0 0 strictly dominates C, ~v > ~v, and so there is an integer d ∈ [1..`] such that vd > vd. Since O is in C , and 0 0 0 0 0 0 0 the label of C is ~v = (v1, v2, . . . , v`), instances of O and registers can solve the (vk, k)-set agreement 0 0 problem for each k ∈ [1..`]. In particular, instances of O and registers can solve the (vd, d)-set agreement problem. Chapter 5. On the Robustness of Object Classifications 81

S∞ For all k > `, let vk = ∞. Then consider the set Cv = k=1{(vk, k)-set agreement}.

Claim 5.2.1. Every object O ∈ O can be implemented using registers and arbitrary solutions to all of the problems in Cv.

Proof. Let O be an object in O, and let (n1, n2, . . . , nk,...) be the set agreement power of O. Since O is in O, O is in an equivalence class that is dominated by C. Thus for all k ∈ [1..`], nk ≤ vk. Furthermore, for all k > `, vk = ∞. So for all k ≥ 1, nk ≤ vk.

Since O is in USA, O is equivalent to its set agreement power, i.e., O is equivalent to the set C = S∞ k=1{(nk, k)-set agreement}. Then for all k ≥ 1, since nk ≤ vk, an arbitrary solution to the (vk, k)-set agreement problem in Cv also solves the (nk, k)-set agreement problem in C. So all of the problems in C can be solved using registers and arbitrary solutions to all of the problems in Cv. Since O is equivalent to C, O can also be implemented using registers and arbitrary solutions to all of the problems in Cv. Claim 5.2.1

Suppose, for contradiction, that O0 can be implemented using instances of objects in O and regis- 0 0 ters. Then, since the (vd, d)-set agreement problem can be solved using instances of O and registers, 0 the (vd, d)-set agreement problem can be solved using instances of objects in O and registers. So by 0 Claim 5.2.1, the (vd, d)-set agreement problem can be solved using registers and arbitrary solutions to all of the problems in Cv. Thus by Theorem 5.1, there exists a non-negative integer b and a sequence of non-negative integers

(a1, a2,...) such that:

∞ X 0 b + akvk ≥ vd k=1 ∞ X b + akk ≤ d k=1

P∞ Note that for all k > d, ak = 0, otherwise b + k=1 akk > d. Thus we have:

d X 0 b + akvk ≥ vd k=1 d X b + akk ≤ d k=1

∗ ∗ ∗ ∗ ∗ Let O be an arbitrary object in the equivalence class C, and ~n = (n1, n2, . . . , nk,...) be the set ∗ ∗ agreement power of O . Since O is in C and the label of C is ~v = (v1, v2, . . . , v`), for all k ∈ [1..`], ∗ nk = vk. Thus, since d ∈ [1..`], we have:

d X ∗ 0 b + aknk ≥ vd k=1 d X b + akk ≤ d k=1 Chapter 5. On the Robustness of Object Classifications 82

Since for all k > d, ak = 0, we have:

∞ X ∗ 0 b + aknk ≥ vd k=1 ∞ X b + akk ≤ d k=1

∗ S∞ ∗ 0 Let C be the set k=1{(n , k)-set agreement}. By the above equations and Theorem 5.1, the (vd, d)- set agreement problem can be solved using registers and arbitrary solutions to all of the problems in C∗. ∗ ∗ ∗ ∗ ∗ ∗ Since O is in USA, O is equivalent to its set agreement power. Thus, since ~n = (n1, n2, . . . , nk,...) ∗ ∗ S∞ ∗ is the set agreement power of O , O is equivalent to the set C = k=1{(n , k)-set agreement}. So the 0 ∗ (vd, d)-set agreement problem can also be solved using registers instances of O and registers. Thus the ∗ 0 ∗ d-set agreement number of O is at least vd > vd. However, recall that for all k ∈ [1..`], nk = vk, so in ∗ ∗ particular nd = vd. Therefore the d-set agreement number of O is vd — a contradiction. Theorem 5.2

Recall that Herlihy’s consensus hierarchy [24], when restricted to objects in USA, is simply the 1- partition P1 of USA. Thus by Theorem 5.2,

Corollary 5.3. Herlihy’s consensus hierarchy [24], is robust when restricted to the objects that are equivalent to their set agreement power.

5.3 An Alternate Proof of the Imprecision of the Set Agreement Power Classiﬁcation

In Chapter 3, we presented a detailed and complex proof that the set agreement power classification is imprecise. We now show that the An,k objects defined earlier allow for a significantly simpler proof of the same result by using a result of Lo and Hadzilacos [38], whose proof is itself fairly complex. However, unlike the proof given in Chapter 3, the proof given in this section does not hold when the universe of objects is restricted to contain only deterministic objects.

Lemma 5.4. There exists an object that is not equivalent to its set agreement power.

Proof. Suppose, for contradiction, that every object is equivalent to its set agreement power. Then by Corollary 5.3, Herlihy’s consensus hierarchy is robust — contradicting the result of Lo and Hadzila- cos [38].

Theorem 5.5. There are two (nondeterministic) objects O and O0 such that O and O0 have the same set agreement power, but O is not equivalent to O0.

Proof. By Lemma 5.4, there exists an object O that is not equivalent to its set agreement power. By definition, this implies that O is not equivalent to any set of set agreement problems. Let C be the set of all set agreement problems that can be solved using instances of O and registers. Since C is a set of set agreement problems, O is not equivalent to C. In contrast, by Theorem 4.5, there exists an object O0 that is equivalent to C. So O and O0 are not equivalent. Thus it suffices to show that O and O0 have the same set agreement power. For this it suffices to show that for any set agreement Chapter 5. On the Robustness of Object Classifications 83 problem T , T can be solved using instances of O and registers if and only if T can be solved using instances of O0 and registers: [If] T can be solved using instances of O0 and registers. Then, since O0 is equivalent to C, T can also be solved using registers and arbitrary solutions to all of the problems in C. Since C is the set of all set agreement problems that can be solved using instances of O and registers, T can also be solved using instances of O and registers. [Only If] T can be solved using instances of O and registers. Then T is in the set C of all set agreement problems that can be solved using instances of O and registers. Since O0 is equivalent to C, T can also 0 be solved using instances of O and registers. Theorem 5.5 Chapter 6

The Uncountability of the Precise Classiﬁcation

This chapter shows that the precise classiﬁcation of shared memory objects has uncountably many cells.

Let Sˆ denote the set of all inﬁnite sequences of positive integers ~n = (n1, n2, . . ., nk,...) such that nk+1 ≥ 2nk for all k ≥ 1. We use a result in [15] to prove that:

ˆ For all ~n ∈ S, there is a (linearizable) object O~n with set agreement power ~n. (6.1)

Note that this is not obvious because it is not the case that every inﬁnite sequence of positive integers

~n = (n1, n2, . . ., nk,...) has a corresponding object LR~n with set agreement power ~n. Finally, we use a standard diagonalization argument to prove that the set Sˆ is uncountable. Therefore, by (6.1), there is an uncountable number of (linearizable) objects, namely the LR~n objects, that have distinct set agreement power. Since objects with diﬀerent set agreement power are not equivalent, we conclude that there are uncountably many (linearizable) objects that are not equivalent (and so they have distinct computational power).

ˆ + nk+1 nk Lemma 6.1. For all ~n ∈ S and all k ∈ Z , k+1 ≥ k .

ˆ + + 2 1 Proof. By deﬁnition, for all ~n ∈ S and k ∈ Z , nk+1 ≥ 2nk. Furthermore, since k ∈ Z , k+1 ≥ k . nk+1 2nk 2 1 nk Consequently, k+1 ≥ k+1 = ( k+1 )nk ≥ ( k )nk = k . Lemma 6.1

By induction we have:

ˆ 0 + 0 nk nk0 Corollary 6.2. For all ~n ∈ S and all k, k ∈ Z where k ≥ k , k ≥ k0 .

ˆ + Observation 6.3. For all ~n ∈ S and all k ∈ Z , nk ≥ k. ˆ S∞ For each ~n ∈ S, we denote by C~n the set k=1{(nk, k)-set agreement problem}.

ˆ + Lemma 6.4. For all ~n ∈ S and all k ∈ Z , the (nk + 1, k)-set agreement problem cannot be solved using registers and arbitrary solutions to all of the problems in C~n.

ˆ + Proof. Let ~n = (n1, n2, . . ., nk,...) be an inﬁnite sequence in S and let k ∈ Z . Then let b be an arbitrary non-negative integer, and let ~a = (a1, a2,...) be an arbitrary inﬁnite sequence of non-negative integers.

84 Chapter 6. The Uncountability of the Precise Classification 85

We deﬁne the predicate P(~a,b, ~n,k) to be true if and only if the following inequalities are true:

∞ X b + a`n` ≥ nk + 1 (6.2) `=1 ∞ X b + a`` ≤ k (6.3) `=1

By Theorem 5.1, it suﬃces to show that P(~a,b, ~n,k) is false.

0 Case 1. There exists an integer k > k such that ak0 > 0.

P∞ 0 0 Then b + `=1 a`` ≥ ak0 k ≥ k > k. Thus inequality (6.3) is false, and so P(~a,b, ~n,k) is false.

0 + 0 Case 2. For all k ∈ Z such that k > k, we have ak0 = 0.

Pk Case 2(a). b + `=1 a` = 0 Since ~a is a sequence of non-negative integers and b is a non-negative integer, this implies b = 0 P∞ and for all 1 ≤ ` ≤ k, a` = 0. Then b + `=1 a`n` = 0 < nk + 1. Thus the inequality (6.2) is false, and so P(~a,b, ~n,k) is false.

Pk Case 2(b). b + `=1 a` > 0.

Thus either a` > 0 for some 1 ≤ ` ≤ k, or b > 0. We deﬁne the function:

k k X X f(~a,b, ~n,k) = k b + a`n` − (nk + 1) b + a`` `=1 `=1

f(~a,b, ~n,k) is the left side of (6.2) multiplied by the right side of (6.3), minus the left side of (6.3) multiplied by the right side of (6.2). Thus if P(~a,b, ~n,k) is true, then f(~a,b, ~n,k) ≥ 0.

By algebra,

k k X X f(~a,b, ~n,k) = k b + a`n` − (nk + 1) b + a`` `=1 `=1 k k X X = bk − b(nk + 1) + k a`n` − (nk + 1) a`` `=1 `=1 k k X X = b(k − (nk + 1)) + ka`n` − (nk + 1)a`` `=1 `=1 k X = b(k − (nk + 1)) + (ka`n` − (nk + 1)a``) `=1 k X = b(k − (nk + 1)) + a`(kn` − (nk + 1)`) `=1

By Observation 6.3, k − (nk + 1) < 0. Thus if b > 0, then b(k − (nk + 1)) < 0, whereas if b = 0, Chapter 6. The Uncountability of the Precise Classification 86

then b(k − (nk + 1)) = 0. By Corollary 6.2, for all 1 ≤ ` ≤ k, n n k ≥ ` k `

⇒ nk` ≥ kn`

⇒ kn` − nk` ≤ 0

⇒ kn` − (nk + 1)` < 0

Thus for all ` ∈ [1..k], if a` > 0, then a`(kn` − (nk + 1)`) < 0, whereas if a` = 0, then a`(kn` − Pk (nk + 1)`) = 0. Ergo, every term of the sum b(k − (nk + 1)) + `=1 a`(kn` − (nk + 1)`) is either 0 Pk or negative. Furthermore, since b + `=1 a` > 0, either a` > 0 for some 1 ≤ ` ≤ k, or b > 0, so at least one of the terms is negative. Therefore, f(~a,b, ~n,k) < 0, and so P(~a,b, ~n,k) is false.

So, for all ~a ∈ A and b ∈ N, P(~a,b, ~n,k) is false. Thus the (nk + 1, k)-set agreement problem cannot be solved using registers and arbitrary solutions to all of the problems in C~n. Lemma 6.4 ˆ By Theorem 4.5, for each ~n ∈ S, there exists a linearizable object O~n that is equivalent to the set C~n of set agreement problems. By Lemma 6.4,

ˆ + Corollary 6.5. For all ~n ∈ S and all k ∈ Z , the (nk + 1, k)-set agreement problem cannot be solved using registers and O~n objects. S∞ Then since O~n is equivalent to the set C~n = k=1{(nk, k)-set agreement problem}, by Corollary 6.5: ˆ Corollary 6.6. For all ~n ∈ S, the linearizable object O~n has set agreement power ~n.

We now prove:

Lemma 6.7. Sˆ is uncountable.

Proof. We prove this via a standard diagonalization argument. Assume, for contradiction, that Sˆ is countable. In other words, there exists some enumeration E of all elements in Sˆ. For all i ∈ Z+ and j ∈ Z+, let E[i, j] denote the j-th number in the i-th sequence of the enumeration E. Now consider the inﬁnite sequence ~n = (n1, n2, . . ., nk,...) where:

• n1 = E[1, 1] + 1.

• For all i ≥ 2, ni = E[i, i] + 2ni−1.

+ Since E is an enumeration of infinite sequences of positive integers, ni is a positive integer for all i ∈ Z . Furthermore, by construction, ni is at least twice as large as ni−1 for all i ≥ 2. Consequently, ~n is an infinite sequence of positive integers where each integer is at least twice as large as its predecessor, so ˆ + ~n ∈ S. By construction, however, ni 6= E[i, i] for all i ∈ Z , so ~n is not any of the infinite sequences in the enumeration E of Sˆ. This is a contradiction, so we conclude that Sˆ is uncountable. Lemma 6.7

Theorem 6.8. There are uncountably many linearizable objects that are not equivalent to each other.

Proof. By Lemma 6.7, Sˆ is uncountable. By Corollary 6.6, for all ~n ∈ Sˆ, there is a linearizable object with set agreement power ~n. So there are uncountably many linearizable objects with distinct set agreement power. By deﬁnition, linearizable objects with diﬀerent set agreement power are not equivalent. The theorem immediately follows. Theorem 6.8 Chapter 6. The Uncountability of the Precise Classification 87

6.1 Concluding remark

In this chapter, we showed that there are uncountably many linearizable (nondeterministic) objects that are not equivalent to each other (Theorem 6.8), and thus the precise classiﬁcation of shared memory objects has uncountably many classes. It remains an open question whether the precise classiﬁcation of deterministic shared memory objects has uncountably many classes. Chapter 7

Bounded Disagreement

This chapter introduces bounded disagreement and compares it to consensus and set agreement. Recall that in the n-consensus problem, there are n processes, each process has an input value (its proposal) and must output a value (its decision) such that the following properties are satisﬁed:

• Termination: Every correct process eventually outputs a decision value.

• Validity: Each decision is one of the proposals.

• Agreement: All decisions are identical.

In the (n, k)-set agreement problem, the Agreement property is replaced by:

• k-SA: There exist at most k distinct decision values.

Whereas in the (n, `)-bounded disagreement problem, the Agreement property is replaced by:

• `-BD: There is a value v such that at most ` processes (the disagreers) decide a value other than v.

The organization of this chapter is as follows. In Section 7.1, we determine values of `, m, k, and n such that the (m, `)-BD problem can solve the (n, k)-SA problem. In Section 7.2, we deﬁne BD objects that can be used to solve BD problems, and, in Section 7.3, we investigate what instances of consensus can not be solved using these BD objects. Together, these results allow us to determine the consensus number of every instance of the BD problem and BD object. In Section 7.4, we prove that for all n ≥ 2, there are BD problems and BD objects that have consensus number n, but cannot be implemented using n-consensus and registers. In Section 6.1, we conclude with a brief discussion on bounded disagreement and related problems. All objects considered within this chapter are non-ported objects. (Recall that, as shown in Chapter 2, non-ported objects can be viewed as special cases of ported objects.)

7.1 Using Bounded Disagreement Problems to Solve Set Agree- ment Problems

In this section, we show that for all k ≥ 1, n ≥ 1, ` ≥ 0 and m ≥ max(n + ` + b`/kc , 2` + 2), the (n, k)- set agreement problem can be solved using any solution to the (m, `)-bounded disagreement problem

88 Chapter 7. Bounded Disagreement 89 and registers. For k = 1 and n = m − 2`, this says that the (m − 2`, 1)-set agreement problem, i.e., the (m − 2`)-consensus problem, can be solved using any solution to the (m, `)-bounded disagreement problem and registers. We prove this result in two steps: we ﬁrst assume that processes have access to a Fetch & Increment object, and then we remove this assumption. Intuitively, a Fetch & Increment object F is a simple counter with initial value 1, that has a single operation fetch&increment(). Each operation on F returns the current value of the counter and then increments it by one [5]. Multisets notation. It is convenient to use a multiset to store the outputs of a BD problem. Given any multiset M, the mode of M, denoted mode(M), is the value that appears the most often in M (ties are broken arbitrarily), and |M| is the total number of (non-necessarily distinct) elements in M. The + operator denotes the disjoint union of two multisets. This is distinct from the normal union operator ∪ in that the elements are added regardless of whether they are distinct from other elements. More precisely, the multiplicity of an element x in A + B is the sum of the multiplicities of x in A and in B.

Theorem 7.1. For all k ≥ 1, n ≥ 1, ` ≥ 0 and m ≥ n + ` + b`/kc, the (n, k)-set agreement problem can be solved using any solution to the (m, `)-bounded disagreement problem, a Fetch & Increment object, and registers.

Proof. Consider any k ≥ 1, n ≥ 1, ` ≥ 0 and m ≥ n + ` + b`/kc. We present an algorithm that solves the (n, k)-SA problem using:

•P [1..m], a solution for the (m, `)-BD problem, where P[i] denotes the protocol of process i in the solution of (m, `)-BD.

•X , an n-process snapshot object where each ﬁeld contains a multiset of integers; initially empty. (Recall that snapshot objects can be implemented using registers [1].)

•F , a Fetch & Increment object; initialized to 1.

Recall that an n-process snapshot object X is composed of n fields, and supports two operations: write(v) and snapshot(). A write operation, denoted by X [p].write(v), replaces the value of field X [p] by v. A snapshot operation, denoted by X .snapshot(), returns the values in all n fields atomically. In addition to these shared objects, each process p also uses the following local variables:

• received: a multiset used to store all outputs that p has received from executing protocols in P[1..m]; initially empty.

• snap: a multiset used to store a snapshot of X ; initially empty.

• next: a positive integer.

We denote by varp the local variable var of process p, and by vp the input value of p. Intuitively, n processes solve the (n, k)-set agreement problem as follows: each process executes diﬀerent protocols of the solution P[1..m] to the (m, `)-bounded disagreement problem, using its input to the (n, k)-set agreement problem. To ensure that each protocol of P[1..m] is executed at most once, each process uses the Fetch & Increment object F and executes P[i] if and only if it receives i from F. Each process executes as many protocols of P[1..m] as it can, recording in the snapshot object X the Chapter 7. Bounded Disagreement 90

Algorithm 15 Solving the (n, k)-SA problem using an (m, `)-BD problem solution P[1..m], an atomic snapshot X , and a Fetch & Increment object F. 1: while TRUE do 2: next ← F.fetch&increment() 3: if next ≤ m then 4: received ← received + {P[next].execute(vp)} 5: X [p].write(received) 6: else 7: snap ← X .snapshot() 8: decide mode(snap) 9: halt multiset of outputs it has received from the protocols it has executed so far. When there are no more protocols of P[1..m] to execute (i.e., the process receives a value greater than m from F), each process takes a snapshot of X to obtain the multiset of outputs of P[1..m] and decides a mode of that multiset. The pseudocode for each process p is shown in Algorithm 15. We use the notation P[i].execute(v) to denote the output produced by the execution of P[i] with input v. In a useful abuse of notation, we use X to denote the snapshot object and the multiset formed by the disjoint union of the multisets in the n ﬁelds of X .

Claim 7.1.1. Each protocol of P[1..m] is executed at most once.

Proof. This follows immediately from the fact that each protocol P[i], for all 1 ≤ i ≤ m, is executed only when the Fetch & Increment object F gives the response i to some process. Claim 7.1.1

Thus the outputs of P[1..m] must satisfy the requirements of the (m, `)-BD problem. Let Output(P) be the multiset of all outputs from P[1..m] so far, and let Outputp(P) be the multiset of all outputs from P[1..m] given to process p so far. To begin the analysis of this algorithm, ﬁrst observe that the following invariant holds:

Invariant 7.1.2. For each process p,

(a) X [p] ⊆ Outputp(P), and hence X ⊆ Output(P).

(b) X [p], and hence X , is monotonically increasing.

Claim 7.1.3. (Termination) Every correct process eventually decides exactly one value.

Proof. Let p be a correct process. Since nextp is derived from repeated operations on the Fetch & Increment object F, it will eventually become greater than m. Then the conditional on line 3 will evaluate to false, causing p to decide exactly one value and halt (line 8). Claim 7.1.3

Claim 7.1.4. (Validity) Every decision v was proposed by some process, i.e., v = vq for some process q.

Proof. Suppose a process p decides a value v. Clearly, v ∈ X . By Invariant 7.1.2(a), X is a subset of Output(P), so v is an output from P[1..m]. Since P[1..m] is a solution for the (m, `)-BD problem, by the Validity property of this problem, v must be one of the input values for P[1..m]. The claim follows from the fact that each process q only uses its own proposal value vq as the input value for P[1..m].

Claim 7.1.4 Chapter 7. Bounded Disagreement 91

Claim 7.1.5. At any time, if X contains h ≥ ` elements, the mode of X appears at least h − ` times in Output(P).

Proof. Since P[1..m] is a solution for the (m, `)-BD problem, by the `-BD property of this problem, there is a value v such that at most ` of the outputs in Output(P) are not v. By Invariant 7.1.2(a), X is a subset of Output(P). Consequently, X contains at most ` outputs that are not v, and the rest of the outputs must equal v. In other words, if X contains h ≥ ` elements, then v appears at least h − ` times in X . Thus the mode of X appears at least h − ` times in X . By Invariant 7.1.2(a), the mode of

X also appears at least h − ` times in Output(P). Claim 7.1.5

Claim 7.1.6. If a process decides a value v, then v appears at least b`/kc + 1 times in Output(P).

Proof. Suppose a process p decides a value v (on line 8). Then let S be the multiset in X at the time tS when p executes the snapshot() operation on X in line 7; so v is the mode of S. By the conditional on line 3, tS is after p gets a value greater than m from F. Since F is initialized to 1, at least m + 1 operations were performed on F before tS. Note that each time a process receives a response from F that is at most m, it cannot perform another operation on F before executing a protocol of P[1..m] and adding the output of that execution to the multiset stored by X . Thus, since the n processes perform a total of at least m + 1 operations on F before tS, at least m − n + 1 outputs were added to the multiset stored by X before tS. Thus S contains at least m − n + 1 outputs. Now, recall that m ≥ n+`+b`/kc, so m−n+1 ≥ `+b`/kc+1. Thus S contains at least `+b`/kc+1 outputs. By Claim 7.1.5, the mode v of S appears at least b`/kc + 1 times in Output(P). Claim 7.1.6

Claim 7.1.7. (k-SA) There are at most k distinct decision values.

Proof. Assume, for contradiction, that there are at least k + 1 distinct decision values. By Claim 7.1.6, each decision value appears at least b`/kc + 1 times in Output(P). In other words, Output(P) contains at least k + 1 distinct output values, each appearing at least b`/kc + 1 times. This implies that for every value v, at least k(b`/kc + 1) > ` of the outputs given by P[1..m] are not v. This violates the `-BD property of the (m, `)-BD problem, contradicting the fact that P[1..m] solves the (m, `)-BD problem. Claim 7.1.7

Thus the Termination (Claim 7.1.3), Validity (Claim 7.1.4), and k-SA (Claim 7.1.7) properties of the (n, k)-SA problem all hold. So Algorithm 15 solves the (n, k)-SA problem. Theorem 7.1

Theorem 7.2. For all ` ≥ 0 and m ≥ 2` + 2, the (m, `)-BD problem and registers can solve the 2- consensus problem.

Proof. Consider any ` ≥ 0 and m ≥ 2` + 2. We present a modiﬁcation of Algorithm 15 that solves the 2-consensus problem without using a Fetch & Increment object. In addition to a solution P[1..m] for the (m, `)-BD problem, the new algorithm uses only a (shared) 2-process snapshot object X , and local variables received and snap as described in Algorithm 15. Recall that the Fetch & Increment object F used in Algorithm 15 serves two roles:

• It ensures each protocol of P[1..m] is executed at most once.

• It tells processes when to take a snapshot of X and decide the mode. Chapter 7. Bounded Disagreement 92

Algorithm 16 Solving consensus for 2 processes using an (m, `)-BD problem solution P[1..m] and an atomic snapshot X 1: while TRUE do 2: snap ← X .snapshot() 3: if |snap| ≤ m − 2 then 4: if p = 1 then 5: received ← received + {P[|received| + 1].execute(vp)} 6: else 7: received ← received + {P[m − |received|].execute(vp)} 8: X [p].write(received) 9: else 10: decide mode(snap) 11: halt

Roughly speaking, processes can decide as soon as suﬃciently many outputs have been written into X . Thus, the second role can be replaced simply by having processes take a snapshot of X and counting the current number of outputs. For the ﬁrst role, we can ensure that each protocol of P[1..m] is executed at most once by using the fact that there are only two processes involved in the 2-consensus problem. Intuitively, this is achieved by starting the two processes at opposite ends of the range [1..m], and having them sequentially execute the protocols of P[1..m] until they meet each other.

As before, let vp denote p’s input value. Let the two processes be numbered 1 and 2. We now describe the algorithm that each process p executes to solve the 2-consensus problem:

1. It takes a snapshot S of X .

2. If S contains more than m − 2 outputs, it decides the mode.

3. If p is process 1, it executes protocol P[i1 + 1] with input value vp, where i1 is the number of protocols process 1 has executed.

4. If p is process 2, it executes protocol P[m − i2] with input value vp, where i2 is the number of protocols process 2 has executed.

5. It adds the output to X [p].

6. It repeats from the ﬁrst step.

The pseudocode for each process p in shown in Algorithm 16.

Claim 7.2.1. Each protocol of P[1..m] is executed at most once.

Proof. Assume for contradiction, that for some 1 ≤ i ≤ m, protocol P[i] is executed more than once. Observe that in Algorithm 16, this can only happen if both processes execute protocol P[i], since each process sequentially traverses the range [1..m]. This means both processes took a snapshot of X and found at most m − 2 outputs right before executing protocol P[i]. However, in order for both processes to reach protocol P[i], they must ﬁrst execute all other protocols in P[1..m] and write their outputs into X . Hence one of the two processes must have found more than m − 2 outputs in its snapshot, a contradiction. Claim 7.2.1. Chapter 7. Bounded Disagreement 93

Since each protocol of P[1..m] is executed at most once, their outputs must satisfy the requirements of the (m, `)-BD problem. As before, let Output(P) be the multiset of all outputs from P[1..m] so far, and let Outputp(P) be the multiset of all outputs from P[1..m] given to process p so far. First, note that Invariant 7.1.2 still holds for this modiﬁed algorithm. Furthermore, Claim 7.1.4 and Claim 7.1.5 also hold here: their proofs are exactly as before. The remaining claims also have similar proofs:

Claim 7.2.2. (Termination) Every correct process eventually decides exactly one value.

Proof. Let p be a correct process. Observe that each time the conditional on line 3 evaluates to true, process p will execute a protocol of P[1..m], and write its output to X , increasing the number of outputs in X by one. By Invariant 7.1.2(b), the number of outputs in X is monotonically increasing. Thus it will eventually become greater than m − 2. Then the conditional on line 3 will evaluate to false, causing p to decide exactly one value and halt (lines 10-11). Claim 7.2.2

Claim 7.2.3. If a process decides a value v, then v appears at least ` + 1 times in Output(P).

Proof. Suppose a process p decides a value v (on line 10). Then let S be the multiset in X at time tS when p executes the snapshot() operation on X in line 2; so v is the mode of S. By the conditional on line 3, S contains at least m−1 outputs. Since m ≥ 2`+2, m−1 ≥ 2`+1. Thus S contains at least 2`+1 outputs. By Claim 7.1.5, the mode v of S appears at least ` + 1 times in Output(P). Claim 7.2.3

Claim 7.2.4. (Agreement) All decisions have the same value.

Proof. Assume, for contradiction, that there are at least two distinct decision values. By Claim 7.2.3, each decision value appears at least ` + 1 times in Output(P). Thus Output(P) contains at least two distinct output values, each appearing at least ` + 1 times. This implies that for every value v, at least `+1 > ` of the outputs given by P[1..m] are not v. This violates the `-BD property of the (m, `)-BD problem, contradicting the fact that P[1..m] solves the (m, `)-BD problem. Claim 7.2.4

Thus the Termination (Claim 7.2.2), Validity (Claim 7.1.4), and Agreement (Claim 7.2.4) properties of the 2-consensus problem all hold. So Algorithm 16 solves the 2-consensus problem. Theorem 7.2

We can now prove the main result of this section:

Theorem 7.3. For all k ≥ 1, n ≥ 1, ` ≥ 0 and m ≥ max(n + ` + b`/kc , 2` + 2), the (n, k)-set agreement problem can be solved using any solution to the (m, `)-bounded disagreement problem and registers.

Proof. In previous work, Afek et al. [5] proved that (any solution to) the 2-consensus problem, together with registers, can be used to implement a Fetch & Increment object for any number of processes. Thus from Theorem 7.2, for all ` ≥ 0 and m ≥ 2` + 2, the (m, `)-BD problem and registers can implement a Fetch & Increment object for any number of processes. The result now follows from

Theorem 7.1. Theorem 7.3

Corollary 7.4. For all ` ≥ 0 and m > 2`, the (m − 2`)-consensus problem can be solved using any solution to the (m, `)-bounded disagreement problem and registers.

Proof. Let ` ≥ 0. For m = 2` + 1, the (m − 2`)-consensus problem is the trivial 1-consensus problem. For m ≥ 2` + 2, by setting k = 1 and n = m − 2` in Theorem 7.3, the (m − 2`, 1)-set agreement problem, i.e., the (m − 2`)-consensus problem, can be solved using any solution to the (m, `)-bounded disagreement problem and registers. Corollary 7.4 Chapter 7. Bounded Disagreement 94

7.2 Bounded Disagreement Objects

+ In this section, we define the linearizable oblivious (m, `)-BD objects for all m, ` ∈ Z∞. Intuitively, with an (m, `)-BD object, processes can propose any value to the object, and the object responds to the first m such proposals with values that satisfy the Validity and `-BD properties; the (m + 1)-th proposal permanently upsets the (m, `)-BD object, causing it to nondeterministically respond with an arbitrary value to this proposal and all the subsequent ones. Thus, as long as the BD object is not upset, every response is one of the proposals (Validity) and satisfies the following property:

• `-BD: There exists a value v such that at most ` responses are not v.

Note that m processes can trivially use a single (m, `)-BD object to solve the (m, `)-BD problem: each process just proposes its input value to the object and decides the object’s response. However, since the (m, `)-BD object gets upset if more than m proposal operations are applied, it cannot be used in this trivial way by m0 > m processes to solve the (m0, `)-BD problem. We now deﬁne the (m, `)-BD object more precisely. The sequential speciﬁcation of the (m, `)-BD object is given by Algorithm 17. The (m, `)-BD object is a non-ported object that supports propose(v) operations for any integer v ∈ Z. The state of an (m, `)-BD object D is either a tuple (SOP,MRES), where SOP is the set of all the values proposed to D and MRES is the multiset of all responses given by D, or the upset state ⊥. If the state of D is a pair (SOP,MRES), we denote by |MRES| the size of MRES; note that |MRES| is also the number of operations performed on D so far. We say that the state (SOP,MRES) is full if |MRES| = m, since the next operation performed on D will cause D to permanently enter the upset state ⊥. Note that the state of D does not record information about the order of operations and responses.

Algorithm 17 Sequential speciﬁcation of the (m, `)-BD object. 1: procedure propose(v) 2: if state = (SOP,MRES) such that |MRES| = m then 3: state ← ⊥ 4: if state = ⊥ then 5: return arbitrary value

6: SOP ← SOP ∪ {v} 0 7: v ← arbitrary value in SOP such that there exists a value u such that at most ` elements 0 in MRES + {v } are not u. 0 8: MRES ← MRES + {v } 9: return v0

The sequential specification of the (m, `)-BD object can be formally given in terms of a set of states, a set of operations, a set of responses, and a state transition relation. For brevity, we omit this formal definition here. From the above definition of the (m, `)-BD object, we have:

Observation 7.5. The response to the ﬁrst operation performed on a BD object is the operation’s own proposal value.

Observation 7.6. A BD object that is in either a full state or the upset state ⊥ can nondeterministically respond with any value to all future operations.

Furthermore, the validity and `-BD properties hold as long as the object is not upset: Chapter 7. Bounded Disagreement 95

Observation 7.7. If at most m operations are applied to an (m, `)-BD object D, then each response given by D is a value proposed to D, and there is a value v such that at most ` of the responses given by D are not v.

This immediately implies:

Observation 7.8. The (m, `)-BD problem can be solved with a single (m, `)-BD object.

From the above observation, it is clear that Theorem 7.3 and Corollary 7.4 also apply for BD objects.

7.3 Unsolvability of (m − 2` + 1)-Consensus by (m, `)-BD Objects

We now show that (m, `)-BD objects and registers cannot solve the (m − 2` + 1)-consensus problem. This, together with Corollary 7.4, allows us to determine the consensus number of every BD problem and BD object.

Theorem 7.9. For all ` ≥ 0 and m > 2`, (m, `)-BD objects and registers cannot solve the (m − 2` + 1)- consensus problem.

Proof. Consider any ` ≥ 0 and m > 2`. For the special case of ` = 0, observe that the (m, `)-BD object is equivalent to the m-consensus object.1 Thus the theorem immediately follows from the fact that m-consensus objects and registers cannot solve the (m + 1)-consensus problem. Hence it suffices to consider the case where ` ≥ 1. For ` ≥ 1, we prove a stronger result, namely, that (m, `)-BD objects and registers cannot solve the binary2 (m − 2` + 1)-consensus problem, even when the (m, `)-BD objects are further strengthened by restricting their nondeterministic behavior as follows: As long as it is not upset, each (m, `)-BD object outputs at most two distinct response values. Assume, for contradiction, that for some ` ≥ 1 and m > 2`, there is a wait-free algorithm that solves binary consensus among m − 2` + 1 ≥ 2 processes using only registers and the strengthened (m, `)-BD objects. It suffices to prove the existence of an infinite execution of this algorithm where processes never decide. The proof uses the bivalency technique introduced by Fischer et al. [17]. We assume the reader is familiar with bivalency proof terminology such as configuration, bivalent, and 0-valent [17, 24]. In the following, we omit the proofs of Claims 7.9.1 to 7.9.4 since they are standard (see [24]).

Claim 7.9.1. The algorithm has an initial bivalent conﬁguration Cinit.

Claim 7.9.2. There is a bivalent conﬁguration Cbi, reachable from Cinit, such that if any process takes a step, the resulting conﬁguration is univalent.

Claim 7.9.3. At Cbi, every process is about to perform an operation on the same object D.

Claim 7.9.4. D is not a register.

Since the algorithm only uses registers and (m, `)-BD objects, we have the following:

Claim 7.9.5. D is an (m, `)-BD object. 1It is identical to a non-ported version of the m-consensus object, which is equivalent to the ported version of m- consensus deﬁned in Chapter 2; see Appendix A. 2In the binary consensus problem, every proposal value is restricted to being either 0 or 1. Chapter 7. Bounded Disagreement 96

Claim 7.9.6. There exists a pair of distinct processes p0 and p1, and a pair of (not necessarily distinct) values v0 and v1, such that starting from Cbi,

• p0 can take a step with response v0, and the resulting conﬁguration C0 is 0-valent.

• p1 can take a step with response v1, and the resulting conﬁguration C1 is 1-valent.

Proof. This follows immediately from Claim 7.9.2 since there are at least 2 processes. Claim 7.9.6

Given a conﬁguration C such that D is not upset in C, we denote by D.MRES(C) the multiset MRES in the state of D in C.

Claim 7.9.7. D is not upset in Cbi and |D.MRES(Cbi)| < 2`.

Proof. Assume, for contradiction, that either D is upset in Cbi or |D.MRES(Cbi)| ≥ 2`. By Claim 7.9.3, every process was about to apply an operation on D at Cbi, and so at both C0 and C1, every process other than p0 and p1 will perform an operation on D as its next step. Thus consider the following conﬁgurations:

0 • A 0-valent conﬁguration C0 reached from C0 by letting every process other than p0 and p1 (if any)

take a step in an arbitrary order (while p0 and p1 take no steps).

0 • A 1-valent conﬁguration C1 reached from C1 by letting every process other than p0 and p1 (if any)

take a step in an arbitrary order (while p0 and p1 take no steps).

There are two cases:

0 0 1. D is upset in Cbi. Clearly, D is also upset in both C0 and C1.

0 0 2. |D.MRES(Cbi)| ≥ 2`. Since there are m − 2` + 1 processes, by Claim 7.9.3, in both C0 and C1, the 0 number of operations performed on D is |D.MRES(Cbi)| + (m − 2` + 1) − 1 ≥ m. Thus in both C0 0 and C1, D is in either a full state or the upset state ⊥.

0 0 So in all cases, D is in either a full state or the upset state ⊥, in both C0 and C1. By Observation 7.6, D is henceforth allowed to nondeterministically return any response to all subsequent operations. Thus consider the following conﬁgurations:

00 0 • The 0-valent conﬁguration C0 reached from C0 by letting p1 take a step with response v1.

00 0 • The 1-valent conﬁguration C1 reached from C1 by letting p0 take a step with response v0.

00 00 00 Since p0 received v0 as the response from D in both C0 and C1 , the state of p0 is the same in C0 as 00 00 00 in C1 . Furthermore, the state of every object is the same in C0 as in C1 : D is in the upset state ⊥, and all other objects have not changed their state since Cbi. Consequently, if all processes other than p0 crash, p0 00 00 will not be able to distinguish between the 0-valent configuration C0 and the 1-valent configuration C1 , 00 00 so if p0 runs solo starting from configurations C0 and C1 , it decides the same value in both runs — a contradiction. Claim 7.9.7

Claim 7.9.8. |D.MRES(Cbi)| ≥ 1. Chapter 7. Bounded Disagreement 97

Proof. Assume, for contradiction, that |D.MRES(Cbi)| = 0. In other words, D is in the initial state in Cbi. By Observation 7.5, v0 is the proposal value of the operation that p0 is about to invoke on D at conﬁguration Cbi, and similarly v1 is the proposal value of the operation that p1 is about to invoke on D at conﬁguration Cbi. We claim that:

0 • Starting from C0, p1 can take a step with response v1, and the resulting conﬁguration C0 is 0-valent.

0 • Starting from C1, p0 can take a step with response v0, and the resulting conﬁguration C1 is 1-valent.

0 0 To see why, recall that D is a strengthened (m, `)-BD object where ` ≥ 1. In both C0 and C1, the number of distinct response values from D is at most two, and the number of responses diﬀerent from v0 0 0 is at most one. Thus the above steps required to reach C0 and C1 are possible. Finally, recall that the state of D does not record information about the order of operations and responses: it records only the set of proposals and the multiset of responses so far. Consequently, the 0 0 state of D is the same in C0 as in C1. Furthermore, since p0 received v0 as the response from D in 0 0 0 0 0 both C0 and C1, the state of p0 is the same in C0 as in C1. Similarly, the state of p1 is the same in C0 0 as in C1. Finally, observe that all other objects and processes have not changed their state since Cbi. 0 0 0 0 0 Thus their states are also the same in C0 as in C1. Therefore C0 = C1, contradicting the fact that C0 0 is 0-valent and C1 is 1-valent. Claim 7.9.8

By Claim 7.9.7 and Claim 7.9.8, D is not upset in Cbi and 1 ≤ |D.MRES(Cbi)| < 2`. Let vmode denote the mode of D.MRES(Cbi) (ties can be decided arbitrarily).

Claim 7.9.9. At most ` − 1 of the responses in D.MRES(Cbi) are not vmode.

Proof. By Claim 7.9.7, D is not upset at Cbi and D.MRES(Cbi) contains at most 2`−1 responses. Recall that D is a strengthened (m, `)-BD object that outputs at most two distinct values as long as it is not upset. Thus D.MRES(Cbi) contains at most two distinct values. So the mode vmode of D.MRES(Cbi) appears at least d|D.MRES(Cbi)|/2e times. Thus, the number of responses in D.MRES(Cbi) that are not vmode is at most b|D.MRES(Cbi)|/2c ≤ b(2` − 1)/2c = ` − 1. Claim 7.9.9

∗ ∗ ∗ Claim 7.9.10. There exists a pair of distinct processes p0 and p1, and a value v1 (not necessarily distinct from vmode), such that starting from Cbi,

∗ ∗ • p0 can take a step with response vmode, and the resulting conﬁguration C0 is univalent.

∗ ∗ ∗ • p1 can take a step with response v1 , and the resulting conﬁguration C1 is univalent, with diﬀerent ∗ valence from C0 .

Proof. Consider the processes p0 and p1, the values v0 and v1, and the configurations C0 and C1 stated in Claim 7.9.6. If v0 = vmode or v1 = vmode, we are done. 0 Suppose v0 6= vmode and v1 6= vmode. Let C0 be the configuration reached when, starting from Cbi, p0 takes a step with response vmode (note that this step is possible, since the number of distinct response 0 values, and the number of responses different from vmode, do not increase). Similarly, let C1 be the 0 0 configuration reached when, starting from Cbi, p1 takes a step with response vmode. If C0 and C1 have ∗ ∗ ∗ different valence, we are done (with p0 = p0, p1 = p1, and v1 = vmode). 0 0 Suppose C0 and C1 have the same valence. Without loss of generality, suppose they are both 0- 0 valent. Observe that, starting from Cbi, the 0-valent configuration C0 is reached when p0 takes a step Chapter 7. Bounded Disagreement 98

with response vmode, and the 1-valent conﬁguration C1 is reached when p1 takes a step with response v1. ∗ ∗ ∗ Thus the claim holds with p0 = p0, p1 = p1, and v1 = v1. Claim 7.9.10

We claim that:

∗ ∗ ∗ ∗ • Starting from C0 , p1 can take a step with response v1 , and the resulting conﬁguration Cb0 is univalent.

∗ ∗ ∗ • Starting from C1 , p0 can take a step with response vmode, and the resulting conﬁguration Cb1 is ∗ univalent, with diﬀerent valence from Cb0 .

∗ ∗ To see why, note that in both Cb0 and Cb1 , the number of distinct response values from D is at most two, and from Claim 7.9.9, the number of responses diﬀerent from vmode is at most (` − 1) + 1 = `. Thus the ∗ ∗ above steps required to reach Cb0 and Cb1 are possible. Recall that the state of D does not record information about the order of operations and responses: it records only the set of proposals and the multiset of responses so far. Consequently, the state of D ∗ ∗ ∗ ∗ is the same in Cb0 as in Cb1 . Furthermore, since p0 received vmode as the response from D in both Cb0 ∗ ∗ ∗ ∗ ∗ ∗ ∗ and Cb1 , the state of p0 is the same in Cb0 as in Cb1 . Similarly, the state of p1 is the same in Cb0 as in Cb1 .

Finally, observe that all other objects and processes have not changed their state since Cbi. Thus their ∗ ∗ ∗ ∗ ∗ ∗ states are also the same in Cb0 as in Cb1 . Therefore Cb0 = Cb1 , contradicting the fact that Cb0 and Cb1 have opposite valence. Theorem 7.9

We can now determine the consensus number for every BD problem and object as follows:

Theorem 7.10. For all ` ≥ 0 and m > 0, both the (m, `)-BD problem and the (m, `)-BD object have consensus number max(m − 2`, 1).

Proof. Consider any ` ≥ 0 and m > 0. There are two cases: either m > 2`, or 0 ≤ m ≤ 2`. First, consider the case where m > 2`. Since BD objects are at least as strong as their corresponding BD problems (Observation 7.8), by Corollary 7.4 both the (m, `)-BD problem and the (m, `)-BD object can, together with registers, solve the (m − 2`)-consensus problem. Similarly, by Theorem 7.9 both the (m, `)-BD problem and the (m, `)-BD object, together with registers, cannot solve the (m − 2` + 1)-consensus problem. Thus both the (m, `)-BD problem and the (m, `)-BD object have consensus number m − 2` = max(m − 2`, 1). Now consider the case where m ≤ 2`. From the above case, we have that both the (2` + 1, `)-BD problem and the (2` + 1, `)-BD object have consensus number 1. For m ≤ 2`, it is clear that the (2` + 1, `)-BD object is at least as strong as the (m, `)-BD object, and the (2` + 1, `)-BD problem is at least as diﬃcult to solve as the (m, `)-BD problem. Thus both the (m, `)-BD problem and the (m, `)-BD object also have consensus number 1 = max(m − 2`, 1). Theorem 7.10

7.4 An Unusual Property of Bounded Disagreement

We now show that every level n ≥ 2 of Herlihy’s consensus hierarchy contains a BD object that cannot be implemented using n-consensus and registers.

Theorem 7.11. For all n ≥ 2, both the (9n, 4n)-BD problem and the (9n, 4n)-BD object have consensus number n, but n-consensus objects and registers can neither solve the (9n, 4n)-BD problem nor implement a (9n, 4n)-BD object. Chapter 7. Bounded Disagreement 99

Proof. Let n ≥ 2, and consider the (m, `)-BD object for m = 9n and ` = 4n.

1. Since ` ≥ 0 and m > 0, by Theorem 7.10, the (9n, 4n)-BD object has consensus number max(m − 2`, 1) = 9n − 2(4n) = n.

2. Consider the (n0, k)-SA problem for n0 = 3n and k = 2. Note that m = n0 +`+b`/kc (because 9n = 3n + 4n + b4n/2c), and m ≥ 2` + 2 (because for n ≥ 2, 9n ≥ 2(4n) + 2). Then, since k ≥ 1, n0 ≥ 1, ` ≥ 0, and m ≥ max(n0 + ` + b`/kc , 2` + 2), by Theorem 7.3 and the fact that the (9n, 4n)-BD object is at least as strong as the (9n, 4n)-BD problem (Observation 7.8), (9n, 4n)-BD objects and registers can solve the (3n, 2)-SA problem.

We claim that a (9n, 4n)-BD object cannot be implemented using n-consensus objects and registers. Suppose, for contradiction, that n-consensus objects and registers can implement a (9n, 4n)-BD object. Since (9n, 4n)-BD objects and registers can solve the (3n, 2)-SA problem, this implies that n-consensus objects and registers can solve the (3n, 2)-SA problem. This contradicts the following important result about set agreement: for all n ≥ 2, n-consensus objects and registers cannot solve the (3n, 2)-SA problem [14]. Thus, the (9n, 4n)-BD object has consensus number n, but n-consensus objects and registers cannot implement a (9n, 4n)-BD object. The proof that this also holds for the (9n, 4n)-BD problem is almost identical. Theorem 7.11

Prior to this thesis, the only known objects with consensus number n ≥ 2 and not implementable using n-consensus and registers were the ad hoc objects deﬁned by Rachman [42] and Afek et al. [2]. We now prove that these objects are fundamentally diﬀerent from our BD objects.

7.4.1 BD Objects are not Equivalent to Rachman’s Objects

Rachman [42] deﬁned the 2-set object with parameter m, which intuitively is a composite of the m- consensus problem and the (∞, 2)-set agreement problem. Rachman proved that this object has consensus number m but cannot be implemented by m-consensus objects and registers.

Theorem 7.12. For all m ≥ 1, the 2-set object with parameter m cannot be implemented by (n, `)-BD objects and registers for any n ≥ 1 and ` ≥ 0.

Proof. For all n ≥ 1, n-consensus objects and registers cannot solve the (∞, 2)-SA problem [14]. Since the n-consensus object can trivially implement the (n, `)-BD object for all ` ≥ 0, (n, `)-BD objects and registers cannot solve the (∞, 2)-SA problem. In contrast, by deﬁnition, the 2-set object with parameter m ≥ 1 solves the (∞, 2)-SA problem [42]. Theorem 7.12

7.4.2 BD Objects are not Equivalent to Afek et al.’s Objects

Afek et al. [2] deﬁned a family of deterministic objects Om,k, where m ≥ 2 and k ≥ 2, such that: (a) Om,k has consensus number m, (b) Om,k solves the (km + k − 1, k)-SA problem, and (c) Om,k+1 implements

Om,k. They also proved that Om,k objects and registers cannot solve the (km+m+k, k+1)-SA problem, which is solved by the Om,k+1 object since km + m + k = (k + 1)m + (k + 1) − 1. Thus Om,k does not implement Om,k+1, and so each level of the consensus hierarchy contains an inﬁnite hierarchy of strictly stronger deterministic objects. Chapter 7. Bounded Disagreement 100

Theorem 7.13. For all n ≥ 1, m ≥ 2, and k ≥ 2, the (9n, 4n)-BD object is not equivalent to the Om,k object.

Proof. Consider any n ≥ 1, m ≥ 2, and k ≥ 2. By Theorem 7.10, the (9n, 4n)-BD object has consensus number n. On the other hand, the Om,k object has consensus number m [2]. If m 6= n, it is clear that the two objects are not equivalent. So suppose that n = m. Case 1. k is odd. Thus there exists a positive integer q such that k + 1 = 2q. Observe that:

• Any solution to the (2q(m + 1), 2q)-SA problem can be used to solve the (km + m + k, k + 1)-SA problem; this is because km + m + k = (k + 1)(m + 1) − 1 = 2q(m + 1) − 1 < 2q(m + 1).

• Any solution to the (2(m + 1), 2)-SA problem can be used, together with registers, to solve the (2q(m + 1), 2q)-SA problem; the algorithm to do so is obvious: divide the 2q(m + 1) processes into q groups of 2(m + 1) processes, and have each group solve the (2(m + 1), 2)-SA problem independently [14].

• Since m ≥ 2, 2(m + 1) ≤ 3m, and so any solution to the (3m, 2)-SA problem can be used to solve the (2(m + 1), 2)-SA problem.

• Since m = n, the (3m, 2)-SA problem is the (3n, 2)-SA problem.

• By Theorem 7.3 and the fact that the (9n, 4n)-BD object is at least as strong as the (9n, 4n)-BD problem (Observation 7.8), (9n, 4n)-BD objects and registers can solve the (3n, 2)-SA problem.

Thus, by transitivity, we have that (9n, 4n)-BD objects and registers can solve the (km+m+k, k+1)-SA problem. In contrast, as we mentioned above, Om,k objects and registers cannot solve the (km + m + k, k + 1)-SA problem [2]. We conclude that the (9n, 4n)-BD object cannot be implemented using Om,k objects and registers. Case 2. k is even. Then k+1 is odd. From Case 1, the (9n, 4n)-BD object cannot be implemented using

Om,k+1 objects and registers. Since the Om,k+1 object implements the Om,k object [2], by transitivity, the (9n, 4n)-BD object cannot be implemented using Om,k objects and registers. Theorem 7.13

7.5 Conclusion

In this chapter, we introduced a novel and natural generalization of consensus that we call bounded disagreement, and investigated its relation to consensus and set agreement. Our results show that, despite apparent similarities, bounded disagreement is fundamentally diﬀerent from these two problems:

• BD and consensus are distinct problems. In fact, by Theorem 7.11, for all n ≥ 2, the (9n, 4n)-BD problem [object] have consensus number n, but cannot be solved [implemented] by n-consensus and registers. Thus for all k ≤ n, k-consensus and registers cannot solve the (9n, 4n)-BD problem, and for all k > n, the (9n, 4n)-BD problem and registers cannot solve k-consensus: so, for all n ≥ 2, there is no k such that k-consensus is equivalent to the (9n, 4n)-BD problem. Therefore there are inﬁnitely many instances of the bounded disagreement problem that are not equivalent to any consensus problem.

• BD and SA are distinct problems. Except for the special case of k = 1 where the (n, k)-SA problem is simply the n-consensus problem, the (n, k)-SA problem and registers cannot solve the 2-consensus Chapter 7. Bounded Disagreement 101

problem [14]. In contrast, by Theorem 7.10, the (4, 1)-BD problem has consensus number 2, and in fact, for all n ≥ 2, there are ` ≥ 1 and m such that the (m, `)-BD problem has consensus number n. So there are inﬁnitely many instances of the bounded disagreement problem that are not equivalent to any set agreement problem. Chapter 8

Conclusion

In this thesis, we first showed that the classification of shared memory objects based on their set agreement power is not precise, even for the subset of deterministic objects. It remains open whether the classification is precise for the subset of deterministic objects that are oblivious. + We then showed that for all n, k ∈ Z∞, there exists an object An,k that is equivalent to the (n, k)-set agreement problem. Using this result, we showed that every classification of objects based on a prefix of their set agreement power (including Herlihy’s consensus hierarchy [24]) is robust for the subset of objects that are equivalent to their set agreement power. Thus if the consensus hierarchy is not robust for a given subset U 0 of objects, then the set agreement power classification is not precise for U 0.

Also using the An,k objects, we showed that the precise classification of shared memory objects has uncountably many classes. It remains an open question whether the precise classification of deterministic shared memory objects has uncountably many classes. Finally, we introduced a novel and natural generalization of consensus that we call bounded disagreement, and showed that, despite apparent similarities, bounded disagreement is fundamentally different from these two problems. In fact, there are infinitely many instances of the bounded disagreement problem that are not equivalent to any consensus or set agreement problem. Consensus is one of the fundamental problems in distributed computing, and its generalization set agreement has produced deep and valuable insights about the field as a whole. The history of set agreement is one such example: since its introduction in 1990 [12], research on this problem has led to significant results, such as linking topology and distributed computing [26, 44]. It has also elucidated the relationship between consensus and other important problems in distributed computing, such as renaming and symmetry breaking [9, 20, 21]. The bounded disagreement problem may provide a similar impetus in the future.

102 Bibliography

[1] Yehuda Afek, Hagit Attiya, Danny Dolev, Eli Gafni, Michael Merritt, and Nir Shavit. Atomic snapshots of shared memory. J. ACM, 40(4):873–890, Sep 1993.

[2] Yehuda Afek, Faith Ellen, and Eli Gafni. Deterministic objects: Life beyond consensus. In Pro- ceedings of the 2016 ACM Symposium on Principles of Distributed Computing, PODC ’16, pages 97–106, New York, NY, USA, 2016. ACM.

[3] Yehuda Afek, Eli Gafni, and Adam Morrison. Common2 extended to stacks and unbounded concurrency. Distributed Computing, 20(4):239–252, 2007.

[4] Yehuda Afek, Adam Morrison, and Guy Wertheim. From bounded to unbounded concurrency objects and back. In Proceedings of the 30th Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, PODC ’11, pages 119–128, New York, NY, USA, 2011. ACM.

[5] Yehuda Afek, Eytan Weisberger, and Hanan Weisman. A completeness theorem for a class of synchronization objects. In Proceedings of the Twelfth Annual ACM Symposium on Principles of Distributed Computing, PODC ’93, pages 159–170, New York, NY, USA, 1993. ACM.

[6] James Aspnes, Hagit Attiya, and Keren Censor. Max registers, counters, and monotone circuits. In Proceedings of the 28th ACM Symposium on Principles of Distributed Computing, PODC ’09, pages 36–45, New York, NY, USA, 2009. ACM.

[7] Elizabeth Borowsky and Eli Gafni. Generalized FLP impossibility result for t-resilient asynchronous computations. In Proceedings of the Twenty-ﬁfth Annual ACM Symposium on Theory of Computing, STOC ’93, pages 91–100, New York, NY, USA, 1993. ACM.

[8] Elizabeth Borowsky, Eli Gafni, Nancy Lynch, and Sergio Rajsbaum. The BG distributed simulation algorithm. Distributed Computing, 14(3):127–146, Oct 2001.

[9] Armando Casta˜neda,Damien Imbs, Sergio Rajsbaum, and Michel Raynal. Renaming is weaker than set agreement but for perfect renaming: A map of sub-consensus tasks. In Proceedings of the 10th Latin American International Conference on Theoretical Informatics, LATIN’12, pages 145–156, Berlin, Heidelberg, 2012. Springer-Verlag.

[10] Armando Casta˜neda,Sergio Rajsbaum, and Michel Raynal. Specifying concurrent problems: Be- yond linearizability and up to tasks. In Yoram Moses, editor, 29th International Symposium on Distributed Computing (DISC 2015), pages 420–435, Berlin, Heidelberg, 2015. Springer Berlin Hei- delberg.

103 Bibliography 104

[11] David Yu Cheng Chan, Vassos Hadzilacos, and Sam Toueg. Life beyond set agreement. In Pro- ceedings of the 2017 ACM Symposium on Principles of Distributed Computing, PODC ’17, pages 345–354, New York, NY, USA, 2017. ACM.

[12] Soma Chaudhuri. Agreement is harder than consensus: Set consensus problems in totally asynchronous systems. In Proceedings of the Ninth Annual ACM Symposium on Principles of Distributed Computing, PODC ’90, pages 311–324, New York, NY, USA, 1990. ACM.

[13] Soma Chaudhuri. More choices allow more faults: Set consensus problems in totally asynchronous systems. Information and Computation, 105(1):132 – 158, 1993.

[14] Soma Chaudhuri and Paul Reiners. Understanding the set consensus partial order using the Borowsky-Gafni simulation (extended abstract). In Proceedings of the 10th International Workshop on Distributed Algorithms, WDAG ’96, pages 362–379, London, UK, UK, 1996. Springer-Verlag.

[15] Carole Delporte-Gallet, Hugues Fauconnier, Eli Gafni, and Petr Kuznetsov. Set-consensus collec- tions are decidable. In Proceedings of the 20th International Conference on Principles of Dis- tributed Systems, OPODIS 2016, pages 7:1–7:15, Dagstuhl, Germany, 2017. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik.

[16] E. W. Dijkstra. Solution of a problem in concurrent programming control. Communications of the ACM, 8(9):569–, Sep 1965.

[17] Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. Impossibility of distributed consensus with one faulty process. J. ACM, 32(2):374–382, Apr 1985.

[18] Eli Gafni. The extended BG-simulation and the characterization of t-resiliency. In Proceedings of the Forty-ﬁrst Annual ACM Symposium on Theory of Computing, STOC ’09, pages 85–92, New York, NY, USA, 2009. ACM.

[19] Eli Gafni and Petr Kuznetsov. N-consensus is the second strongest object for N + 1 processes. In Proceedings of the 11th International Conference on Principles of Distributed Systems, OPODIS’07, pages 260–273, Berlin, Heidelberg, 2007. Springer-Verlag.

[20] Eli Gafni, Sergio Rajsbaum, and Maurice Herlihy. Subconsensus tasks: Renaming is weaker than set agreement. In Shlomi Dolev, editor, Distributed Computing, volume 4167 of Lecture Notes in Computer Science, pages 329–338. Springer Berlin Heidelberg, 2006.

[21] Eli Gafni, Michel Raynal, and Corentin Travers. Test&set, adaptive renaming and set agreement: a guided visit to asynchronous computability. In Reliable Distributed Systems, 2007. SRDS 2007. 26th IEEE International Symposium on, pages 93–102, Oct 2007.

[22] Vassos Hadzilacos and Sam Toueg. On deterministic abortable objects. In Proceedings of the 2013 ACM Symposium on Principles of Distributed Computing, PODC ’13, pages 4–12, New York, NY, USA, 2013. ACM.

[23] Maurice Herlihy. Impossibility results for asynchronous pram (extended abstract). In Proceedings of the Third Annual ACM Symposium on Parallel Algorithms and Architectures, SPAA ’91, pages 327–336, New York, NY, USA, 1991. ACM. Bibliography 105

[24] Maurice Herlihy. Wait-free synchronization. ACM Trans. Program. Lang. Syst., 11(1):124–149, Jan 1991.

[25] Maurice Herlihy, Victor Luchangco, and Mark Moir. Obstruction-free synchronization: double- ended queues as an example. In Distributed Computing Systems, 2003. Proceedings. 23rd Interna- tional Conference on, pages 522–529, May 2003.

[26] Maurice Herlihy and Sergio Rajsbaum. Set consensus using arbitrary objects (preliminary version). In Proceedings of the Thirteenth Annual ACM Symposium on Principles of Distributed Computing, PODC ’94, pages 324–333, New York, NY, USA, 1994. ACM.

[27] Maurice Herlihy and Eric Ruppert. On the existence of booster types. In Proceedings of the 41st Annual Symposium on Foundations of Computer Science, FOCS ’00, pages 653–, Washington, DC, USA, 2000. IEEE Computer Society.

[28] Maurice Herlihy and Nir Shavit. The asynchronous computability theorem for t-resilient tasks. In Proceedings of the Twenty-ﬁfth Annual ACM Symposium on Theory of Computing, STOC ’93, pages 111–120, New York, NY, USA, 1993. ACM.

[29] Maurice Herlihy and Nir Shavit. The topological structure of asynchronous computability. J. ACM, 46(6):858–923, Nov 1999.

[30] Maurice Herlihy and Jeannette Wing. Linearizability: A correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst., 12(3):463–492, Jul 1990.

[31] Damien Imbs and Michel Raynal. Visiting Gafni’s reduction land: From the BG simulation to the extended BG simulation. In Proceedings of the 11th International Symposium on Stabilization, Safety, and Security of Distributed Systems, SSS ’09, pages 369–383, Berlin, Heidelberg, 2009. Springer-Verlag.

[32] Damien Imbs and Michel Raynal. The multiplicative power of consensus numbers. In Proceedings of the 29th ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, PODC ’10, pages 26–35, New York, NY, USA, 2010. ACM.

[33] Amos Israeli and Amnon Shaham. Optimal multi-writer multi-reader atomic register. In Proceedings of the Eleventh Annual ACM Symposium on Principles of Distributed Computing, PODC ’92, pages 71–82, New York, NY, USA, 1992. ACM.

[34] Prasad Jayanti. On the robustness of Herlihy’s hierarchy. In Proceedings of the Twelfth Annual ACM Symposium on Principles of Distributed Computing, PODC ’93, pages 145–157, New York, NY, USA, 1993. ACM.

[35] Leslie Lamport. Concurrent reading and writing. Commun. ACM, 20(11):806–811, Nov 1977.

[36] Leslie Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7):558–565, Jul 1978.

[37] Daniel Lister. A pedagogical presentation of the BG-simulation. Master’s thesis, University of Toronto, 2013. Bibliography 106

[38] Wai-Kau Lo and Vassos Hadzilacos. All of us are smarter than any of us: Nondeterministic wait-free hierarchies are not robust. SIAM Journal on Computing, 30(3):689–728, 2000.

[39] Michael C Loui and Hosame H Abu-Amara. Memory requirements for agreement among unreliable asynchronous processes. Advances in Computing research, 4(163-183):31, 1987.

[40] Nancy Lynch. Distributed Algorithms. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1996.

[41] Gil Neiger. Set-linearizability. In Proceedings of the Thirteenth Annual ACM Symposium on Prin- ciples of Distributed Computing, PODC ’94, pages 396–, New York, NY, USA, 1994. ACM.

[42] Ophir Rachman. Anomalies in the wait-free hierarchy. In Proceedings of the 8th International Workshop on Distributed Algorithms, WDAG ’94, pages 156–163, London, UK, UK, 1994. Springer- Verlag.

[43] Eric Ruppert. Determining consensus numbers. SIAM Journal on Computing, 30(4):1156–1168, Oct 2000.

[44] Michael Saks and Fotios Zaharoglou. Wait-free k-set agreement is impossible: The topology of public knowledge. SIAM J. Comput., 29(5):1449–1483, Mar 2000.

[45] Paul Vitanyi and Baruch Awerbuch. Atomic shared register access by asynchronous hardware. In 27th Annual Symposium on Foundations of Computer Science, SFCS 1986, pages 233–243, Oct 1986. Appendix A

A Comparison of Consensus Objects

In this thesis, the consensus objects we refer to have the sequential specification given by Algorithm 18. + For all n ∈ Z∞, the n-consensus object has n ports: each process can apply a propose(v) operation for any value v to any port i ∈ [1..n]. The state of the n-consensus object consists of a single field val that stores the first value proposed, or ⊥ if no propose operation has been applied yet. The first operation sets val to its proposal value, and all operations return the value now stored in val, thus achieving consensus.

Algorithm 18 Sequential speciﬁcation of the n-consensus object. 1: procedure propose(v) applied to port i ∈ [1..n] 2: if val = ⊥ then 3: val ← v 4: return val

The sequential specification of the n-consensus object can be formally given in terms of a set of states, a set of operations, a set of responses, and a state transition relation. For brevity, we omit this formal definition here. From the above definition, it is straightforward to show that:

+ Theorem A.1. For all n ∈ Z∞, the n-consensus object is equivalent to the n-consensus task in the sense that:

(a) The n-consensus object can solve the n-consensus task.

(b) The n-consensus object can be implemented using registers and an arbitrary solution to the n- consensus task.

Proof. (a): Algorithm 19 shows how the n-consensus task can be solved using an n-consensus object

D. Each process p ∈ [1..n] simply performs a propose(vp) operation on port p of D, where vp is the proposal value of p, and decides the return value. The correctness of this trivial implementation is straightforward. (b): Algorithm 20 shows how the n-consensus object can be implemented using an n-consensus task solution P [1..n], and a register X that initially contains ⊥. Intuitively, to determine its return value, a propose(v) operation on port i simply needs to execute protocol P [i] with input v and return the output. However, if there are multiple operations on port i, protocol P [i] would be used more than once.

107 Appendix A. A Comparison of Consensus Objects 108

Algorithm 19 Solving the n-consensus task using an n-consensus object D.

1: decide D.propose(vp) on port p

Algorithm 20 Implementing the n-consensus object using an n-consensus task solution P [1..n], and a register X. 1: procedure propose(v) on port i ∈ [1..n] 2: if X.read() = ⊥ then 3: v0 ← P [i].execute(v) 4: X.write(v0) 5: return X.read()

To prevent this, each operation that executes a protocol of P [1..n] writes the output to X, so subsequent operations can simply return the value read from X without needing to execute any protocol of P [1..n]. Thus, to perform an operation propose(v) on port i of the n-consensus object, a process executes the following steps:

1. It reads X to check whether any operation has determined a return value. If not, it executes protocol P [i] with input v and writes the output to X.

2. It returns the value now stored in X.

The correctness of this direct implementation is also straightforward. Theorem A.1

There exists another version of n-consensus objects that we will refer to as breakable n-consensus ob- + jects. For all n ∈ Z∞, the breakable n-consensus object is a non-ported object that supports propose(v) operations for any value v. The state of the breakable n-consensus object consists of two fields: val and count. The val field stores the first value proposed, or ⊥ if no propose operation has been applied yet. The count field stores the number of operations proposed so far, so initially contains 0 and is incremented by every operation. The first operation sets val to its proposal value, and the first n operations return the value now stored in val, thus achieving consensus. All subsequent operations will return arbitrary values.

Algorithm 21 Sequential speciﬁcation of the breakable n-consensus object. 1: procedure propose(v) 2: count ← count + 1 3: if count > n then 4: return arbitrary value 5: if val = ⊥ then 6: val ← v 7: return val

The sequential specification of the breakable n-consensus object can be formally given in terms of a set of states, a set of operations, a set of responses, and a state transition relation. For brevity, we omit this formal definition here. From the above definition, we can show that:

+ Theorem A.2. For all n ∈ Z∞, the breakable n-consensus object is equivalent to the n-consensus task in the sense that: Appendix A. A Comparison of Consensus Objects 109

Algorithm 22 Solving the n-consensus task using a breakable n-consensus object D.

1: decide D.propose(vp)

(a) The breakable n-consensus object can solve the n-consensus task.

(b) The breakable n-consensus object can be implemented using registers and an arbitrary solution to the n-consensus task.

Algorithm 23 Implementing the breakable n-consensus object using an n-consensus task solution P [1..n], and a Fetch & Increment object F. 1: procedure propose(v) 2: j ← F.fetch&increment() 3: if j > n then 4: return arbitrary value 5: return P [j].execute(v)

Proof. (a): Algorithm 22 shows how the n-consensus task can be solved using a breakable n-consensus object D. Each process p ∈ [1..n] simply performs a propose(vp) operation on D, where vp is the proposal value of p, and decides the return value. The correctness of this trivial implementation is straightforward. (b): If n = 1, then the breakable 1-consensus object is trivial to implement: simply have every operation return its own proposal value. So suppose n ≥ 2. In previous work, Afek et al. [5] proved that (any solution to) the 2-consensus task, together with registers, can be used to implement a Fetch & Increment object for any number of processes. Hence Algorithm 23 shows how the breakable n-consensus object can be implemented using an n- consensus task solution P [1..n], and a Fetch & Increment object F. To perform an operation propose(v) on the breakable n-consensus object, a process executes the following steps:

1. It gets an integer j from F.

2. If j > n, it returns an arbitrary value.

3. It executes protocol P [j] with input v and returns the output.

The correctness of this direct implementation is also straightforward. Theorem A.2

By Theorems A.1 and A.2, we have:

+ Corollary A.3. For all n ∈ Z∞, the n-consensus object is equivalent to the breakable n-consensus object.