Probabilistic Indistinguishability and the Quality of Validity in Byzantine Agreement

Guy Goren∗1, Yoram Moses†1, and Alexander Spiegelman2

1Technion 2Novi Research

Abstract Lower bounds and impossibility results in distributed computing are both intellectually challenging and practically important. Hundreds if not thousands of proofs appear in the literature, but surprisingly, the vast majority of them apply to deterministic algorithms only. Probabilistic distributed problems have been around for at least four decades and receive a lot of attention with the emergence of blockchain systems. Nonetheless, we are aware of only a handful of randomized lower bounds. In this paper we provide a formal framework to reason about randomized distributed algorithms. We generalize the notion of indistinguishability, the most useful tool in deterministic lower bounds, to apply to a probabilistic setting. The power of this framework is applied to completely characterize the quality of decisions in the randomized multi-valued Consensus problem in an asynchronous environment with Byzantine faults. That is, we provide a tight bound on the probability of honest parties deciding on a possibly bogus value and prove that, in a precise sense, no algorithm can do better. arXiv:2011.04719v1 [cs.DC] 9 Nov 2020

∗The research of Guy Goren was partly supported by a grant from the Technion Hiroshi Fujiwara cyber security research center and the Israel cyber bureau, as well as by a Jacobs fellowship. †Yoram Moses is the Israel Pollak academic chair at the Technion. His work was supported in part by the Israel Science Foundation under grant 2061/19, and by the U.S.-Israel Binational Science Foundation under grant 2015820. 1 Introduction

Randomized algorithms have a long tradition in distributed computing [33], where they have been applied to many different problems in a variety of models [28]. In the context of fault-tolerant agreement they have served to overcome the impossibility of agreement in asynchronous settings [20, 34, 8], and have significantly improved efficiency compared to deterministic solutions [17, 24]. With the recent prevalence of blockchain systems, Byzantine agreement algorithms that can overcome malicious parties have found renewed interest in both industry and academia. For obvious reasons, blockchain systems should strive to minimize the share of decisions that originate from malicious parties, and to increase the share originating from honest ones. A natural question, then, is what are the inherent limits on the quality of Byzantine agreement algorithms in this regard? Namely, what can we say about the probability with which an algorithm can guarantee that a good decision is made? Given their practical importance, characterizing the power and limitations of randomized distributed algorithms for agreement has become ever more desirable. However, obtaining tight, or nontrivial, probabilistic bounds on properties in the asynchronous Byzantine setting can be a challenging task. As is well known, there are “Hundreds of impossibility results for distributed computing” [19]. But very few of them apply to randomized protocols. Unfortunately, there is currently a dearth of general tools for characterizing the properties of randomized algorithms. The notion of indistinguishability has for years been one of the most useful tools for proving deterministic lower bounds and impossibility results in distributing computing [6]. Such deterministic lower bounds typically rely on the fact that if a correct party cannot distinguish between two executions of a deterministic protocol (i.e., its local state is the same in both), then it performs the same actions in both. In a randomized algorithm, the fact that two executions are indistinguishable to a given party up to a certain time does not ensure that its next action will be the same in both. Moreover, a single execution does not by itself provide information on the probability with which actions are performed. As a result, the classic notion of indistinguishability does not directly capture many of the probabilistic aspects of a randomized algorithm. A key contribution of this paper is the introduction of a framework for reasoning about properties of probabilistic protocols. In a nutshell, we generalize deterministic executions, which are modeled as sequences of alternating configurations and local events, into tree-based probabilistic scenarios. Just as, in deterministic algorithms, a protocol execution is precisely determined by the adversary’s strategy, the adversary’s strategy determines a scenario in a randomized algorithm. A scenario is a tree in which each node represents a configuration, and (directed) edges from a node point at the possible successor configurations. The tree edges are weighted, with the weight of an edge specifying the probability that the successor that it points to is chosen when the computation reaches the current node. An important feature of scenarios is that they allow us to formally define probabilistic properties of distributed algorithms. Currently, properties such as “all honest parties decide on the same value with high probability” or “all honest parties terminate with probability 1” are not well defined without an explicit definition of the probability space on all possible executions. Such properties cannot be evaluated based on an individual execution. Scenarios, in contrast, capture the algorithmic randomness and allow us to reason about probabilistic events therein. Statements like “the probability that party pi performs an action α in a given scenario scn of algorithm A is at least x” are well-defined. In deterministic algorithms, indistinguishability can be determined based on a party pi’s local history, i.e., the sequence of local states that pi passes through in an execution. In order to maintain

1 relevant probabilistic information, we move from sequences to trees of local states. This facilitates the definition of a notion of probabilistic indistinguishability among scenarios, whereby two scenarios are considered indistinguishable to a process pi if they induce identical trees of pi’s local states. Scenarios and probabilistic indistinguishability provide a formal and convenient framework that can be used to simplify existing proofs and prove new ones. It allows us to construct probabilistic lower bound proofs in a manner that is similar to how they are constructed, based on ordinary indistinguishability, in the deterministic case. A significant feature of this framework is its simplicity and ease of use. The notions contain just enough structure beyond that of their deterministic analogues to capture the desired probabilistic properties. Our original motivation for developing the above framework was to formally prove what is the best guarantee regarding the share of good decisions made by a randomized Byzantine agreement algorithm in our setting. And indeed, we prove a tight bound on the quality of decisions in Section 5. This is evidence that, at least for this problem, our tools are fine enough to capture an essential aspect of a randomized algorithm. Various works that establish randomized lower bounds on distributed algorithm develop approaches that suffice to prove particular results of interest. We suggest that the notions of scenarios and probabilistic indistinguishability are general tools that can be used more broadly. As a case in point, we use them to reprove a central lower bound result of [1] in Appendix B. The new proof follows the outline of the original one and makes use of the same underlying structure. But the use of scenarios and probabilistic indistinguishability provide the arguments with a rigorous foundation and improve their transparency.

2 Related Work

Deterministic indistinguishability in distributed computing. The main inherent limita- tion of distributed computing leveraged by most proofs is the lack of global knowledge [27, 16]. That is, each party needs to evaluate the global state of the system based on its local state and act accordingly. Deterministic indistinguishability captures exactly that: if two executions of a deterministic protocol looks the same from some party’s local point of view, then this party performs exactly the same sequence of actions in both. Good surveys of techniques used in these proofs are presented in [6, 19, 27]. They all utilize indistinguishable executions, but diﬀer in the way they construct them.

Lower bounds approaches for randomized distributed protocols. There are several lower bound and impossibility results for randomized algorithms. One approach is to reduce the distributed problem into a cleaner mathematical one that abstracts away the issues of local knowledge, and apply methodologies from other ﬁelds to the latter problem [3, 26]. Another approach used by Attiya and Censor-Hillel [4, 5] reduces the argument to deterministic indistinguishability. By considering a ﬁxed random tape, they abstract away the randomness from the executions. On top of this, they employ higher-level techniques (e.g., multivalency and combinatorial arguments). A recent work [1] extends a lower bound by Dolev and Reischuk [14] on the communication complexity of Byzantine agreement to the randomized case. They informally apply deterministic indistinguishability arguments to executions of randomized algorithms. The work of Fich, Herlihy and Shavit in [18] proves space lower bounds for randomized shared objects. Their technique exploits

2 covering arguments, which, in turn, rely on deterministic indistinguishability. To this end, they remove the stochastic nature of the problem by considering the “nondeterministic solo termination” property that requires only a non-zero probability for solo termination. They also show that their bounds immediately apply to nondeterministic wait-free objects, but conjecture that the bounds are not tight for that case. A possible reason could be that their technique ignores the quantitative probabilistic nature of the problem. A recent paper by Ellen, Gelashvili, and Zhu [15] shows that nondeterministic solo termination and obstruction freedom are equivalent in the speciﬁc context of space lower bounds. Each one of the above works developed an ad hoc approach in order to obtain lower bounds on an important randomized problem. However, there is currently no standard tool analogous to deterministic indistinguishability that can be applied in a uniform way to such randomized analyses.

Notions of indistinguishability in cryptography. Indistinguishability plays a central role in modern cryptography. In major topic areas such as Zero Knowledge [23] and Multi-Party Compu- tation [37, 22] the goal is to ensure that operations among a pair, or a set, of parties are performed in a manner that preserves the secrecy of specific information. In this context, indistinguishability is defined over distributions over protocol executions, and assumptions about the computational limitations on some of the parties typically play a central role. While the associated literature is large and well developed [21], these techniques are very effective at facilitating the development of cryptographic protocols and establishing cryptographic security, but they do not directly lend themselves to the analysis of probabilistic properties of randomized algorithms as studied in this paper.

3 Model

We consider a standard message passing model with a set Π of n parties and an adversary [7]. Parties communicate via peer-to-peer communication links over which they send and receive messages. The communication links are asynchronous.1 That is, they are reliable but controlled by an adversary, who controls the message delivery times. A message that has been sent on a link but not yet delivered is pending. To be as general as possible, we do not restrict the definition of probabilistic indistinguishability to a specific adversary. In other words, we want the definition to apply to a wide set of assumptions on the adversary, including failure types and computational ability. Every particular set of assumptions specifies a set of possible strategies for the adversary. In Section 5.3, where we present a use of probabilistic indistinguishability, we describe the assumptions under which we prove the lower bound. Each party maintains a well-defined local state at all times. We assume for simplicity that the 0 local state of each party pi at some time T consists of its initial state li and the finite sequence of local events at pi up to time T. This sequence is composed of the actions that pi has performed (including the messages it has sent) before time T, as well as the messages that pi received until 0 T. In particular, its initial local state is hli , []i. (For example, in a Consensus algorithm the initial 0 state of each party is its input value vi, that is, li = vi.) A configuration is a mapping from parties to their local states and from communication links to the set of messages therein (messages that were sent but not yet delivered). An initial configuration

1Our deﬁnitions can be easily translated to synchronous communication and shared memory models.

3 associates with each party its initial state and each link with an empty set of pending messages. An algorithm defines the behavior of parties as state machines, where state transitions are associated with local steps (including message sending) or message deliveries. We consider both local steps and message deliveries to be local events. An execution of a deterministic algorithm A is a (finite or infinite) sequence of the form e = hC0, φ1, C1, φ2, C2, φ3, ...i, where C0 is an initial configuration, Ck is a configuration, and φk is either a local step or a message delivery for every k > 0. In case φk is a local step by an honest party, the transition from Ck−1 to Ck occurs according to the algorithm A. If φk is the delivery of a message m to party pi from party pj, then Ck is similar to Ck−1 except that m is removed from the link between pi and pj and appended to pi’s local state. A legal execution of an algorithm A is an execution that satisfies the following:

• Fair schedule: every party performs an inﬁnite number of steps.2 • Reliable links: every message sent is eventually delivered. Moreover, every delivered message was previously sent.

• Failure assumption: at most t parties are corrupted in the execution. In the next section we will generalize the commonly used notion of indistinguishability to randomized algorithms. For completeness, we ﬁrst provide the deﬁnition of indistinguishable execu- 1 1 1 1 1 1 2 2 2 2 2 2 tions. Two executions e1 = hC0 , φ1, C1 , φ2, C2 , φ3, ...i and e2 = hC0 , φ1, C1 , φ2, C2 , φ3, ...i are 1 2 indistinguishable to a party pi if its local state is the same in Ck and Ck for every k ≥ 0.

4 Probabilistic Indistinguishability

This section deﬁnes the notion of probabilistic indistinguishability. In a nutshell, we generalize deterministic executions by deﬁning probabilistic scenarios, which are tree shaped structures. Similar to deterministic indistinguishability for executions, probabilistic indistinguishability occurs when a party can not distinguish between two scenarios. Indistinguishability of two executions serves to prove that a party must perform the same actions in both. Indistinguishability of two scenarios serves to prove that a party must have the same probabilities to perform the same actions in both.

Coin Abstraction. In modeling probabilistic protocols we will find it convenient to use the notion of a coin object, which generalizes common notions such as local and global coins [25, 3, 12]. A coin is an atomic object (cf. [28, 7]). It is accessed through a flip which immediately returns a value that satisfies a specific set of properties. For example, a simple local unbiased coin returns 0 or 1 with probability 1/2 for each. Another example is a global coin, which, in the unbiased case, returns 0 or 1 with probability 1/2 to the first party that flips it, and returns the same value for every subsequent flip. This definition of a coin object implies that a coin flip is a local step, and in particular, the adversary does not control its outcome but can influence its scheduling. Coin objects allow us to abstract the specific mechanism by which randomness is generated, thus, enabling our results to apply to different contexts.

2Might be an empty step. E.g., if the party has halted.

4 4.1 Scenarios A scenario is a directed weighted tree in which each node is a conﬁguration, and edges represent local events. When clear from the context, we may slightly abuse notation by writing v ∈ scn to denote that node v appears in the scenario scn. A scenario scn of an algorithm A must satisfy the following properties:

• The root of the scenario tree is an initial conﬁguration C0.

• The set Eout(v) of edges connecting a node v to its children represents a single local event. The weights on the edges in Eout(v) are positive and their sum is 1.

– If |Eout(v)| = 1, then the edge (v, u) ∈ Eout(v) represents a deterministic local event (i.e., message delivery) that transforms the configuration represented in v to the configuration represented in u. (Notice that a coin flip might also be a deterministic local event. E.g., the output values of all flips after the first flip of a perfect shared coin are deterministically determined.)

– Otherwise, (when |Eout(v)| > 1), the edges in Eout(v) represent a single coin flip local event. Each edge (v, u) ∈ Eout(v) represents a coin flip output that transforms the configuration represented in v to the configuration represented in u, where the weight on the edge is the probability of the coin flip to return that output.

• Each path in the scenario tree represents a legal execution of A. Additionally, each coin ﬂip in the path satisﬁes the respective coin’s properties.

We note that similarly to how, in deterministic algorithms, an adversary’s strategy determines an execution of A, in randomized algorithms, an adversary’s strategy determines a scenario of A. A probabilistic scenario generalizes an execution in the sense that a deterministic algorithm yields scenarios that consist of a single path.

Identifying probabilities on configurations. A scenario scn induces a probability space defined by the triplet (Ω, F, Pscn) which is specified as follows. Ω is the set of paths (executions) in scn. For each node v in the scenario, define by Sv the set of executions (paths) in scn that pass through v. F is the sigma algebra generated by {Sv : v ∈ scn}. (Closing under comple- ment and countable unions.) Pscn is the probability function defined by Pscn(Sv) , the product of the edge weights along the path from the root to v.3 As required, our definitions satisfy both ¯ Pscn(S) = 1 − Pscn(S) and that Pscn is countably additive.

Indistinguishability. A scenario scn induces an i-local experience for party pi – denoted by scni – which is also a tree-based structure. For each unique local state li of pi that appears in scn, there is a node v ∈ scni labeled with the tuple hli, pli i, where pli ∈ [0, 1] is the probability of pi to reach li in scenario scn. More formally, for a local state li, deﬁne Sli = {Sv | v ∈ scn and v represents a conﬁguration that contains li}. The resulting probability assigned to v ∈ S scni labeled with li is pli = Pscn( Sv). Sv ∈Sli

3 Pscn(root) , 1.

5 0 0 The root of scni is labeled with li , i.e., pi’s local state at the root of scn, and pli = 1. For each node vli ∈ scni labeled with an i-local state li, its children are the set of nodes Child(vli ). A node ˆ u ∈ scni is in Child(vli ) iff it is labeled with an i-local state li(6= li) that may directly follow li in an execution contained in scn. I.e., there are v0, u0 ∈ scn such that u0 is a child of v0, and v0, u0 ˆ represent configurations that contain li and li, respectively. (Notice that according to our definition of local states that contain the full history of local events, once pi has transitioned from a local state li, it never returns. The definition of scni could easily be generalized to capture recurring local states.) Constructing an i-local experience scni is linear in the size of scn. An algorithm is provided in Appendix A for completeness. Definition 1 (Probabilistic Indistinguishability). Two scenarios are indistinguishable to a party pi if they induce the same i-local experience.

For a given scenario scn and a local action α of party pi, we can assign a probability for pi to perform α in scn according to the probability space induced by scn. A central feature of the deﬁnition is captured as follows: Corollary 1. Let scn-A and scn-B be two scenarios of algorithm A that are probabilistically indistiguishable to party pi. For each action α of process pi, the probability that pi performs α is equal in scn-A and scn-B.

Proof. Let scn-A and scn-B satisfy the assumptions, and let α be an action of process pi. Recall that the local state of process pi contains the sequence of all actions that pi has performed up A to its current state. Denote by Vα the set of nodes in scn-A that represent configurations in which α appears only once in the sequence of local events contained in pi’s local state, and it A is the last element in that sequence. As before, for a node v ∈ scn-A define by Sv the set of A executions in scn-A that pass through v. Recall that the set Sv is an event in the probability A S A space induced by scn-A. Thus, Sα , Sv constitutes a measurable event in the probability A v∈Vα A space, and PA(Sα ) corresponds to the probability that process pi performs action α in the scenario B B B scn-A. Similarly, define Vα , Sα , and PB(Sα ) with respect to scn-B. It remains to show that P (SA) = P (SB). Recall that for a local state l , we have defined the set of path sets Sscn = A α B α i li scn {Sv | v ∈ scn s.t. v represents a configuration that contains li}. Ai Let LVα be the set of nodes in the i-local experience scn-Ai that are labeled with a local state li in which α appears only once and is the last element in the sequence of local events. Similarly, Bi define LVα with respect to scn-Bi.

     A [ A  [  [ A X [ A PA(Sα ) = PA  Sv  = PA   Sv  = PA( Sv ). A A A A A A A v∈Vα i S ∈S i S ∈S hli,pi∈LVα v li hli,pli i∈LVα v li 1 2 A 1 2 i Where the last equality follows from the fact that for any hli , pli i= 6 hli , pli i ∈ LVα we have that A T A S 1 S 2 = ∅. This is because α appears only once in the respective l , so a path cannot contain li li i two diﬀerent local states in which α is both last and appears only once. By deﬁnition of scni, we S scn have that pli = Pscn( Sv ). Therefore, Sscn∈Sscn v li

X [ A X PA( Sv ) = pli . Ai SA∈SA Ai hli,pli i∈LVα v li hli,pli i∈LVα

6 Similarly,

B X PB(Sα ) = pli . Bi hli,pli i∈LVα

Ai Bi By deﬁnition of scenario indistiguishability, scn-Ai= scn-Bi, and so LVα = LVα . Hence, A B PA(Sα ) = PB(Sα ), as claimed.

5 Byzantine Agreement with Qualitative Validity

In this section we demonstrate the power of the probabilistic indistinguishability tool. We start in Section 5.1 with a background on existing Byzantine validity conditions and motivation for qualitative validity. We then deﬁne the Byzantine agreement problem with Qualitative Validity in Section 5.2, and use probabilistic indistinguishability in Section 5.3 to prove that the Qualitative Validity condition is optimal in a way that no algorithm can satisfy a strictly stronger validity. For completeness, in Appendix C we present an agreement protocol that satisfy Qualitative Validity. Thus, showing Qualitative Validity to be tight.

5.1 Motivation for qualitative validity Byzantine Agreement is one of the most fundamental problems in distributed computing. A set of n parties, some of which might be Byzantine, need to agree on a unique value. Ideally, we would like the decision to be on a value proposed by an honest party. And indeed, in the classic binary case [8], where the set of possible inputs is {0, 1}, this is exactly what the Validity property of Byzantine Agreement requires. However, in the multi-valued case, in which inputs come from some arbitrary domain V, this is generally impossible since a Byzantine party can propose a value that is not proposed by honest parties and otherwise act honestly [30]. Since multi-valued Byzantine Agreement protocols are the core of many Blockchain systems [10, 36], the issue of preventing malicious attacks on the “quality” of decisions is becoming more and more important. The question is, therefore, what is the best validity property a multi-valued Byzantine Agreement protocol can provide. That is, what are the conditions under which an algorithm can be guaranteed to decide on a value proposed by an honest party and, when these conditions fail to hold, what is the probability at which such a decision can be ensured. Two incomparable validity definitions were previously proposed for the multi-valued case, called weak Validity and external Validity. As in the binary case, weak Validity [31] requires that if all honest parties propose the same value v, then v is the only value that can be decided. However, if honest parties propose different values, then they can decide on some pre-defined default value (which we denote by ⊥). The initial motivation for weak validity was a spaceship cockpit with four sensors, one of which might be broken [32]. However, from a contemporary practical point of view, such a definition is useless for building Byzantine state machine replication (SMR) (e.g., as in blockchains) [2, 29] since a decision of ⊥ in such a setting does not allow the system to progress. Hence, in order to guarantee progress, the honest parties need to agree a priori even in failure-free runs. To deal with this issue, Cachin, Kursawe, Petzold and Shoup [11] introduced the external Validity property, which allows the decision to be any value as long as it is valid according to some external predicate (e.g., a valid transaction in a blockchain system). In particular, external validity does

7 not preclude a situation in which the decision value does not originate from an honest party. To overcome this deficiency, Abraham, Malkhi, and Spiegelman [2] extended the definition of external validity with a decision quality requirement, which bounds the probability of the decision to be a value proposed by the Byzantine parties. Specifically, they provide an algorithm that guarantees probability of at least 1/2 for the decision value to be an input of an honest party. Moreover, they claim in the paper that no algorithm can do better in terms of quality, but provided no proof. While their claim is very intuitive, it is not obvious how to prove it without a notion such as probabilistic indistinguishability. Note that the two variants of multi-valued validity are incomparable. On the one hand, with external Validity parties never agree on ⊥ and thus SMR progress is guaranteed. On the other hand, parties may agree on a bogus value proposed by malicious participants even if all honest parties input the same value (a priori agree). In addition, note that neither definition takes into account the actual number of failures f ≤ t. Below we define the Qualitative Validity property, which promises progress and is stronger than each of these validity conditions.

5.2 Problem definition In this section we assume a computationally bounded adversary that can corrupt up to t of the n parties, where n = 3t + 1. Parties corrupted by the adversary are called Byzantine and may arbitrarily deviate from the protocol. Other parties are honest. Given a scenario, the maximal number of parties the adversary corrupts throughout a single path in the scenario is denoted by f ≤ t. In addition, every party pi starts with some initial input value vi from some domain V, i.e., pi’s local state in the initial configuration (the root of the scenario) is si = vi. We denote by Vin = {{vi | pi ∈ Π}} the multiset of all input values. For every multiset M and value v ∈ M, we denote by mult(v, M) the multiplicity of v in M. The maximum multiplicity in a multiset M is denoted by max mult(M) , max({mult(v, M) | v ∈ M}). We distinguish between static and adaptive adversaries and between weak and strong ones. A weak adversary does not observe the local states of honest parties, whereas a strong one does. A static adversary knows the input values but is limited to corrupt parties before the scenario starts, that is, immediately after the root. An adaptive one is allowed to corrupt parties on the fly. To strengthen our result, we consider a weak and static adversary for the lower bound, and a strong and adaptive one for the algorithm. The Agreement problem exposes an API to propose the input value and to output a decision from the domain V. An Agreement algorithm is one that satisfies the Agreement, Termination and Validity properties. As deterministic solutions in failure-prone asynchronous systems are impossible by FLP [20], we are interested in algorithms that never compromise safety, and satisfy liveness almost surely. That is, we require that every scenario scn of the algorithm must satisfy the following properties:

• Agreement: In each path (i.e., every execution) of scn, all honest parties that decide, decide on the same value.

• Probabilistic Termination: An honest party decides with probability 1.

As for validity, we extend previous definitions [32, 13, 11] to capture the optimal conditions under which parties decide on a value proposed by an honest party. We remark that the initial values are determined by the initial configuration in an execution, and such configurations are root nodes of

8 scenarios. It follows that all executions in a scenario share the same input vector. The scenario notation allows us to require the following non-deterministic property:

• Qualitative Validity: If max mult(Vin)−f ≥ 2t+1, then all honest parties that decide, decide f on values in Vin. Otherwise, the probability to decide on a value in Vin is at least 1 − n−t . One important feature of Qualitative Validity is that in scenarios without failures honest parties can only decide on a value in Vin. Moreover, parties never decide on a pre-deﬁned ⊥ and the probability to decide on a value in Vin is proportional to the actual number of Byzantine parties.

5.3 Tight bounds on Qualitative Validity In this section we prove that we can not hope that an algorithm for multi-valued Byzantine Agree- ment that satisfies Agreement and Probabilistic Termination will provide more than Qualitative Validity. Moreover, we then show that this validity condition is the best we can achieve, by pre- senting an algorithm that satisfies this validity property. The following lemma states that if we have f = x failures and the condition in the Qualitative Validity definition (max mult(Vin) − f ≥ 2t + 1) does not hold, then we can always find a multiset M ⊂ Vin of size n − t − f such that there is no value in M with multiplicity higher than t.

Lemma 2. Consider a multiset Vin of n = 3t+1 values and an integer 0 < x ≤ t. If max mult(Vin)− x < 2t + 1, then there is a multiset M ⊂ Vin such that |M| = n − t − x and max mult(M) ≤ t.

Proof. Let V>t = {v | mult(v, Vin) > t} be the set of values that appear more than t times in Vin. Since |Vin| = 3t + 1, |V>t| ≤ 2. Now consider 3 case:

• First, |V>t| = 0. The lemma trivially follows.

• Second, |V>t| = 1. Let V>t = {v1}. let X , T ⊂ M be multisets s.t. |X | = x, |T | = t, and ∀u ∈ X ∪ T , u = v1. Since max mult(Vin) − x < 2t + 1, M = Vin \ (X ∪ T ) satisﬁes the lemma.

0 0 • Third, 2 ≤ |V>t| = 2. Consider v1, v2 ∈ V>t. Let M be a multiset of size 2t s.t. mult(v1, M ) = 0 0 0 mult(v2, M ) = t. Since n = 3t+1, |M | ≥ n−t−x for every x > 0. Therefore, any M ⊆ M satisﬁes the Lemma.

We use Lemma 2 and Corollary 1 to prove that if a value v is decided with a probability that is higher than “its fair share”, then we can construct a probabilistically indistinguishable scenario in which v is proposed only by corrupted parties. As a result, in the constructed scenario, the corrupted parties get more than “their fair share”. Formally, we say that an algorithm A satisﬁes a Validity property that is strictly stronger than Qualitative Validity if (1) For any f ≤ t and Vin such that max mult(Vin) − f ≥ 2t + 1, in every scenarios of A with f maximal corrupted parties and the input multiset Vin an honest party that decides, decides on a value in Vin; and (2) For any f ≤ t and Vin such that max mult(Vin) − f < 2t + 1, every scenarios of A with f maximal f corrupted parties and the input multiset Vin have a probability of at least than 1 − n−t to decide on a value in Vin; and (3) There exist f ≤ t and Vin that satisﬁes max mult(Vin) − f < 2t + 1, and every scenario of A with f maximal corrupted parties and the input multiset Vin have a probability f that is strictly higher than 1 − n−t to decide on a value in Vin. To show that Qualitative Validity is optimal we prove the following:

9 Theorem 1. There is no asynchronous Byzantine Agreement algorithm that satisfies a validity property that is strictly stronger than Qualitative Validity. Proof. Assume, by way of contradiction, that there is such an algorithm A. In particular, there exist f ≤ t and Vin that satisfies max mult(Vin) − f < 2t + 1, and every scenario of A with f maximal corrupted parties and the input multiset Vin have a probability that is strictly higher than f 1 − n−t to decide on a value in Vin. To show the contradiction we use Vin to construct a scenario 0 0 with a multiset Vin 6= Vin of input values and f corrupted parties such that the probability of A to 0 f 0 agree on a value in Vin is strictly lower than 1 − n−t , which contradicts (1) or (2) from the strictly stronger conditions. We next describe a few scenarios under different adversarial strategies and use Probabilistic Indistinguishability between them to prove the theorem. 1 1 1 Scenario 1. Vin = Vin and f = f. By Lemma 2, there is a multiset M ⊂ Vin such that |M| = n − t − x and max mult(M) ≤ t. Let M ⊂ Π be the set of parties with inputs in M, i.e., the multiset M = {vi | pi ∈ M}. Let F , T ⊂ Π be two sets of parties such that |F | = f, |T | = t, and Vin \M = {vi | pi ∈ F ∪ T }. The adversary (statically) corrupts the parties in F and delays all messages from parties in T until all honest parties in M decide. Messages sent among parties 1 in M ∪ F are immediately delivered. Let vf 6∈ Vin, the corrupted parties act like honest parties that get vf as an input. 2 1 Scenario 2. Consider a multiset of input values Vin that is identical to Vin except that parties in F get vf as an input. The adversary corrupt parties in T . Messages sent among parties in F ∪ M are immediately delivered. The corrupted parties send no messages.

F F Byzantine. Act Honest. as if input � ! M input �! M input values in input values in ℳ T ℳ T Byzatnine. Send delayed no messages

(a) Scenario 1. Parties in F are Byzantine that act (b) Scenario 2. Parties in F are honest and vf ∈ 1 2 as if they honest with input vf 6∈ Vin. Messages Vin. Parties in T are Byzantine that send no mes- from parties in T are delayed. sages.

Figure 1: Sets F , T and M divide the parties into 3 disjoint sets. In both scenarios the set M contains parties with inputs from M. Scenario 1 and scenario 2 are probabilistically indistinguishable to all processes in M.

Clearly, by Deﬁnition 1, scenario 1 and scenario 2 are probabilistically indistinguishable to all processes in M (see illustration in Figure 1). By the Probabilistic termination property, all parties in M in scenario 2 decide with probability 1. Therefore, Corollary 1 implies that, in scenario 1, all parties in M decide with probability 1 as well. 1 For every value v ∈ Vin denote by P (v) the probability of the honest parties in M to decide on v in scenario 1. By assumption, the probability to decide on a value in Vin is strictly higher than 1 − f in scenario 1, i.e., P P 1(v) > 1 − f . Therefore, there are two cases for scenario 1: n−t v∈Vin n−t

10 1 mult(u,M) • First, there is a value u ∈ M s.t. P (u) > n−t .

1 • Otherwise, there is a value w 6∈ M ∪ {vf } s.t. P (w) > 0.

First case: For the ﬁrst case, let U = {pi ∈ M | vi = u}. Since max mult(M) ≤ t, we get |U| ≤ t. Consider the following scenario: 3 1 Scenario 3. Consider a multiset of input values Vin that is identical to Vin except (1) parties in F 0 get vf as an input; and all other parties that get u as an input in scenario 1 get some value v 6= u. 3 (Note that u 6∈ Vin). The adversary corrupts all parties in U s.t. corrupted parties act as honest parties that got u as an input. Messages from parties in T are delayed until all parties in M \ U decide, and messages sent among parties in M ∪ F are immediately delivered.

F F Byzantine. Act Honest. U as if input �! input �! M Byzantine. Act input values in T as if input u T ℳ delayed delayed M\U

(a) Scenario 1. Parties in F are Byzantine that act (b) Scenario 3. Parties in F are honest and vf ∈ 1 3 as if they honest with input vf 6∈ Vin. Messages Vin. Messages from parties in T are delayed. The from parties in T are delayed. set U ⊂ M contains parties that input u in Sce- nario 1. Here they are Byzantine that act as if they 3 honest with input u. The value u is not in Vin.

Figure 2: In both scenarios parties in M \ U input the same values. Scenario 1 and Scenario 3 are probabilistically indistinguishable to all processes in M \ U.

Note that scenario 1 and scenario 3 are probabilistically indistinguishable for parties in M \ U (see illustration in Figure 2). Therefore, by Corollary 1, the probability of parties in M \ U to |U| 3 agree on u in scenario 3 is higher than n−t and thus the probability to agree on a value in Vin in |U| f3 scenario 3 is strictly lower than 1 − n−t = 1 − n−t . A contradiction for A being an algorithm that satisﬁes a validity property that is strictly stronger than Qualitative Validity.

Second case: For the second case, consider the following scenario: 4 1 Scenario 4. Consider Vin to be the multiset of input values that is identical to Vin except parties 1 in F get vf as an input and all other parties that are assigned w from Vin as an input, are assigned 4 4 instead some v 6= w (w 6∈ Vin). The adversary corrupts no parties (f = 0). Messages from parties in T are delayed until all parties in M decide, and messages sent among parties in M ∪ F immediately delivered. Note that Scenario 1 and Scenario 4 are probabilistically indistinguishable for parties in M (see illustration in Figure 3). Therefore, by Corollary 1, the probability of a party in M to agree on 4 4 w in Scenario 4 is higher than 0. Thus, since w 6∈ Vin, the probability to agree on a value in Vin in Scenario 4 is strictly lower than 1. Since f4 = 0 in Scenario 4, we get that the probability to

11 F F Byzantine. Act Honest. as if input � ! M input �! M input values in input values in ℳ T ℳ T Delayed. Input delayed v instead of w

(a) Scenario 1. Parties in F are Byzantine that act (b) Scenario 4. All parties are honest. Messages 1 4 as if they honest with input vf 6∈ Vin. Messages from parties in T are delayed. The value vf ∈ Vin, 4 from parties in T are delayed. whereas the value w 6∈ Vin.

Figure 3: Scenario 1 and Scenario 4 are probabilistically indistinguishable to all processes in M.

4 f4 agree on a value in Vin in Scenario 4 is strictly lower than 1 − n−t . A contradiction for A being an algorithm that satisﬁes a validity property that is strictly stronger than Qualitative Validity.

The bound is tight. We show in Appendix C that a sequential composition of two known algorithms, from [35] and [2], yields the first Agreement protocol that satisfies the Agreement, Probabilistic Termination, and Qualitative Validity properties. In a nutshell, we first run the algorithm in [35]. By its weak validity property, if all honest start with the same value, then all honest decide on it. Otherwise, they decide on ⊥, in which case we invoke the algorithm in [2] and output its decision value. The resulting asynchronous algorithm achieves O(n2) communication complexity, resilience against t < n/3, and operates in expected constant number of rounds. This is known to be optimal in this setting [20, 2, 9].

6 Discussion

Motivated by the need to characterize the quality of decisions that is possible in an asynchronous setting with malicious participants, we developed the framework of scenarios and probabilistic indistinguishability. Using it, we were able to formally establish tight bounds for Byzantine agreement. The new tools are not limited to the above problem, however. Indeed, we used it to give a more rigorous formal proof of the lower bound of [1] on the expected total message complexity in Byzantine Broadcast. Our framework is likely to be appropriate for the analysis of a variety of randomized distributed algorithms. For example, we believe that probabilistic indistinguishability can be used to yield an alternative formulation of the lower-bound arguments in [4, 5]. In a broader sense, we anticipate that probabilistic indistinguishability will allow to mitigate the diﬃculty of proving lower bounds and impossibility results in randomized distributed computing by adapting known techniques that rely on deterministic indistinguishability. Finally, we expect that formulating arguments and proofs using the new framework will facilitate greater transparency and understanding in the analysis of randomized distributed algorithms.

12 References

[1] Ittai Abraham, TH Hubert Chan, Danny Dolev, Kartik Nayak, Rafael Pass, Ling Ren, and Elaine Shi. Communication complexity of byzantine agreement, revisited. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, pages 317–326, 2019. [2] Ittai Abraham, Dahlia Malkhi, and Alexander Spiegelman. Asymptotically optimal validated asynchronous byzantine agreement. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, PODC ’19, New York, NY, USA, 2019. ACM. [3] James Aspnes. Lower bounds for distributed coin-flipping and randomized consensus. Journal of the ACM (JACM), 45(3):415–450, 1998. [4] Hagit Attiya and Keren Censor. Tight bounds for asynchronous randomized consensus. Journal of the ACM (JACM), 55(5):1–26, 2008. [5] Hagit Attiya and Keren Censor-Hillel. Lower bounds for randomized consensus under a weak adversary. SIAM Journal on Computing, 39(8):3885–3904, 2010. [6] Hagit Attiya and Faith Ellen. Impossibility results for distributed computing. Synthesis Lec- tures on Distributed Computing Theory, 5(1):1–162, 2014. [7] Hagit Attiya and Jennifer Welch. Distributed computing: fundamentals, simulations, and advanced topics, volume 19. John Wiley & Sons, 2004. [8] Michael Ben-Or. Another advantage of free choice (extended abstract) completely asynchronous agreement protocols. In Proceedings of the second annual ACM symposium on Prin- ciples of distributed computing, pages 27–30, 1983. [9] Gabriel Bracha. Asynchronous byzantine agreement protocols. Information and Computation, 75(2):130–143, 1987. [10] Christian Cachin et al. Architecture of the hyperledger blockchain fabric. In Workshop on distributed cryptocurrencies and consensus ledgers, volume 310, 2016. [11] Christian Cachin, Klaus Kursawe, Frank Petzold, and Victor Shoup. Secure and efficient asynchronous broadcast protocols. In Advances in Cryptology, 2001. [12] Ran Canetti and Tal Rabin. Fast asynchronous byzantine agreement with optimal resilience. In Proceedings of the twenty-fifth annual ACM symposium on Theory of computing, pages 42–51, 1993. [13] Miguel Correia, Nuno Ferreira Neves, and Paulo Ver´ıssimo.From consensus to atomic broadcast: Time-free byzantine-resistant protocols without signatures. The Computer Journal, 49(1):82–96, 2006. [14] Danny Dolev and Rudiger Reischuk. Bounds on information exchange for byzantine agreement. JACM, 1985. [15] Faith Ellen, Rati Gelashvili, and Leqi Zhu. Revisionist simulations: a new approach to proving space lower bounds. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing, pages 61–70, 2018.

13 [16] Ronald Fagin, Yoram Moses, Joseph Y Halpern, and Moshe Y Vardi. Reasoning about knowledge. MIT press, 2003. [17] Pesech Feldman and Silvio Micali. An optimal probabilistic protocol for synchronous byzantine agreement. SIAM Journal on Computing, 26(4):873–933, 1997.

[18] Faith Fich, Maurice Herlihy, and Nir Shavit. On the space complexity of randomized synchro- nization. Journal of the ACM (JACM), 45(5):843–862, 1998. [19] Faith Fich and Eric Ruppert. Hundreds of impossibility results for distributed computing. Distributed computing, 16(2-3):121–163, 2003. [20] Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. Impossibility of distributed consensus with one faulty process. JACM, 1985. [21] Oded Goldreich. Foundations of cryptography: volume 2, basic applications. Cambridge uni- versity press, 2009. [22] Oded Goldreich, Silvio Micali, and Avi Wigderson. How to play any mental game or A completeness theorem for protocols with honest majority. In Alfred V. Aho, editor, Proceedings of the 19th Annual ACM Symposium on Theory of Computing, 1987, New York, New York, USA, pages 218–229. ACM, 1987. [23] Shaﬁ Goldwasser, Silvio Micali, and Charles Rackoﬀ. The knowledge complexity of interactive proof systems. SIAM Journal on computing, 18(1):186–208, 1989.

[24] Jonathan Katz and Chiu-Yuen Koo. On expected constant-round protocols for byzantine agreement. In Annual International Cryptology Conference, pages 445–462. Springer, 2006. [25] Valerie King and Jared Saia. Byzantine agreement in expected polynomial time. Journal of the ACM (JACM), 63(2):13, 2016.

[26] Eyal Kushilevitz, Yishay Mansour, Michael O Rabin, and David Zuckerman. Lower bounds for randomized mutual exclusion. In Proceedings of the twenty-ﬁfth annual ACM symposium on Theory of computing, pages 154–163, 1993. [27] Nancy Lynch. A hundred impossibility proofs for distributed computing. In Proceedings of the eighth annual ACM Symposium on Principles of distributed computing, pages 1–28, 1989.

[28] Nancy A Lynch. Distributed algorithms. Elsevier, 1996. [29] Dahlia Malkhi. Concurrency: The Works of Leslie Lamport. Morgan & Claypool, 2019. [30] Gil Neiger. Distributed consensus revisited. Information processing letters, 49(4):195–201, 1994.

[31] M. Pease, R. Shostak, and L. Lamport. Reaching agreement in the presence of faults. J. ACM, 27(2), April 1980. [32] Marshall Pease, Robert Shostak, and Leslie Lamport. Reaching agreement in the presence of faults. Journal of the ACM (JACM), 27(2):228–234, 1980.

14 [33] Michael O Rabin. Probabilistic algorithms in finite fields. SIAM Journal on computing, 9(2):273–280, 1980. [34] Michael O Rabin. Randomized byzantine generals. In 24th Annual Symposium on Foundations of Computer Science (sfcs 1983), pages 403–409. IEEE, 1983. [35] Michel Raynal and Achour Mostéfaoui. Signature-free asynchronous byzantine systems: from multivalued to binary consensus with t < n/3, o(n2) messages, and constant time. Acta Informatica, 54(5), 2017. [36] VMware. Concord: An open source decentralized trust infrastructure. https://blogs. vmware.com/opensource/2018/08/28/meet-project-concord/, 2018. [37] Andrew Chi-Chih Yao. How to generate and exchange secrets. In 27th Annual Symposium on Foundations of Computer Science (sfcs 1986), pages 162–167. IEEE, 1986.

A Pseudo-code for Construction of a local experience

We construct scni in two reiterating steps: (1) We connect a node directly to its closest descendants in which i’s local state does not change (and remove all nodes in between). (2) For each node – starting from the root and going down the tree – we combine sons that represent the same local state by merging their subtrees and assigning its root (the combined node) the sum of the probabilities. From a complexity perspective this equals to two BFS runs.

Algorithm 1 gather data from the probabilistic scenario (v.GDFPS)

1: input: li, set of nodes in scn Startl with weights pv∀v ∈ Startl P i i 2: pli ← pv v∈Startli

3: for each v ∈ Startli do: 4: for each u ∈ Child(v) 5: propagationQueue.enqueue(hu, pv · weight(v, u)i) 6: while propagationQueue not empty do 7: hu, pui ← propagationQueue.dequeue() 8: if u.configuration[i] = li then 9: for each w ∈ Child(u) 10: propagationQueue.enqueue( hw, pu · weight(u, w)i ) 11: else(u.configuration[i] 6= li) 12: SETofCHILDEREN.insert( u.configuration[i]) 13: if the set Startu.configuration[i] does not exist then create it 14: Startu.configuration[i].insert( hu, pui )

15: 1 2 3 vli .CLES(li, pli , SETofCHILDEREN, Startli , Startli , Startli , ... )

B Proving Theorem 4 from [1] (ACDNPLS)

In (binary) Byzantine Broadcast, an a priori ﬁxed designated sender starts out with an input bit b ∈ {0, 1}. An algorithm A solves binary Byzantine Broadcast with probability at least x if, in

15 Algorithm 2 create local experience structure (v.CLES)

1: 1 2 3 input: li, pli SETofCHILDEREN, Startli , Startli , Startli , ...

2: label node v with hli, pli i 0 0 3: for each li ∈SETofCHILDEREN create node vli and add it to Child(vli ) 0 4: for each node vli ∈ Child(vli ) do: 0 0 0 5: vli .GDFPS( li , Startli ) every scenario of A, with the probability of at least x all of the three following properties hold.

• Agreement. All honest parties that output a bit, output the same bit. • Termination. Every honest party outputs a bit.

• Validity. If the sender is honest and the sender’s input is b, then all honest parties output b. The following theorem from [1] considers a model with a non-uniform p.p.t. strongly adaptive adversary. That is, the adversary can observe that a message is sent at time T by any party pi to any other party, decide to adaptively corrupt pi, and remove the messages sent by pi at time T. In line with our adaptive adversary’s definition, parties are corrupted on-the-fly and the coin flip results are visible to the adversary once they have occurred.

3 Theorem 2 (ACDNPLS [1]). If a protocol solves Byzantine Broadcast with 4 + probability against a strongly adaptive adversary, then in expectation, honest nodes collectively need to send at least (t)2 messages.

Using our deﬁnitions from this paper, we can now better formalize their statement as follows.

Theorem 20. If an algorithm A solves (in a model with a strongly adaptive adversary) Byzantine 3 Broadcast with probability at least 4 + , then there exists a scenario of A in which honest parties collectively send at least (t)2 messages in expectation.

3 Proof. Let A be an algorithm that solves Byzantine Broadcast with probability at least 4 + . Assume by way of contradiction that in every scenario of A, the honest parties collectively send fewer than (t)2 messages in expectation. Without loss of generality, assume that there exist dn/2e parties each of which outputs 0 with probability at most 1/2 if they receive no messages. (Otherwise, then there must exist dn/2e nodes that output 1 with at most 1/2 probability if they receive no messages, and the entire proof follows from a symmetric argument.) Formally, the set N0 ⊂ Π is of size dn/2e, and for every pi ∈ N0 it holds that in every scenario scn of A, Pscn[pi outputs 0 | pi receives no messages in the path] ≤ 1/2. Let F ⊂ N0 be a set of t/2 these parties, not containing the designated sender. Note that these nodes may output 1 or they may simply not terminate if they receive no messages. (We can always ﬁnd such an F because t/2 < dn/2e.) Consider the following scenario scn-A.

Scenario A. The sender’s input bit is 0, all messages that are sent, are synchronously delivered, and the parties in F are corrupted. Speciﬁcally, a party in F behaves honestly (according to its protocol) except for (a) not sending messages to any other party in F, and (b) ignoring the ﬁrst t/2 messages it receives from parties in Π \ F .

16 Let zA be a random variable, in scn-A, denoting the number of messages sent by parties in 2 2 (t) Π \ F to F . By the assumption we have that E[zA] < (t) . Let X1 be the event that zA ≤ 2 . 1 By Markov’s inequality, Pscn-A[zA > 2 E[zA]] < 2. Thus, the probability of the event X1 in scn-A t2 1 is Pscn-A[zA ≤ 2 ] ≥ Pscn-A[zA ≤ 2 E[zA]] > 1 − 2. t2 Let pf be the party in F with the highest probability of receiving at most t/2 from the ﬁrst 2 messages sent by honest parties in scn-A. Formally, denote by xi the random variable, in scn-A, t2 corresponding to the number of messages received by pi out of the ﬁrst 2 messages sent by honest parties. Then pf , arg max Pscn-A[xi ≤ t/2]. Notice that in each path in scn-A there are a total of pi∈F t2 at most 2 messages to distribute among the t/2 parties in F . Therefore, fewer than t parties in F receive more than t/2 messages, and at least |F | − t = (1/2 − )t parties receive at most t/2 of the mentioned messages. This implies that the expected probability of a party in F to receive at |F |−t most t/2 of the messages is at least |F | = 1 − 2. Let Xpf be the event in scn-A that xf ≤ t/2.

By choosing pf in the way we did, we obtain that Pscn-A[Xpf ] ≥ 1 − 2. (t)2 In scn-A, the probability that at most 2 messages from honest parties are sent to F , whilst party pf receives at most t/2 of those messages is:

Pscn-A[X1 ∩ Xpf ] = Pscn-A[X1] + Pscn-A[Xpf ] − Pscn-A[X1 ∪ Xpf ] > (1 − 2) + (1 − 2) − 1 = 1 − 4. Now consider the following scenario scn-B, which is very similar to scn-A.

Scenario B. The sender’s input bit is 0, and parties in F \{pf } are corrupted and behave exactly as in scn-A. In addition, parties in Π\F behave as in scn-A (according to the algorithm A) except that the ﬁrst t/2 messages that are supposed to be sent by Π \ F to pf are now omitted from scn-B. In order to do so, at most t/2 parties in Π\F are also corrupted (in an adaptive manner) in scn-B. Other than this, the corrupted parties in Π \ F behave exactly as in scn-A (including sending later messages to pf ).

We observe that for a party pi ∈ Π \ F that is honest in scn-B, the scenarios scn-A and scn- B are probabilistically indistinguishable. Therefore, by Corollary 1, their protocols prescribe the same (possibly probabilistic) behavior in both of the scenarios. The fact that all honest parties act according to A, together with the fact that the maximal number of corrupted parties in scn-B at most |F | − 1 + t/2 = t − 1 ≤ t, mean that scn-B is a scenario of A. By construction, pf receives no messages in scn-B with the same probability as Pscn-A[X1 ∩

Xpf ] = 1 − 4. Recall that (by the deﬁnition of F ), party pf outputs 0 with probability at most 1/2 on the set of paths in which it receives no messages whatsoever. Let Yf be the complementary event, in scn-B, to pf outputting 0. That is, Yf is the event that pf outputs 1 or does not terminate 1 at all. Then Pscn-B[Yf ] > 2 (1 − 4). Moreover, since scn-A and scn-B are probabilistically indistinguishable to honest parties in Π \ F , by Corollary 1, the probability of these parties to output 0 is equal in scn-A and scn-B, 3 and is at least 4 + by assumption. Let Y0 be the event in scn-B that all honest parties in Π \ F output 0. If both Y0 and Yf occur, then this path (execution) in scn-B violates either Agreement of Termination. We get 3 1 1 P [Y ∩ Y ] = P [Y ] + P [Y ] − P [Y ∪ Y ] > ( + ) + (1 − 4) − 1 = − . scn-B 0 f scn-B 0 scn-B f scn-B 0 f 4 2 4

17 3 This contradicts the assumption that A solves Byzantine Broadcast with probability at least 4 + .

This proof followed the outline of the original one and made use of the same underlying structure. But the use of scenarios and probabilistic indistinguishability provided the arguments with a rigorous foundation and improved transparency.

C Consensus with Qualitative Validity is Solvable

In this section we show that a sequential composition of two known algorithms, from [35] and [2], yields an Agreement protocol that satisﬁes the Agreement, Probabilistic Termination, and Quali- tative Validity properties. We next overview the properties guaranteed by each of the algorithms, then show how to combine them to achieve Qualitative Validity, and ﬁnally prove correctness and analyse complexity.

Overview of the RM protocol’s properties. The asynchronous Agreement protocol proposed by Raynal and Mostefaoui (RM) [35] satisfies Agreement, Probabilistic termination, Weak validity, and Non-intrusion against an adaptive adversary. The Weak validity property is a variant of the first part of the Qualitative Validity. That is, if all input values are the same (max mult(Vin) = 3t + 1), then honest parties can only decide on this value. However, if parties start with different values (max mult(Vin) < 3t + 1), then they are allowed to agree on a pre-defined ⊥ value. The Non- intrusion property requires that honest parties decides on values in Vin ∪ {⊥}. That is, honest parties never decide on a value promoted by the adversary. The complexity of RM [35] is the following: the protocol (1) tolerates up to t < n/3 Byzantine parties, (2) runs in expected constant number of rounds, and (3) sends O(n2) words in O(n2) messages where a word contains a constant number of signatures and values.

Overview of the AMS protocol’s properties. The asynchronous agreement protocol proposed by Avraham, Malkhi, and Spiegelman (AMS) [2] satisﬁes the Agreement, Probabilistic termination, External validity, and Quality properties against an adaptive adversary. The External validity property requires that honest parties decide on values that are valid by some external predicate. The Quality property requires that the probability that the decision value is in Vin is at least 1/2. The complexity of AMS is similar to that of RM.

Sequential composition We show that a sequential composition As of the RM and AMS protocols satisfies Agreement, Probabilistic termination, and Qualitative Validity with an optimal resilience and complexity in an asynchronous setting with an adaptive adversary. The pseudocode of As appears as Algorithm 3. First, parties try to reach agreement via the RM protocol and if its decision value is not ⊥, then the parties decide on this value. Otherwise, they propose their input value in the AMS protocol and decide on its decision value. Although the Weak validity property that is proved for RM in [35] does not imply the first part of Qualitative Validity, a small modification of the proof proves that the protocol indeed satisfies it. Moreover, the non-intrusion property guarantees that even if max mult(Vin) − f < 2t + 1 then, in scenarios s of A with f maximal corrupted parties and an input multiset Vin, parties never decide in line 3 of Algorithm 3 on a value that is not in Vin. To prove that the protocol in Algorithm 3 satisfies

18 qualitative validity, we need to show that the AMS protocol satisﬁes the second part of Qualitative Validity. That is, in every scenario of As with f maximal corrupted parties and an input multiset f Vin, the probability to decide on a value in Vin is at least 1 − n−t . By the quality property of AMS we get a probability of at least 1/2 in the worst case when f = t. Below we overview the main part of the AMS algorithm and prove that the protocol indeed satisﬁes a stronger property, i.e., the second part of Qualitative Validity.

Algorithm 3 An agreement algorithm with Qualitative validity: protocol for party pi

1: yi ←RM-propose(vi) 2: if yi 6= ⊥ then 3: decide yi 4: else 5: decide AMS-propose(vi)

C.1 Analysis The Agreement and Probabilistic-termination properties of the composition follows immediately from the ones in RM and AMS since yi 6= ⊥ in line 2 is true or false for all correct processes categorically. As for Qualitative Validity, it follows from the weak validity proof in RM that if max mult(Vin) − f ≥ 2t + 1, then all honest parties that decide, decide on values in Vin. Moreover, by the non-intrusion property of RM, honest parties can only decide in line 3 on a value in Vin. Thus, to prove Qualitative Validity we need to show that AMS satisfies that the probability to f decide on a value in Vin is at least 1 − n−t . Below we overview the relevant parts of AMS and then prove that it indeed satisfies the required property. In a nutshell, in every round of the AMS protocol, parties concurrently promote their values via some broadcast algorithm until at least n − t broadcasts complete. Then, using a shared global coin, parties elect one broadcast instance uniformly at random and ignore the rest. If a completed broadcast is elected, then its value is fixed and parties will eventually decide on this value. Otherwise, parties continue to the next round with either their value from the previous round or with the value of the elected broadcast in this round. Given the above description we prove the following lemma: Lemma 3. The probability to decide on a value proposed by an honest party in the AMS protocol f is at least 1 − n−t . Proof. We prove the lemma by showing that the probability to decide on a value proposed only f by Byzantine parties is at most n−t . To bound this probability from above, we assume that if a broadcast by a Byzantine sender is elected in some round r, then all honest parties decide on its value in round r. That is, we assume that all Byzantine parties complete their broadcasts in all rounds before the parties randomly elect one broadcast instance. By the AMS protocol, for every round r, if a not completed broadcast is elected, then honest parties continue to the next round with either their values from the previous round or with the value of the elected broadcast. Therefore, it follows by induction that in all rounds until a broadcast with a Byzantine sender is elected honest parties broadcast values proposed by honest parties. Denote by x1, x2, x3, ... the number of completed broadcasts in rounds r1, r2, r3, ..., respectively. xi Thus, for every round ri, n is the probability to elect a completed broadcast and fix its decision

19 n−xi value, whereas n is the probability to elect an uncompleted broadcast and continue to the next round without ﬁxing a decision. Let Q be the probability to decide on a value proposed only by Byzantine parties. We get that: x f n − x x f n − x x f n − x x f n − x Q = 1 + 1 ( 2 + 2 ( 3 + 3 ( 4 + 4 (...)))) n x1 n n x2 n n x3 n n x4 n

Let x = min({xi | i ∈ N}), we get that x f n − x x f n − x x f n − x x f n − x Q ≤ + ( + ( + ( + (...)))) = n x n n x n n x n n x n

∞ f n − x n − x f Y n − x f n f (1 + + ( )2 ...) = ( ( )i) = = n n n n n n n − (n − x) x i=0 f By the protocol, xi ≥ n − t for every round ri. Therefore, Q ≤ n−t .

Complexity. Since the sequential composition uses one instance of RM and one of AMS, its asymptotic complexity and resilience are equal to that of RM and AMS, which are proven to be optimal [20, 2, 9]. That is, it (1) tolerates up to t < n/3 Byzantine parties, (2) runs in expected constant number of rounds, and (3) sends O(n2) words in O(n2) messages where a word contains a constant number of signatures and values.