Scaling Distributed Ledgers and Privacy-Preserving Applications Alberto Sonnino A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy of University College London. arXiv:2102.12273v1 [cs.CR] 24 Feb 2021 Department of Computer Science University College London February 25, 2021 2 I, Alberto Sonnino, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the work. Abstract This thesis proposes techniques aiming to make blockchain technologies and smart contract platforms practical by improving their scalability, latency, and privacy. This thesis starts by presenting the design and implementation of Chainspace, a distributed ledger that supports user defined smart contracts and execute user-supplied transac- tions on their objects. The correct execution of smart contract transactions is publicly verifiable. Chainspace is scalable by sharding state; it is secure against subsets of nodes trying to compromise its integrity or availability properties through Byzantine Fault Tolerance (BFT). This thesis also introduces a family of replay attacks against sharded distributed ledgers targeting cross-shard consensus protocols; they allow an attacker, with network access only, to double-spend resources with minimal efforts. We then build Byzcuit, a new cross-shard consensus protocol that is immune to those attacks and that is tailored to run at the heart of Chainspace. Next, we propose FastPay, a high-integrity settlement system for pre-funded payments that can be used as a financial side-infrastructure for Chainspace to support low-latency retail payments. This settlement system is based on Byzantine Consistent Broadcast as its core primitive, foregoing the expenses of full atomic commit channels (consen- sus). The resulting system has extremely low-latency for both confirmation and payment finality. Finally, this thesis proposes Coconut, a selective disclosure creden- tial scheme supporting distributed threshold issuance, public and private attributes, re-randomization, and multiple unlinkable selective attribute revelations. It ensures authenticity and availability even when a subset of credential issuing authorities are malicious or offline, and natively integrates with Chainspace to enable a number of scalable privacy-preserving applications. Impact Statement The work in this thesis can inform the design of new and existing projects that implement distributed ledgers, smart contract platforms or applications, in order to increase their scalability, security, and privacy. Chainspace (Chapter 3) and Co- conut (Chapter 6) are used as part of DECODE (DEcentralized Citizen-owned Data Ecosystems) [1], a European project with a digital democracy pilot in Barcelona implementing a decentralized petitions platform. Chainspace was also commer- cialized in a company (chainspace.io) co-founded by the author of this thesis, and the team was acquired by Facebook. Coconut was also integrated into Cos- mos SDK [2] and commercialized in a company (Nym [3]) aiming to provide an open-ended anonymous overlay network that disguise patterns in internet traffic; it uses the scheme described in Chapter 6 as anonymous authentication credentials to enable privacy-enhanced data transfer and decentralized identity. Byzcuit and the work done on replay attacks against sharded distributed ledgers (Chapter 4) have profound impact on the security of recently proposed systems such as Omniledger [4] and RapidChain [5]; these systems were presented at top security conferences and form the basis of numerous start-ups and open-source projects such as Harmony [6]. Finally, the content of this thesis has been presented on multiple occasions, both at academic venues and industry conferences; some of its chapters have been published at top-tier security conferences. It produced multiple tools and open-source software, and is freely available online. Acknowledgements This work would not have been possible without my primary supervisor George Danezis, who helped me throughout the past few years of research, and to whom I wish to express my profound gratitude. Special thanks to my secondary super- visors Jens Groth and Ioannis Psaras for their unwavering support and generous encouragement, and to my close collaborators, Mustafa Al-Bassam and Shehar Bano, who have been the source of many fruitful discussions. I have been privileged to have had the opportunity to work with many brilliant and helpful people around the world. Specifically, I thank all my co-authors (in alphabetic order) Christos Andrikos, Sarah Azouvi, Lejla Batina, Mathieu Baudet, Vitalik Buterin, Avery Ching, Lukasz Chmielewski, Andrey Chursin, Dave Hrycyszyn, Ismail Khoffi, Michał Krol,´ Liran Lerman, Zekun Li, Dahlia Malkhi, Vasilis Mavroudis, Sarah Meiklejohn, Patrick McCorry, Kostas Papagiannopoulos, Dmitri Perelman, Guilherme Perin, Giorgos Rassias, Etienne Riviere,` Argyrios Tasiopoulos, Lixia Zhang, and Zhiyi Zhang. I would also like to thank Ramsey Khoury, my other co-founder at chainspace.io not mentioned above, and all my other former colleagues at chainspace.io: Penny Andrews, Andy Bennett, Stuart Chinery, Jer´ emy´ Letang, and Lola Oyelayo-Pearson. I also thank everyone at Facebook Novi, specifically David Marcus, James Evering- ham, Christian Catalini, Kevin Weil, Ben Maurer, Morgan Beller, and Evan Cheng for providing a supportive working environment. Finally, I would like to thank the European Commission for funding my research with a PhD scholarship, and all my friends and family for their continuous support and encouragement. Contents 1 Introduction 14 1.1 Problem Statement . 14 1.2 Overview . 16 1.3 Dissertation Organization and Contributions . 18 1.4 Additional Work . 21 1.5 Work Done in Collaboration . 22 2 Background and Related Work 24 2.1 Terminology and Assumptions . 24 2.2 Consensus in the Age of Blockchains . 27 2.2.1 Classical Consensus . 28 2.2.2 Elected Leader Consensus . 31 2.2.3 Hybrid Consensus: Single Committee . 34 2.2.4 Hybrid Consensus: Multiple Committees . 37 2.3 Cross-Shard Consensus Protocols . 39 2.3.1 Two-Phases Atomic Commit Protocols . 39 2.3.2 Mutex-Based Consensus Protocols . 40 2.4 Sybil Resistance and Committee Management . 41 2.4.1 Sybil Resistance . 41 2.4.2 Committee Reconfiguration . 44 2.5 Selective Disclosure Credentials . 45 2.5.1 Cryptographic Building Blocks . 46 2.5.2 The Predecessors of Coconut . 47 Contents 7 2.6 Blockchains Data Structures . 49 2.7 Chapter Summary . 50 3 Chainspace: A Sharded Smart Contracts Platform 51 3.1 Overview . 53 3.1.1 Data Model: Objects, Contracts, Transactions. 53 3.1.2 System Design, Threat Model and Security Properties . 56 3.2 The Chainspace Application Interface . 58 3.3 The Chainspace System Design . 62 3.3.1 High-Integrity Data Structures . 62 3.3.2 Distributed Architecture & Consensus . 65 3.3.3 Cross-Shard Consensus Protocol . 67 3.4 Security and Correctness . 68 3.4.1 Auditability . 68 3.5 System and Applications Smart Contracts . 70 3.5.1 System Contracts . 70 3.5.2 Application Level Smart Contracts . 72 3.6 Smart Contract Evaluation . 74 3.7 Limitations . 77 3.8 Comparison with Related Work . 78 3.9 Chapter Summary . 79 4 Replay Attacks and Defenses Against Cross-shard Consensus 80 4.1 Attack Overview . 83 4.2 Shard-led Cross-Shard Consensus Protocol . 85 4.2.1 S-BAC Overview . 85 4.2.2 Message Recording . 87 4.2.3 Attacks on the First Phase of S-BAC . 88 4.2.4 Attacks on the Second Phase of S-BAC . 89 4.2.5 Real-world Impact . 91 4.3 Client-led Cross-shard Consensus Protocol . 92 Contents 8 4.3.1 Atomix Overview . 92 4.3.2 Message Recording . 93 4.3.3 Attacks on the First Phase of Atomix . 94 4.3.4 Attacks on the Second Phase of Atomix . 95 4.3.5 Real-world Impact . 97 4.4 Eliciting Messages to Replay . 98 4.4.1 Shard-led Cross-Shard Consensus . 98 4.4.2 Client-led Cross-Shard Consensus . 99 4.5 Defenses Against Replay Attacks . 100 4.6 The Byzcuit Atomic Commit Protocol . 100 4.6.1 Byzcuit Protocol Design . 101 4.6.2 Security against Replay Attacks . 105 4.6.3 Byzcuit Security & Correctness . 108 4.7 Implementation & Evaluation . 110 4.8 Chapter Summary . 113 5 FastPay: High-Performance Byzantine Fault Tolerant Settlement 114 5.1 Background . 117 5.2 Overview . 118 5.2.1 Participants . 118 5.2.2 Accounts & Actions . 119 5.2.3 Protocol Messages . 119 5.2.4 Security Properties & Threat Model . 120 5.3 The FastPay Protocol . 121 5.3.1 Transferring Funds within FastPay . 123 5.3.2 Sharding authorities . 127 5.3.3 Interfacing with the Primary . 128 5.3.4 State Recovery & Auditing . 130 5.3.5 Correct Users & Client Implementation . 131 5.4 Security Analysis . 131 5.4.1 Safety . 132 Contents 9 5.4.2 Liveness . 135 5.4.3 Performance under Byzantine Failures . 137 5.4.4 Worst-Case Efficiency of FastPay Clients . 138 5.5 Implementation . 138 5.6 Evaluation . 139 5.6.1 Microbenchmarks . 139 5.6.2 Throughput . 140 5.6.3 Latency . 144 5.7 Limitations & Future Work . 147 5.8 Comparison with Related Work . 149 5.9 Chapter Summary . 151 6 Coconut: Threshold Issuance Selective Disclosure Credentials 153 6.1 Overview . 157 6.2 The Coconut Construction . 159 6.2.1 Notations & Assumptions . 159 6.2.2 Scheme Definitions and Security Properties . 160 6.2.3 Foundations of Coconut . 161 6.2.4 The Coconut Threshold Credential Scheme . 162 6.2.5 Multi-Attribute Credentials . 168 6.3 Sketch of Security Proofs . 171 6.4 Implementation . 172 6.4.1 The Coconut Smart Contract Library . 173 6.4.2 Ethereum Smart Contract Library . 174 6.4.3 Deeper Blockchain Integration . 175 6.5 Applications . 176 6.5.1 Coin Tumbler . 177 6.5.2 Privacy-Preserving Petition . 180 6.5.3 Censorship-Resistant Distribution of Proxies .