Mast Cornellgrad 0058F 12280.Pdf (2.365Mb)
Total Page:16
File Type:pdf, Size:1020Kb
Protocols for Building Secure and Scalable Decentralized Applications A Dissertation Presented to the Faculty of the Graduate School of Cornell University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy by Kai Mast December 2020 © 2020 Kai Mast ALL RIGHTS RESERVED Protocols for Building Secure and Scalable Decentralized Applications Kai Mast, Ph.D. Cornell University 2020 Decentralized ledger technologies distribute data and execution across a public peer- to-peer network, which allows for more democratic governance of distributed systems and enables tolerating Byzantine failures. However, current protocols for such decen- tralized ledgers are limited in performance as they require every participant of the protocol to execute and validate every operation. Because of this, systems such as Bitcoin or Ethereum are limited in their throughput to around 10 transaction per sec- ond. Additionally, current implementations provide virtually no privacy to individual users, which precludes decentralized ledgers from being used in many real-world ap- plications. This thesis analyses the scalability and privacy limitations of current protocols and discusses means to improve them in detail. It then outlines two novel protocols for building decentralized ledgers, their implementation, and evaluates their performance under realistic workloads. First, it introduces the BitWeave, a blockchain protocol enabling parallel transac- tion validation and serialization while maintaining the same safety and liveness guaran- tees provided by Bitcoin. BitWeave partitions the system’s workload across multiple distinct shards, each of which then executes transactions mostly independently, while allowing for serializable cross-shard transactions. Second, it discusses DataPods, which is a database architecture and programming abstraction that combines the safety properties of decentralized systems with the scala- bility and confidentiality of centralized systems. Each data pod is akin to a conventional database instance with the addition of enabling users to detect and resolve misbehav- ior with the help of a global ledger. Further, data pods are interoperable with each other through federated transactions, enable confidentiality of data, and allow users to migrate their data in case of failure. BIOGRAPHICAL SKETCH Kai Mast was born in Böblingen, Germany. He received his Bachelor of Science degree from the University of Bamberg in 2014. While working towards his undergraduate degree, he researched general artificial intelligence with Dietrich Dörner and peer-to- peer networks with Udo R. Krieger. He wrote his undergraduate thesis on on-chip networks while visiting Intel Mobile Communications. His interest in peer-to-peer networks lead to him pursuing a Ph.D. at Cornell Uni- versity, working with Emin Gün Sirer. During his time at Cornell, his work revolved around secure database management systems and blockchain protocols. He spent the summer of 2016 at Microsoft Research Cambridge working with Anthony Rowstron and the summer of 2019 at U.C. Berkeley working with Dawn Song. For his minor studies in sociology, Kai Mast researched political alignment in social networks with Yongren Shi and Michael W. Macy. He received his Master of Science degree from Cornell in 2017 and his in doctorate in 2020. iii ACKNOWLEDGEMENTS There are many people without which I would not have been able to make this journey. During my undergraduate studies, Philipp Eittenberger and Dietrich Dörner were the first encouraging me to conduct research. Additionally, Udo Krieger and Todor Mladenov have been amazing mentors while at the University of Bamberg. My adviser Gün Sirer always helped me improve my research, coding, and technical writing skills. All members of the Cornell Community, and in particular the Cornell System Lab, have been great friends and colleagues. This includes, but is not limited to, Ayush, Deniz, Edward, Efe, Ethan, Florian, Harjasleen, Jack, Kevin, Matthew, Natacha, Robert, Shir, Soumya, Ted, Tom, Vera, Vlad, Xanda, and Yunhao. A particular highlight of my doctoral studies were the undergraduates I had the pleasure to work with: Lequn, Charles, Aaron, and Arzu. Internships have been an invaluable experience during, both, my undergraduate and graduate studies. I want to thank everybody at Elektrobit Automotive, Siemens Healthcare, Intel Mobile Communications, and Microsoft Research that I had the chance to interact with. Here, I want to especially thank Andreas Pokorny who helped me find and secure my first internship. Finally, I want to thank my family, my friends, my housemates, and, especially, my partner Sophie for all their support during the ups and downs of graduate school. iv TABLE OF CONTENTS 1 Introduction 1 1.1 Motivation .................................. 1 1.2 Decentralized Ledger Abstraction ..................... 3 1.2.1 Consistency ............................. 4 1.2.2 Immutability ............................ 5 1.2.3 Auditability ............................. 6 1.3 Decentralized Ledger Technologies .................... 6 1.3.1 Assumptions and Attack Model .................. 7 1.3.2 Sybil Detection ........................... 7 1.3.3 Committee-Based Consensus ................... 8 1.3.4 Nakamoto Consensus ....................... 9 1.3.5 Bottlenecks ............................. 11 1.4 Existing Approaches for Scaling Blockchains ............... 13 1.4.1 Off-Chain Protocols ........................ 13 1.4.2 Sharding Blockchains ....................... 13 1.5 Challenges in Sharding Blockchains .................... 14 1.5.1 Maintaining Safety ......................... 14 1.5.2 Ensuring Consistency ....................... 15 1.5.3 Maintaining Decentralization ................... 15 1.5.4 Providing Sound Incentive Mechanisms ............. 16 1.6 Thesis Contributions ............................ 16 1.6.1 BitWeave: Audit-based Sharding for Blockchains ....... 17 1.6.2 DataPods: Federated Decentralized Databases ......... 17 1.7 Thesis Outline ................................ 18 2 Abstractions for Scalable Decentralized Applications 19 2.1 Transaction Fees and Digital Payments .................. 19 2.2 Existing Data Models for Decentralized Ledgers ............. 19 2.2.1 The UTXO Model .......................... 20 2.2.2 The Accounts Model ........................ 22 2.3 Smart Contracts ............................... 24 2.3.1 Limitations of Smart Contracts .................. 26 2.4 Concurrent Decentralized Applications .................. 26 2.4.1 Objects and Object Types ..................... 27 2.4.2 Application Functions ....................... 28 2.4.3 Reservations ............................. 28 2.4.4 Implementation ........................... 30 2.5 Chapter Summary .............................. 30 v 3 BitWeave: Audit-based Sharding for Blockchains 31 3.1 Foundation: Bitcoin-NG ........................... 33 3.2 Blockchain Structure ............................ 33 3.3 Consensus Abstraction ........................... 35 3.4 Roles in BitWeave ............................. 36 3.4.1 Epoch Leaders ............................ 36 3.4.2 Shard Followers ........................... 36 3.4.3 Shard Commanders ......................... 37 3.5 Transaction Processing Overview ..................... 37 3.5.1 Reservations ............................. 38 3.5.2 Commits and Aborts ........................ 39 3.5.3 Efficient Cross-Shard Communication .............. 39 3.5.4 Transaction Fees and Miner Rewards ............... 40 3.6 Fault-Tolerant Transaction Processing ................... 42 3.6.1 Detecting Fraud ........................... 43 3.6.2 Fraud-Proofs ............................ 44 3.6.3 Incentivizing Fraud-Finding Behavior .............. 45 3.6.4 Ensuring Shard Availability .................... 46 3.6.5 Adaptive Confirmation Intervals ................. 48 3.7 Correctness ................................. 49 3.7.1 Safety ................................ 51 3.7.2 Liveness ............................... 53 3.8 Case Studies ................................. 54 3.8.1 Applying BitWeave to Ethereum ................. 54 3.8.2 Applying BitWeave to Bitcoin .................. 56 3.9 Implementation Details ........................... 57 3.9.1 Reducing Transaction Footprint .................. 58 3.9.2 Block Size and Epoch Length ................... 58 3.9.3 Congestion Control ......................... 59 3.10 Experimental Evaluation .......................... 60 3.10.1 How well does BitWeave’s overall throughput scale? ..... 63 3.10.2 How does sharding affect the transaction footprint? ...... 63 3.10.3 What is the overhead generated by cross-shard messages? ... 64 3.10.4 How well does the protocol handle failures? ........... 64 3.11 Discussion and Open Problems ....................... 65 3.11.1 Shortening CHALLENGE Periods .................. 65 3.11.2 Reducing Chain Size ........................ 66 3.11.3 Adapting to Changing Workloads ................. 67 3.12 Chapter Summary .............................. 68 4 DataPods: Federated Decentralized Databases 69 4.1 Foundation: Trusted Execution Environments .............. 70 4.2 The DataPods Architecture ........................ 72 4.2.1 Assumptions and Attack Model .................. 74 vi 4.2.2 Global Ledger Abstraction ..................... 74 4.2.3 Application Programming Interface ................ 75 4.2.4 Authenticated Private Storage ................... 77 4.2.5 Secure Function Evaluation .................... 81 4.2.6 Detecting Data Pod Failure .................... 81 4.2.7