Decentralised computer systems Mansoor Anwar Ahmed University of Cambridge Department of Computer Science and Technology Queens' College January 2021 This dissertation is submitted for the degree of Doctor of Philosophy Declaration This thesis is the result of my own work and includes nothing which is the outcome of work done in collaboration except as declared in the Preface and specified in the text. I further state that no substantial part of my thesis has already been submitted, or, is being concurrently submitted for any such degree, diploma or other qualification at the University of Cambridge or any other University or similar institution except as declared in the Preface and specified in the text. It does not exceed the prescribed word limit for the relevant Degree Committee. Decentralised computer systems Mansoor Anwar Ahmed Summary The architecture of the Web was designed to enable decentralised exchange of in- formation. Early architects envisioned an egalitarian yet organic society thriving in cyberspace. The reality of the Web today, unfortunately, does not bear out these vi- sions: information networks have repeatedly shown a tendency towards consolidation and centralisation with the current Web split between a handful of large corporations. The advent of Bitcoin and successor blockchain networks re-ignited interest in de- veloping alternatives to the centralised Web and paving a way back to the earlier archi- tectural visions for the Web. This has led to immense hype around these technologies with the cryptocurrency market valued at several hundred billions of dollars at the time of writing. With great hype, apparently, come great scams. I start off by analysing the use of Bitcoin as an enabler for crime and then present both technical solutions as well as policy recommendations to mitigate the harm these crimes cause. These policy recommendations then lead us on to look more closely at cryptocur- rency's tamer cousin: permissioned blockchains. These systems, while less revolutionary in their premise, nevertheless aim to provide sweeping improvements in the efficiency and transparency of existing enterprise systems. To see whether they work in practice, I present the results of my work in delivering a production permissioned blockchain sys- tem to real users. This involves comparing several permissioned blockchain systems, exploring their deficiencies and developing solutions for the most egregious of those. Lastly, I do a deep dive into one of the most persistent technical issues with permis- sioned blockchains, and decentralised networks in general: the lack of scalability in their consensus mechanisms. I present two novel consensus algorithms that aim to improve upon the state of the art in several ways. The first is designed to enable existing per- missioned blockchain networks to scale to thousands of nodes. The second presents an entirely new way of building decentralised consensus systems utilising a trie-based data structure at its core as opposed to the usual linear ledgers used in current systems. \The roots below the earth claim no rewards for making the branches fruitful." |Rabindranath Tagore, Stray Birds Acknowledgements A lot of people have directly and indirectly contributed to the completion of this thesis. I would like to start by thanking my supervisor, Ross Anderson, for giving me free reign over the research agenda and for pushing me to look beyond the traditional confines of computer science and see the broader context in which computer systems operate. I would also like to thank my examiners, Rainer B¨ohmeand Jon Crowcroft, for their helpful feedback and proactive engagement with this thesis. I would like to acknowledge nCipher Security and TODAQ for generously funding portions of my PhD. Thanks are due to Ilia Shumailov, my fellow PhD and frequent co-author, for helping me convert chats around the coffee table into full-fledged papers. I would also like to thank Alastair Beresford and the Digital Technology Group for supporting me through- out the PhD and for helping me find an academic community I could feel a part of. I am grateful to the founding engineering team at Jitsuin (Robin Bryce, Jon Geater, John Hartley and Waldemar Parzonka) for trusting a na¨ıve academic like me with building a production system and helping me understand real-world software engineering. I owe a lot to Dann Toliver who has not only been a co-author but also a mentor and a friend; our weekly calls were the highlight of my work over these three years. Outside of the lab, there are more people to thank than the word limit allows me to mention. Starting off with the fantastic people I met at Queens' college: I am fortunate to consider Andrea Wessendorf a friend; she has been a patient listener and a source of benevolent peer pressure throughout this thesis. Another gem of a person that I've had the pleasure of knowing is Jasper Ma who has the distinction of not just being the best man but also the \minister" at my wedding. Elizabeth Weir has helped more than she knows, sharing many a warm cup over many a dreary day. I would be remiss if I didn't mention Apolline Blandin, Dante McGrath, Francesca van Tartwijk, Joe Stallard and Jolly Dusabe who have, in their own way, brought me joy. Next on the list are my fellow writers from the Creative Writing Sessions. Rhian Holvey, Maria Bolson, Grace Rosemin, Matt Cooper, Emily Sandford, Sophia Cameron- Christie and the rest of the group have been a constant source of camaraderie and support. No matter how difficult a given week might have been, our writing sessions and meandering conversations never failed to bring back some light. My parents, Asma and Anwar Ahmed, have always pushed me towards academic excellence and deciding to pursue this thesis is no doubt a direct consequence of that. My sister, Sarah Ahmed, has always been the invisible support structure that kept me safe even when I didn't realise it. Lastly, words cannot express my gratitude to whatever vagaries of the universe led to me meeting my wife, Emma. She is my family, my inspiration and my home. This thesis is dedicated to her and my sister. Contents 1 Preface 11 1.1 Declaration . 12 1.2 Contributions . 13 1.2.1 Textbook . 13 1.2.2 Academic papers . 13 1.2.3 Patents . 14 1.2.4 Invited talks . 15 1.2.5 Outreach and open source . 15 2 Background 17 2.1 The Promised Land . 17 2.1.1 2020 hindsight . 19 2.1.2 Problems with centralisation . 20 2.1.2.1 Single point of failure . 21 2.1.2.2 Power dynamics . 23 2.2 A new hope . 24 2.2.1 Bitcoin primer . 24 2.2.2 Ethereum and altcoins . 25 2.3 Next steps for the blockchain . 26 2.3.1 Out of the trough . 27 2.3.2 My contributions . 27 3 Mitigating cryptocurrency-enabled crime 29 3.1 Bitcoin and crime . 29 3.2 What the law says . 31 3.3 Bitcoin tracing . 32 3.3.1 Bitcoin mixing . 33 3.3.2 TaintChain: practical FIFO tracing . 35 3.4 Finding patterns in the noise . 37 3.4.1 Preliminary model . 37 3.4.1.1 Limitations of the preliminary model . 38 3.4.2 Interactive visualisation . 39 5 3.4.3 Other visualisation tools . 41 3.4.4 Future work for taint visualisation . 42 3.5 Understanding the theft reporting ecosystem . 42 3.5.1 Incentives of the taint tracking ecosystem . 42 3.5.2 Theft reporting in practice . 44 3.6 How the market really works now . 44 3.6.1 Who owns the bitcoin stock anyway? . 45 3.6.2 Off-chain transactions . 47 3.6.3 The E-Money Directive . 48 3.6.4 Directive PE CONS 72/17 . 50 3.6.5 Positions of UK stakeholders . 51 3.7 Policy recommendations . 52 3.7.1 Regulated exchanges . 52 3.7.2 Consumer protection . 53 3.7.3 Unregistered exchanges . 53 3.7.4 Innovation and the role of central bank cryptocurrency . 55 3.7.5 Nature of ownership . 57 3.7.6 Dark market currencies . 58 3.7.7 Capital requirements . 60 3.7.8 Mitigating environmental harm . 60 3.8 Conclusion . 61 4 Taming the blockchain 65 4.1 Why permissioned? . 66 4.1.1 Issues with permissionless blockchains . 66 4.1.2 Permissioned blockchain primer . 68 4.1.3 What is the point? . 69 4.2 Comparison of permissioned blockchain frameworks . 69 4.2.1 Hyperledger Fabric . 70 4.2.2 Hyperledger Sawtooth . 71 4.2.3 Hyperledger Burrow . 72 4.2.4 Hyperledger Iroha . 72 4.2.5 Hyperledger Besu . 73 4.2.6 JP Morgan Quorum . 73 4.2.7 R3 Corda . 73 4.2.8 Other permissioned models . 74 4.3 Blockchain archival . 75 4.3.1 Regenesis overview . 76 4.3.2 Regenesis request . 77 4.3.3 Transition block . 77 4.3.4 Regenesis block . 78 6 4.3.5 Validation . 78 4.3.6 Dealing with state . 78 4.4 Modifiable storage . 79 4.4.1 Modifiable transactions overview . 80 4.4.2 Modification request . 80 4.4.3 Processing a modification transaction . 81 4.4.4 Validating a chain with modified transactions . 81 4.5 Conclusion . 82 5 Scaling blockchain consensus 83 5.1 Motivation . 84 5.1.1 Naive random selection . 84 5.1.2 Sophisticated random selection . 84 5.1.3 Selection using TEEs . 86 5.1.4 A deterministic approach to consensus . 87 5.2 Robust Round Robin overview . 88 5.2.1 Assumptions . 89 5.2.2 Identity creation . 89 5.2.3 Starting point: deterministic selection . 90 5.2.3.1 Security and liveness challenges .