On the Complexity of Synchronization by Rati Gelashvili B.S., Swiss Federal Institute of Technology (2012) S.M., Massachusetts Institute of Technology (2014) Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science and Engineering at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2017 ○c Massachusetts Institute of Technology 2017. All rights reserved. Author................................................................ Department of Electrical Engineering and Computer Science May 19, 2017 Certified by. Nir Shavit Professor of Electrical Engineering and Computer Science Thesis Supervisor Accepted by . Leslie A. Kolodziejski Professor of Electrical Engineering and Computer Science Chair, Department Committee on Graduate Students 2 On the Complexity of Synchronization by Rati Gelashvili Submitted to the Department of Electrical Engineering and Computer Science on May 19, 2017, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science and Engineering Abstract The field of distributed algorithms revolves around efficiently solving synchronization tasks, such as leader election and consensus. We make contributions towards a better understanding of the complexity of central tasks in standard distributed models. In the population protocols model, we demonstrate how to solve majority and leader election efficiently, in time O(log2 n), using O(log n) states per node, for n nodes. Central to our algorithms is a new leaderless phase clock technique. We also prove tight lower bounds on the state complexity of solving these tasks. In shared memory, we prove that any nondeterministic solo terminating consen- sus algorithm for anonymous processes has to use Ω(n) read-write registers. Then, we show how to solve n-process wait-free consensus by combining synchronization instructions that would be considered “weak” according to Herlihy’s consensus hier- archy. This collapses the hierarchy when instructions can be applied to the same memory location, as is the case in all existing multicore processors. We suggest an alternative hierarchy and provide a practical universal construction using only “weak” instructions, that performs as well as the Compare-and-Swap-based solution. Space complexity of solving k-set agreement is a problem that highlights important gaps in our understanding and state-of-the-art methods. No general lower bound better than 2 is known. We introduce a new technique based on an indirect black-box application of Sperner’s Lemma through an algorithmic reduction to the impossibility of wait-free k-set agreement. We design a simulation such that for any protocol Π either the simulating processes solve wait-free k-set agreement (impossible), or they simulate an execution of Π that uses many registers. Finally, time complexity of leader election is a long-standing open problem. We give an algorithm with O(log? k) time complexity in asynchronous message-passing system, for k participants. Thesis Supervisor: Nir Shavit Title: Professor of Electrical Engineering and Computer Science 3 4 Acknowledgments I would like to thank Professor Nir Shavit for supervision, an incredible support and for introducing me to the exciting field of concurrent algorithms. It is my privilege and pleasure to be a student of Nir who I respect and admire as a wonderful human being and a brilliant researcher and advisor, my true role model. In many ways interaction with Nir shaped me as a scientist, a friend and a world citizen. I want to thank Professor Faith Ellen - for thorough feedback, outstanding sup- port and encouragement. I am very much looking forward to the postdoc under her supervision. Special thanks to Professors James Aspnes and Aleksander Madry for their valu- able feedback and the very fact of serving on my thesis committee. This thesis is based on joints works with Dan Alistarh, Jim Aspnes, David Eisen- stat, Faith Ellen, Idit Keidar, Ron Rivest, Nir Shavit, Alexander (Sasha) Spiegelman, Adrian Vladu, Milan Vojnovic, Roger Wattenhofer, Leqi (Jimmy) Zhu. Razor-sharp discussions were part of these enjoyable intellectual journeys under which I learnt, grew, and matured as a scientist. I would like to specially thank Dan Alistarh for taking me under his wing as a young student, and Jimmy Zhu for the most productive, challenging and very pleasant collaborations. I would also like to thank Philipp Woelfel, Yoram Moses, Idit Keidar, Sasha Spiegelman, Faith Ellen, Jimmy Zhu, Dan Alistarh for hosting me as a visitor at various points during my PhD - you have been amazing hosts and inspiring presence! Family and friends from all around the world - without your support I would not have been where I am today! Friends in the Boston area with whom I spent past five years (including frosty winters). Among them I want to say special thanks to Guranda Darchidze, Ilya Razenshteyn, Shibani Santurkar, David Budden, Adrian Vladu, Mohsen Ghaffari, Jerry Li, Guatam 1 “G” Kamath, Mira Radeva, Mari Kobiashvili, Tornike Metreveli, Sasha Konstantinov, Achuta Kadambi, Merav Parter and Stephan Holzer. 1Gautam 5 A huge part of this experience has been internships. Thus, I would like to thank Milan Vojnovic, Dan Alistarh, David Amenhauser, Debabrata Banerjee, Bogdan Munteanu, Diwaker Gupta, Mirjam Wattenhofer and Kriti Puniyani for providing outstanding personal and work environment. I am very thankful to MIT Department of Computer Science and Electrical En- gineering for providing the funding for my education and research. Lastly, I want to mention Soso (Sensei) Dzmanashvili for unconditional support and extracurricular inspiration. 6 Contents 1 Introduction 13 1.1 On Population Protocols . 14 1.1.1 Leader Election and Majority . 16 1.1.2 Summary . 19 1.1.3 Chapter Outline . 19 1.2 On Shared Memory . 22 1.2.1 Complexity-Based Hierarchy . 22 1.2.2 Towards Reduced Instruction Sets for Synchronization . 24 1.2.3 Anonymous Space Lower Bound . 25 1.2.4 k-Set Agreement . 27 1.2.5 Chapter Outline . 30 1.3 On Message Passing . 32 2 Population Protocols 35 2.1 Model . 35 2.2 Leader-Minion Algorithm . 40 2.2.1 Analysis . 42 2.3 Leaderless Phase Clock . 48 2.3.1 Analysis . 49 2.4 Phased Majority Algorithm . 51 2.4.1 Analysis . 55 2.5 Synthetic Coin Flips . 66 2.5.1 Analysis . 66 7 2.5.2 Approximate Counting . 69 2.6 Phased Leader Election . 69 2.6.1 Analysis . 71 2.7 Lower Bounds . 72 2.7.1 Technical Tools . 72 2.7.2 Output-Dominant Majority . 78 2.7.3 General Lower Bound . 84 3 Shared Memory 97 3.1 Anonymous Space Lower Bound . 97 3.1.1 Definitions and Notation . 98 3.1.2 A Square-Root Lower Bound . 100 3.1.3 Linear Lower Bound . 104 3.2 The Space Hierarchy . 117 3.2.1 Model . 117 3.2.2 Arithmetic Instructions . 118 3.2.3 Increment . 121 3.2.4 Buffers . 123 3.2.5 Multiple Assignment . 132 3.3 Universality using “Weak” Instructions . 140 3.3.1 Algorithm . 142 3.4 Enter the Simulation: k-Set Agreement . 145 3.4.1 Model . 145 3.4.2 Local Argument . 146 3.4.3 Global Argument . 151 4 Message Passing 163 4.1 Definitions and Notation . 163 4.2 Leader Election Algorithm . 166 4.2.1 The PoisonPill Technique . 166 4.2.2 Heterogeneous PoisonPill . 167 8 4.2.3 Final construction . 170 5 Conclusions 179 9 10 List of Figures 1-1 Summary of results on Majority and Leader Election. 20 2-1 The state update rules for the LM algorithm. 41 2-2 Pseudocode for the phased majority algorithm, part 1/2 . 95 2-3 Pseudocode for the phased majority algorithm, part 2/2 . 96 3-1 Proof of Lemma 3.1.10, Case 1 . 112 3-2 Proof of Lemma 3.1.10, Case 2 . 114 3-3 Illustration of Case 2 in History object emulation . 126 3-4 Element of A. ................................ 142 3-5 Pseudocode for the Implementation of a 1-Augmented Snapshot Object 154 4-1 PoisonPill Technique . 166 4-2 Heterogeneous PoisonPill . 168 4-3 PreRound procedure . 171 4-4 Doorway procedure . 172 4-5 Leader election algorithm . 172 11 12 Chapter 1 Introduction To solve a problem in a distributed fashion, nodes peforming parts of the computa- tion need to synchronize with each other. The common synchronization requirements are often abstracted and captured by synchronization tasks. Examples of important synchronization tasks include consensus (agreement) [LSP82, PSL80], leader elec- tion (test-and-set) [AGTV92], majority [Tho79], mutual exclusion [Dij65], renam- ing [ABND+90], task allocation (do-all) [KS92] and timestamps [Lam78]. These tasks are usually considered in two classical models for distributed com- putation: asynchronous shared memory and asynchronous message-passing [Lyn96]. Additionally, population protocols [AAD+06] are a popular model of distributed com- puting, in which randomly-interacting agents with little computational power coop- erate to jointly perform complex computation. In all these models, randomization plays a critical role in solving synchronization tasks. In population protocols, interactions happen according to a randomized sched- uler. In the classical models, celebrated impossibility results of [FLP85, HS99] limit the power of deterministic distributed computation. Fortunately, relaxing the task specifications to allow for randomization [BO83] (and in particular, probabilistic ter- mination) has proved a very useful tool for circumventing fundamental impossibilities, and for obtaining efficient algorithms. In each model, there are two standard complexity measures. Time complexity, defined appropriately, can be formulated in all three models. In population protocols, 13 the other (more important) measure is state complexity of the agents. In asynchronous shared memory, space complexity denotes the number of shared memory locations that can be concurrently accessed by processors. Such locations have historically also been called registers. In asynchronous message-passing, the other natural measure is message complexity of an algorithm.