review articles DOI:10.1145/3363823 we do not have good tools to build ef- A 50-year history of concurrency. ficient, scalable, and reliable concur- rent systems. BY SERGIO RAJSBAUM AND MICHEL RAYNAL Concurrency was once a specialized discipline for experts, but today the chal- lenge is for the entire information tech- nology community because of two dis- ruptive phenomena: the development of Mastering networking communications, and the end of the ability to increase processors speed at an exponential rate. Increases in performance come through concur- Concurrent rency, as in multicore architectures. Concurrency is also critical to achieve fault-tolerant, distributed services, as in global databases, cloud computing, and Computing blockchain applications. Concurrent computing through sequen- tial thinking. Right from the start in the 1960s, the main way of dealing with con- through currency has been by reduction to se- quential reasoning. Transforming problems in the concurrent domain into simpler problems in the sequential Sequential domain, yields benefits for specifying, implementing, and verifying concur- rent programs. It is a two-sided strategy, together with a bridge connecting the Thinking two sides. First, a sequential specificationof an object (or service) that can be ac- key insights ˽ A main way of dealing with the enormous challenges of building concurrent systems is by reduction to sequential I must appeal to the patience of the wondering readers, thinking. Over more than 50 years, more sophisticated techniques have been suffering as I am from the sequential nature of human developed to build complex systems in communication. this way. 12 ˽ The strategy starts by designing —E.W. Dijkstra, 1968 sequential specifications, and then mechanisms to associate a concurrent execution to a sequential execution that itself can be tested against the ONE OF THE most daunting challenges in information sequential specification. science and technology has always been mastering ˽ The history starts with concrete, physical mutual exclusion techniques, and evolves concurrency. Concurrent programming is enormously up to today, with more abstract, scalable, and fault-tolerant ideas including difficult because it copes with many possible distributed ledgers. nondeterministic behaviors of tasks being done at ˽ We are at the limits of the approach, encountering performance limitations the same time. These come from different sources, by the requirement to satisfy a including failures, operating systems, shared memory sequential specification, and because not all concurrent problems have architectures, and asynchrony. Indeed, even today sequential specifications. 78 COMMUNICATIONS OF THE ACM | JANUARY 2020 | VOL. 63 | NO. 1 cessed concurrently states the desired efficient, scalable, and fault-tolerant between the executions of a concurrent behavior only in executions where the concurrent objects. Locks enforce ex- program and the sequential specifica- processes access the object one after clusive accesses to shared data, and tion. It enforces safety properties by the other, sequentially. Thus, famil- concurrency control protocols. More which concurrent executions appear as iar paradigms from sequential com- abstract and fault-tolerant solutions if the operations invoked on the object puting can be used to specify shared that include agreement protocols that where executed instantaneously, in objects, such as classical data struc- can be used for replicated data to lo- some sequential interleaving. This is tures (for example, queues, stacks, cally execute object operations in the captured by the notion of a consistency and lists), registers that can be read same order. Reliable communication condition, which defines the way con- or modified, or database transactions. protocols such as atomic broadcast and current invocations to the operations This makes it easy to understand the gossiping are used by the processes to of an object correspond to a sequential object being implemented, as op- communicate with each other. Distrib- interleaving, which can then be tested posed to a truly concurrent specifi- uted data structures, such as block- against the its sequential specification. cation which would be hard or un- chains. Commit protocols to ensure at- A brief history and some examples. natural. Instead of trying to modify omicity properties. Several techniques The history of concurrency is long and the well-understood notion of say, a are commonly useful, such as time- the body of research enormous; a few queue, we stay with the usual sequen- stamps, quorums, group membership milestones are in the sidebar “A Few tial specification, and move the mean- services, and failure detectors. Inter- Dates from the History of Synchroniza- ing of a concurrent implementation of esting liveness issues arise, specified tion.” The interested reader will find a queue to another level of the system. by progress conditions to guarantee many more results about principles The second part of the strategy is to that operations are actually executed. of concurrency in textbooks.4,21,35,37 We IMAGE FROM SHUTTERSTOCK.COM IMAGE provide implementation techniques for The bridge establishes a connection concentrate here only on a few signifi- JANUARY 2020 | VOL. 63 | NO. 1 | COMMUNICATIONS OF THE ACM 79 review articles foundational articles on concurrent pro- gramming appears in Brinch.6 As soon as the programs being run A Few Dates from the concurrently began to interact with each other, it was realized how difficult History of Synchronization it is to think concurrently. By the end of the 1960s a crisis was emerging: 1965 Mutual exclusion from atomic read/write registers Dijkstra11 programming was done without any 1965 Semaphores Dijkstra13 conceptual foundation and programs Mutual exclusion from non-atomic 1971 Lamport23 were riddled with subtle errors caus- read/write registers ing erratic behaviors. In 1965, Dijks- 1974 Concurrent reading and writing Lamport,24 Peterson33 1977, 1983 Distributed state machine (DA 2000) Lamport25 tra discovered that mutual exclusion Byzantine failures in synchronous systems of parts of code is a fundamental con- 1980 Pease, Shostak, Lamport31 (DA 2005) cept of programming and opened the 32 1981 Simplicity in mutex algorithms Peterson way for the first books of principles on 1983 Asynchronous randomized consensus (DA 2015) Ben-Or,5 Rabin34 1985 Liveness (progress condition) (DA 2018) Alpern, Schneider1 concurrent programming, which ap- Impossibility of asynchronous deterministic peared at the beginning of the 1970s. 1985 consensus in the presence of process crashes Fischer, Lynch, Paterson16 Locks. A mutual exclusion algorithm (DA 2001) 1987 Fast mutual exclusion Lamport27 consists of the code for two opera- 1991 Wait-free synchronization (DA 2003) Herlihy19 tions—acquire() and release()— Herlihy, Moss,20 Shavit, 1993, 1997 Transactional memory (DA 2012) that a process invokes to bracket a Touitou40 section of code called a critical section. Shared memory on top of asynchronization 1995 message-passing systems despite a minority Attiya, Bar Noy, Dolev2 The usual environment in which it is of process crashes (DA 2011) executed is asynchronous, where pro- Weakest information on failures to solve consensus cess speeds are arbitrary, independent 1996 in the presence of asynchrony and process crashes Chandra, Hadzilacos, Toueg8 (DA 2010) from each other. A mutual exclusion 2008 Scalability, accountability Nakamoto30 algorithm guarantees two properties. ˲ Mutual exclusion. No two processes A paper that received the Dijkstra ACM Award in the year X is marked (DA X). are simultaneously executing their crit- ical section. ˲ Deadlock-freedom. If one or several cant examples of sequential reasoning be that it is expensive to implement, processes invoke acquire() opera- used to master concurrency, providing and furthermore, there are inher- tions that are executed concurrently, a sample of fundamental notions of ently concurrent problems with no eventually one of them terminates its this approach, and we describe several sequential specifications. invocation, and consequently executes algorithms, both shared memory and its critical section. message passing, as a concrete illustra- Mutual Exclusion Deadlock-freedom does not pre- tion of the ideas. Concurrent computing began in 1961 vent specific timing scenarios from oc- We tell the story through an evo- with what was called multiprogramming curring in which some processes can lution that starts with mutual ex- in the Atlas computer, where concur- never enter their critical section. The clusion, followed by implementing rency was simulated—as we do when stronger starvation-freedom progress read/write registers on top of mes- telling stories where things happen con- condition states that any process that sage passing systems, then imple- currently—interlacing the execution of invokes acquire() will terminate its menting arbitrary objects through sequential programs. Concurrency was invocation (and will consequently ex- powerful synchronization mecha- born in order to make efficient use of a ecute its critical section). nisms. We discuss the modern dis- sequential computer, which can execute A mutual exclusion algorithm. The tributed ledger trends of doing so in only one instruction at a time, giving us- first mutual exclusion algorithms were a highly scalable, tamper-proof way. ers the illusion that their programs are difficult to understand and prove cor- We conclude with a discussion of