Lock-free Shared Data Structures 10th Estonian Summer School on Computer and Systems Science Introduction and Motivation Eric Ruppert DisCoVeri Group York University Toronto, Canada August, 2011 Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Moore’s Law Moore’s Law for Automobiles Gordon Moore [1965]: “If the automobile industry advanced as rapidly as the The number of components in semiconductor industry, a Rolls Royce would get a integrated circuits doubled each year million miles per gallon, and it would be cheaper to between 1958 and 1965. throw it away than to park it.” – Gordon Moore Continued doubling every 1 to 2 years since. This yielded exponential increases in performance: clock speeds increased memory capacity increased processing power increased Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Limits of Moore’s Law Multicore Machines Exponential improvements cannot go on forever. Multicore machines have multiple processor chips. Components will soon be getting down to the scale of atoms. Each chip has multiple cores. Even now, increases in computing power come from having Each core can run multiple programmes (or processes). more processors, not bigger or faster processors. Today: Machines can run 100 processes concurrently. Big performance gains in the future will come from parallelism. Future: Many more processes. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures The Big Challenge for Computer Science Amdahl’s Law Suppose we only know how to do 75% of some task in parallel. # Cores Time required Speedup 1 100 1.0 2 63 1.6 Most algorithms are designed for one process. 3 50 2.0 4 44 2.3 . We must design efficient parallel solutions to problems. 10 32 3.1 . 100 26 3.9 . 1 25 4.0 Better parallelization is needed to make use of extra cores. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures One Challenge of Parallelization What These Lectures Are About Many processes must cooperate efficiently to solve a problem. Designing implementations of shared data structures. Cooperation often requires communication between processes and sharing data. Desirable properties: load-balancing requires transferring tasks between (reasonably) easy to use processes can be accessed concurrently by several processes distributing input to processes efficient (in terms of time and space) (inputs may arrive online at different processes) scalable aggregating results of different processes’ computations (fault-tolerant) communicating partial results from one process to another Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Models of Distributed Systems Two main models of communication in distributed computing. Shared Memory Formalizing the Problem Message Passing Processes op response Memory Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Shared-Memory Model Types of Shared Objects We will focus on the shared-memory model. (It closely resembles multicore machines.) Read-write register Shared Memory Each register stores a value and provides two operations: Shared memory contains READ(): returns current value stored and does not change objects of various types. Processes the value Each process can perform op WRITE(v): changes the stored value to v and returns ack operations on the objects. response The operation can change The most basic type of shared object. the state of the object and Memory Provided in hardware. return a response. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Another Type Formal Definition of an Object Type FIFO Queue Sequential specification describes how object behaves when Stores a list of items and provides two operations: accessed by a single process at a time. ENQUEUE(x): adds x to the end of the list and returns ack Q is the set of states DEQUEUE(): removes and returns one item from the front of the list OP is the set of operations that can be applied to the object RES is the set of responses the object can return 17 24 97 12 12 δ ⊆ Q × OP × RES × Q is the transition relation If (q; op; res; q0) 2 δ, it means that ENQUEUE(23) DEQUEUE!12 when op is applied to an object in state q, the operation can return response res and Useful for dividing up work among a collection of processes. the object can change to state q0. Not usually provided in hardware; must build it in software. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Example: Read/Write Register Example: Queue Object stores a natural number and allows processes to write A FIFO queue of natural numbers. values and read current value. Q = set of all finite sequences of natural numbers Q = IN OP = fENQUEUE(v): v 2 INg [ fDEQUEUE()g OP = fWRITE(v): v 2 INg [ fREADg RES = IN [ fNIL; ACKg RES = IN [ fACKg δ = f(σ; ENQUEUE(v); ACK; hvi · σ): σ 2 Q; v 2 INg [ 0 0 0 δ = f(v; WRITE(v ); ACK; v ): v; v 2 INg [ f(σ · hvi; DEQUEUE(); v; σ): σ 2 Q; v 2 INg [ f(v; READ; v; v): v 2 INg f(hi; DEQUEUE(); NIL; hi)g Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures How to Share an Object An Example: Read-Write Register Suppose we have a read-write register initialized to value 0. Process In reality, performing an operation is not instantaneous; P it takes some interval of time to complete. WRITE(1) What happens when processes access an object concurrently? Q WRITE(2) How should the object behave? R READ!? time READ can output 1 or 2. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures An Example: Read-Write Register An Example: Read-Write Register Suppose we have a read-write register initialized to value 0. Suppose we have a read-write register initialized to value 0. Process Process P WRITE(1) P WRITE(1) WRITE(3) Q WRITE(2) READ!? Q WRITE(2) READ!? R READ!? R READ!? time time Both READS can output 1 or 2. For linearizability, they should Both READS can output 1, 2 or 3. For linearizability, they should both output the same value. not output 1 and 2. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures An Example: Read-Write Register An Example: Read-Write Register Suppose we have a read-write register initialized to value 0. Suppose we have a read-write register initialized to value 0. Process Process ? 4 P WRITE(1) P WRITE(1) Not permitted 6 ?? Q WRITE(2) READ!1 Q WRITE(2) READ!1 ? R READ!2 R READ!2 time time Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Linearizability A More Complicated Execution An object (built in hardware or software) is linearizable if its Process operations seem to occur instantaneously. ? ? ? 4 P READ!0 READ!1 WRITE(2) More formally: ? ? For every execution that uses the objects, Q WRITE(1) READ!1 we can choose a ? inside each operation’s time interval so that ? ? all operations would return the same results if they were R READ!0 READ!2 performed sequentially in the order of their ?s. The ? is called the linearization point of the operation. time Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Linearizability Example with a Queue Alternative Definition of Linearizability Consider an initially empty queue. What are the possible values for A and B? Process For every execution that uses the objects, there exists a sequential order of all the operations such that P ENQUEUE(1) DEQUEUE!B 1 all operations return the same results in the sequential order, and Q DEQUEUE!A 2 if op1 ends before op2 begins, then op1 precedes op2 in the sequential order. R ENQUEUE(2) Original (and equivalent) definition of Herlihy and Wing [1990]. time A = 1; B = 2 OR A = 2; B = 1 OR A = NIL; B = 1 OR A = 1; B = NIL Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Consistency Conditions Specifying Correctness: Summary Linearizability is a consistency condition. There are many other weaker ones: sequential consistency How to specify correctness of a shared data structure? processor consistency Safety = sequential specification + linearizability causal consistency (Q; OPS; RES; δ) etc. Progress: operations terminate (discussed later) Weaker conditions are easier to implement but harder to use. We will focus on linearizable objects. ) When using linearizable objects, can assume operations happen atomically. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures What are These Talks About? Challenges of Implementing Shared Objects Goal: Design shared data structure implementations using basic objects provided by system (like registers). Asynchrony (processes run at different, variable speeds). Assume basic objects are linearizable. Concurrent updates should not corrupt data. Prove implemented objects are also linearizable. Reading data while others update it should not produce inconsistent results. Study complexity of implementations. Other unpredictable events (e.g., failures). Prove some implementations are impossible. Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Traditional Approach: Locks Fine-Grained Locking Each object has a lock associated with it. Each part of an object has a lock. Only one process can hold the lock at any time. 1 Obtain locks for parts you want to access 1 obtain lock 2 Access those parts of object 2 access object 3 Release locks 3 release lock Cons: Cons: Pros: Pros: limited parallelism slow some parallelism simple no fault-tolerance avoids problems of no real parallelism reduces problems of concurrent accesses danger of deadlock, concurrent accesses no fault-tolerance livelock Eric Ruppert Lock-free Shared Data Structures Eric Ruppert Lock-free Shared Data Structures Alternative Approach: Lock-Free Implementations Lock-free Implementations Avoid use of locks altogether.