The Art of Multiprocessor Programming Copyright 2007 Elsevier Inc
Total Page：16
File Type：pdf, Size：1020Kb
The Art of Multiprocessor Programming Copyright 2007 Elsevier Inc. All rights reserved Maurice Herlihy Nir Shavit October 3, 2007 DRAFT COPY 2 DRAFT COPY Contents 1 Introduction 13 1.1SharedObjectsandSynchronization.............. 16 1.2AFable.............................. 19 1.2.1 PropertiesofMutualExclusion............. 21 1.2.2 TheMoral......................... 22 1.3TheProducer-ConsumerProblem................ 23 1.4TheReaders/WritersProblem.................. 26 1.5TheHarshRealitiesofParallelization............. 27 1.6Missive.............................. 29 1.7ChapterNotes.......................... 30 1.8Exercises............................. 30 Principles 35 2 Mutual Exclusion 37 2.1Time................................ 37 2.2CriticalSections......................... 38 2.3Two-ThreadSolutions...................... 41 2.3.1 The LockOne Class.................... 41 2.3.2 The LockTwo Class.................... 43 2.3.3 ThePetersonLock.................... 44 2.4TheFilterLock..........................DRAFT COPY 46 2.5Fairness.............................. 49 2.6Lamport’sBakeryAlgorithm.................. 49 2.7 Bounded Timestamps . ................ 51 2.8 Lower Bounds on Number of Locations . ...... 56 2.9ChapterNotes.......................... 60 2.10Exercises............................. 60 3 4 CONTENTS 3 Concurrent Objects 67 3.1ConcurrencyandCorrectness.................. 67 3.2SequentialObjects........................ 71 3.3QuiescentConsistency...................... 72 3.3.1 Remarks.......................... 74 3.4SequentialConsistency...................... 75 3.4.1 Remarks.......................... 76 3.5 Linearizability . ......................... 79 3.5.1 LinearizationPoints................... 79 3.5.2 Remarks.......................... 79 3.6FormalDefinitions........................ 80 3.6.1 Linearizability . .................. 81 3.6.2 Linearizability is Compositional . ........ 82 3.6.3 TheNon-BlockingProperty............... 83 3.7ProgressConditions....................... 84 3.7.1 DependentProgressConditions............. 85 3.8TheJavaMemoryModel.................... 86 3.8.1 LocksandSynchronizedBlocks............. 88 3.8.2 VolatileFields...................... 88 3.8.3 FinalFields........................ 89 3.9Remarks.............................. 90 3.10ChapterNotes.......................... 91 3.11Exercises............................. 91 4 Foundations of Shared Memory 97 4.1TheSpaceofRegisters...................... 98 4.2RegisterConstructions......................104 4.2.1 MRSWSafeRegisters..................105 4.2.2 ARegularBooleanMRSWRegister..........105 4.2.3 A regular M-valuedMRSWregister...........106 4.2.4 AnAtomicSRSWRegister...............109 4.2.5 AnAtomicMRSWRegister...............112 DRAFT4.2.6 AnAtomicMRMWRegister..............115 COPY 4.3AtomicSnapshots........................117 4.3.1 AnObstruction-freeSnapshot..............117 4.3.2 AWait-FreeSnapshot..................119 4.3.3 CorrectnessArguments.................119 4.4ChapterNotes..........................121 4.5Exercises.............................122 CONTENTS 5 5 The Relative Power of Primitive Synchronization Opera- tions 133 5.1ConsensusNumbers.......................134 5.1.1 StatesandValence....................135 5.2AtomicRegisters.........................138 5.3ConsensusProtocols.......................140 5.4FIFOQueues...........................141 5.5MultipleAssignmentObjects..................145 5.6Read-Modify-WriteOperations.................148 5.7Common2RMWOperations..................150 5.8 The compareAndSet() Operation................152 5.9ChapterNotes..........................154 5.10Exercises.............................155 6 Universality of Consensus 163 6.1Introduction............................163 6.2Universality............................164 6.3ALock-freeUniversalConstruction...............165 6.4AWait-freeUniversalConstruction...............169 6.5ChapterNotes..........................176 6.6Exercises.............................176 Practice 179 7 Spin Locks and Contention 181 7.1WelcometotheRealWorld...................181 7.2Test-and-SetLocks........................185 7.3TAS-BasedSpinLocksRevisited................188 7.4ExponentialBackoff.......................190 7.5QueueLocks...........................192 7.5.1 Array-BasedLocks....................192 7.5.2 TheCLHQueueLock..................193 7.5.3 TheMCSQueueLock..................197 7.6AQueueLockwithTimeouts..................200DRAFT COPY 7.7ACompositeLock........................203 7.7.1 AFast-PathCompositeLock..............210 7.8HierarchicalLocks........................212 7.8.1 AHierarchicalBackoffLock...............213 7.8.2 AHierarchicalCLHQueueLock............214 7.9OneLockToRuleThemAll..................220 6 CONTENTS 7.10ChapterNotes..........................220 7.11Exercises.............................221 8 Monitors and Blocking Synchronization 223 8.1Introduction............................223 8.2MonitorLocksandConditions.................224 8.2.1 Conditions........................225 8.2.2 TheLost-WakeupProblem...............228 8.3Readers-WritersLocks......................231 8.3.1 SimpleReaders-WritersLock..............231 8.3.2 FairReaders-WritersLock................232 8.4AReentrantLock.........................234 8.5Semaphores............................237 8.6ChapterNotes..........................239 8.7Exercises.............................240 9 Linked Lists: the Role of Locking 245 9.1Introduction............................245 9.2List-basedSets..........................246 9.3ConcurrentReasoning......................248 9.4Coarse-GrainedSynchronization................250 9.5Fine-GrainedSynchronization..................252 9.6OptimisticSynchronization...................257 9.7LazySynchronization......................261 9.8ALock-FreeList.........................267 9.9Discussion.............................272 9.10ChapterNotes..........................274 9.11Exercises.............................274 10 Concurrent Queues and the ABA Problem 277 10.1Introduction............................277 10.2Queues...............................279 10.3 A Bounded Partial Queue . ..................279 DRAFT10.4 An Unbounded Total Queue . ..................285COPY 10.5 An Unbounded Lock-Free Queue . ........286 10.6MemoryreclamationandtheABAproblem..........290 10.6.1 A Na¨ıveSynchronousQueue..............294 10.7DualDataStructures......................296 10.8ChapterNotes..........................299 10.9Exercises.............................299 CONTENTS 7 11 Concurrent Stacks and Elimination 303 11.1Introduction............................303 11.2 An Unbounded Lock-free Stack . ................303 11.3Elimination............................304 11.4TheEliminationBackoffStack.................305 11.4.1ALock-freeExchanger..................308 11.4.2TheEliminationArray..................311 11.5ChapterNotes..........................314 11.6Exercises.............................315 12 Counting, Sorting, and Distributed Coordination 321 12.1Introduction............................321 12.2SharedCounting.........................321 12.3SoftwareCombining.......................322 12.3.1Overview.........................323 12.3.2AnExtendedExample..................330 12.3.3PerformanceandRobustness..............333 12.4Quiescently-ConsistentPoolsandCounters..........333 12.5CountingNetworks........................334 12.5.1Networksthatcount...................334 12.5.2TheBitonicCountingNetwork.............337 12.5.3PerformanceandPipelining...............345 12.6DiffractingTrees.........................348 12.7ParallelSorting..........................353 12.8SortingNetworks.........................354 12.8.1DesigningaSortingNetwork..............354 12.9SampleSorting..........................357 12.10DistributedCoordination....................360 12.11ChapterNotes..........................361 12.12Exercises.............................362 13 Concurrent Hashing and Natural Parallelism 367 13.1Introduction............................367 13.2Closed-AddressHashSets....................368DRAFT COPY 13.2.1ACoarse-GrainedHashSet...............371 13.2.2AStripedHashSet....................373 13.2.3ARefinableHashSet..................376 13.3ALock-freeHashSet.......................379 13.3.1RecursiveSplit-ordering.................379 13.3.2 The BucketListclass...................383 8 CONTENTS 13.3.3 The LockFreeHashSet<T> class.............385 13.4AnOpen-AddressedHashSet..................388 13.4.1CuckooHashing.....................389 13.4.2ConcurrentCuckooHashing...............390 13.4.3StripedConcurrentCuckooHashing..........395 13.4.4 A Refinable Concurrent Cuckoo Hash Set . 395 13.5ChapterNotes..........................396 13.6Exercises.............................396 14 Skiplists and Balanced Search 405 14.1Introduction............................405 14.2SequentialSkiplists........................406 14.3ALock-BasedConcurrentSkiplist...............407 14.3.1ABird’sEyeView....................407 14.3.2TheAlgorithm......................410 14.4ALock-FreeConcurrentSkiplist................417 14.4.1ABird’sEyeView....................417 14.4.2TheAlgorithminDetail.................420 14.5ConcurrentSkiplists.......................426 14.6ChapterNotes..........................428 14.7Exercises.............................429 15 Priority Queues 433 15.1Introduction............................433 15.1.1ConcurrentPriorityQueues...............434 15.2 An Array-Based Bounded Priority Queue . ........434 15.3 A Tree-Based Bounded Priority Queue . ........435 15.4 An Unbounded Heap-Based Priority Queue