Java Concurrency in Practice

Advance praise for Java Concurrency in Practice

I was fortunate indeed to have worked with a fantastic team on the design and implementation of the concurrency features added to the Java platform in Java 5.0 and Java 6. Now this same team provides the best explanation yet of these new features, and of concurrency in general. Concurrency is no longer a subject for advanced users only. Every Java developer should read this book.

—Martin Buchholz JDK Concurrency Czar, Sun Microsystems

For the past 30 years, computer performance has been driven by Moore’s Law; from now on, it will be driven by Amdahl’s Law. Writing code that effectively exploits multiple processors can be very challenging. Java Concurrency in Practice provides you with the concepts and techniques needed to write safe and scalable Java programs for today’s—and tomorrow’s—systems.

—Doron Rajwan Research Scientist, Intel Corp

This is the book you need if you’re writing—or designing, or debugging, or main- taining, or contemplating—multithreaded Java programs. If you’ve ever had to synchronize a method and you weren’t sure why, you owe it to yourself and your users to read this book, cover to cover.

—Ted Neward Author of Effective Enterprise Java

Brian addresses the fundamental issues and complexities of concurrency with uncommon clarity. This book is a must-read for anyone who uses threads and cares about performance.

—Kirk Pepperdine CTO, JavaPerformanceTuning.com

This book covers a very deep and subtle topic in a very clear and concise way, making it the perfect Java Concurrency reference manual. Each page is ﬁlled with the problems (and solutions!) that programmers struggle with every day. Effectively exploiting concurrency is becoming more and more important now that Moore’s Law is delivering more cores but not faster cores, and this book will show you how to do it.

—Dr. Cliff Click Senior Software Engineer, Azul Systems I have a strong interest in concurrency, and have probably written more thread deadlocks and made more synchronization mistakes than most programmers. Brian’s book is the most readable on the topic of threading and concurrency in Java, and deals with this difﬁcult subject with a wonderful hands-on approach. This is a book I am recommending to all my readers of The Java Specialists’ Newsletter, because it is interesting, useful, and relevant to the problems facing Java developers today.

—Dr. Heinz Kabutz The Java Specialists’ Newsletter

I’ve focused a career on simplifying simple problems, but this book ambitiously and effectively works to simplify a complex but critical subject: concurrency. Java Concurrency in Practice is revolutionary in its approach, smooth and easy in style, and timely in its delivery—it’s destined to be a very important book.

—Bruce Tate Author of Beyond Java

Java Concurrency in Practice is an invaluable compilation of threading know-how for Java developers. I found reading this book intellectually exciting, in part because it is an excellent introduction to Java’s concurrency API, but mostly because it captures in a thorough and accessible way expert knowledge on threading not easily found elsewhere.

—Bill Venners Author of Inside the Java Virtual Machine Java Concurrency in Practice This page intentionally left blank Java Concurrency in Practice

Brian Goetz with Tim Peierls Joshua Bloch Joseph Bowbeer David Holmes and Doug Lea

Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trade- marks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419 [email protected] For sales outside the United States, please contact: International Sales [email protected] Visit us on the Web: www.awprofessional.com

This Book Is Safari Enabled The Safari® Enabled icon on the cover of your favorite technology book means the book is available through Safari Bookshelf. When you buy this book, you get free access to the online edition for 45 days. Safari Bookshelf is an electronic reference library that lets you easily search thousands of technical books, ﬁnd code samples, download chapters, and access technical information whenever and wherever you need it. To gain 45-day Safari Enabled access to this book: • Go to http://www.awprofessional.com/safarienabled • Complete the brief registration form • Enter the coupon code UUIR-XRJG-JWWF-AHGM-137Z If you have difﬁculty registering on Safari Bookshelf or accessing the online edition, please e-mail customer-ser- [email protected].

Library of Congress Cataloging-in-Publication Data Goetz, Brian. Java Concurrency in Practice / Brian Goetz, with Tim Peierls. . . [et al.] p. cm. Includes bibliographical references and index. ISBN 0-321-34960-1 (pbk. : alk. paper) 1. Java (Computer program language) 2. Parallel programming (Computer science) 3. Threads (Computer programs) I. Title.

ISBN 0-321-34960-1 Text printed in the United States on recycled paper at Courier Stoughton in Stoughton, Massachusetts. 13th Printing To Jessica This page intentionally left blank Contents

Listings xii

Preface xvii

1 Introduction 1 1.1 A (very) brief history of concurrency ...... 1 1.2 Beneﬁtsofthreads...... 3 1.3 Risks of threads ...... 5 1.4 Threadsareeverywhere...... 9

I Fundamentals 13

2 Thread Safety 15 2.1 Whatisthreadsafety?...... 17 2.2 Atomicity...... 19 2.3 Locking...... 23 2.4 Guardingstatewithlocks...... 27 2.5 Livenessandperformance...... 29

3 Sharing Objects 33 3.1 Visibility ...... 33 3.2 Publicationandescape...... 39 3.3 Threadconﬁnement...... 42 3.4 Immutability ...... 46 3.5 Safepublication...... 49

4 Composing Objects 55 4.1 Designing a thread-safe class ...... 55 4.2 Instanceconﬁnement...... 58 4.3 Delegatingthreadsafety...... 62 4.4 Addingfunctionalitytoexistingthread-safeclasses...... 71 4.5 Documentingsynchronizationpolicies...... 74

ix x Contents

5 Building Blocks 79 5.1 Synchronizedcollections...... 79 5.2 Concurrent collections ...... 84 5.3 Blockingqueuesandtheproducer-consumerpattern...... 87 5.4 Blocking and interruptible methods ...... 92 5.5 Synchronizers...... 94 5.6 Buildinganefﬁcient,scalableresultcache...... 101

II Structuring Concurrent Applications 111

6 Task Execution 113 6.1 Executingtasksinthreads...... 113 6.2 The Executor framework...... 117 6.3 Finding exploitable parallelism ...... 123

7 Cancellation and Shutdown 135 7.1 Taskcancellation...... 135 7.2 Stoppingathread-basedservice...... 150 7.3 Handlingabnormalthreadtermination...... 161 7.4 JVMshutdown...... 164

8 Applying Thread Pools 167 8.1 Implicitcouplingsbetweentasksandexecutionpolicies...... 167 8.2 Sizingthreadpools...... 170 8.3 Conﬁguring ThreadPoolExecutor ...... 171 8.4 Extending ThreadPoolExecutor ...... 179 8.5 Parallelizingrecursivealgorithms...... 181

9 GUI Applications 189 9.1 Why are GUIs single-threaded? ...... 189 9.2 Short-runningGUItasks...... 192 9.3 Long-runningGUItasks...... 195 9.4 Shareddatamodels...... 198 9.5 Other forms of single-threaded subsystems ...... 202

III Liveness, Performance, and Testing 203

10 Avoiding Liveness Hazards 205 10.1 Deadlock...... 205 10.2 Avoidinganddiagnosingdeadlocks...... 215 10.3 Other liveness hazards ...... 218

11 Performance and Scalability 221 11.1 Thinkingaboutperformance...... 221 11.2 Amdahl’slaw...... 225 11.3 Costsintroducedbythreads...... 229 11.4 Reducinglockcontention...... 232 Contents xi

11.5 Example: Comparing Map performance...... 242 11.6 Reducingcontextswitchoverhead...... 243

12 Testing Concurrent Programs 247 12.1 Testing for correctness ...... 248 12.2 Testingforperformance...... 260 12.3 Avoidingperformancetestingpitfalls...... 266 12.4 Complementarytestingapproaches...... 270

IV Advanced Topics 275

13 Explicit Locks 277 13.1 Lock and ReentrantLock ...... 277 13.2 Performance considerations ...... 282 13.3 Fairness...... 283 13.4 Choosing between synchronized and ReentrantLock ...... 285 13.5 Read-writelocks...... 286

14 Building Custom Synchronizers 291 14.1 Managingstatedependence...... 291 14.2 Usingconditionqueues...... 298 14.3 Explicitconditionobjects...... 306 14.4 Anatomyofasynchronizer...... 308 14.5 AbstractQueuedSynchronizer ...... 311 14.6 AQS in java.util.concurrent synchronizerclasses...... 314

15 Atomic Variables and Nonblocking Synchronization 319 15.1 Disadvantagesoflocking...... 319 15.2 Hardware support for concurrency ...... 321 15.3 Atomicvariableclasses...... 324 15.4 Nonblockingalgorithms...... 329

16 The Java Memory Model 337 16.1 Whatisamemorymodel,andwhywouldIwantone?...... 337 16.2 Publication...... 344 16.3 Initializationsafety...... 349

A Annotations for Concurrency 353 A.1 Class annotations ...... 353 A.2 Field and method annotations ...... 353

Bibliography 355

Index 359 Listings

1 Bad way to sort a list. Don’t do this...... xix 2 Less than optimal way to sort a list...... xx 1.1 Non-thread-safesequencegenerator...... 6 1.2 Thread-safesequencegenerator...... 7 2.1 Astatelessservlet...... 18 2.2 Servlet that counts requests without the necessary synchronization. Don’t do this...... 19 2.3 Race condition in lazy initialization. Don’t do this...... 21 2.4 Servlet that counts requests using AtomicLong...... 23 2.5 Servlet that attempts to cache its last result without adequate atomicity. Don’t do this...... 24 2.6 Servlet that caches last result, but with unnacceptably poor concurrency. Don’t do this...... 26 2.7 Code that would deadlock if intrinsic locks were not reentrant. . . . 27 2.8 Servletthatcachesitslastrequestandresult...... 31 3.1 Sharing variables without synchronization. Don’t do this...... 34 3.2 Non-thread-safemutableintegerholder...... 36 3.3 Thread-safemutableintegerholder...... 36 3.4 Countingsheep...... 39 3.5 Publishing an object...... 40 3.6 Allowing internal mutable state to escape. Don’t do this...... 40 3.7 Implicitly allowing the this reference to escape. Don’t do this. ... 41 3.8 Using a factory method to prevent the this reference from escap- ingduringconstruction...... 42 3.9 Thread confinement of local primitive and reference variables. . . . 44 3.10 Using ThreadLocal toensurethreadconfinement...... 45 3.11 Immutable class built out of mutable underlying objects...... 47 3.12 Immutableholderforcachinganumberanditsfactors...... 49 3.13 Caching the last result using a volatile reference to an immutable holderobject...... 50 3.14 Publishing an object without adequate synchronization. Don’t do this...... 50 3.15 Class at risk of failure if not properly published...... 51 4.1 Simple thread-safe counter using the Java monitor pattern...... 56 4.2 Usingconfinementtoensurethreadsafety...... 59 4.3 Guardingstatewithaprivatelock...... 61

xii Listings xiii

4.4 Monitor-basedvehicletrackerimplementation...... 63 4.5 Mutable point class similar to java.awt.Point...... 64 4.6 Immutable Point class used by DelegatingVehicleTracker..... 64 4.7 Delegating thread safety to a ConcurrentHashMap...... 65 4.8 Returning a static copy of the location set instead of a “live” one. . 66 4.9 Delegating thread safety to multiple underlying state variables. . . 66 4.10 Number range class that does not sufﬁciently protect its invariants. Don’t do this...... 67 4.11 Thread-safemutablepointclass...... 69 4.12 Vehicle tracker that safely publishes underlying state...... 70 4.13 Extending Vector tohaveaput-if-absentmethod...... 72 4.14 Non-thread-safe attempt to implement put-if-absent. Don’t do this. . 72 4.15 Implementingput-if-absentwithclient-sidelocking...... 73 4.16 Implementingput-if-absentusingcomposition...... 74 5.1 Compound actions on a Vector that may produce confusing results. 80 5.2 Compound actions on Vector usingclient-sidelocking...... 81 5.3 Iteration that may throw ArrayIndexOutOfBoundsException... .. 81 5.4 Iterationwithclient-sidelocking...... 82 5.5 Iterating a List with an Iterator...... 82 5.6 Iteration hidden within string concatenation. Don’t do this...... 84 5.7 ConcurrentMap interface...... 87 5.8 Producer and consumer tasks in a desktop search application. . . . 91 5.9 Startingthedesktopsearch...... 92 5.10 Restoring the interrupted status so as not to swallow the interrupt. 94 5.11 Using CountDownLatch for starting and stopping threads in timing tests...... 96 5.12 Using FutureTask to preload data that is needed later...... 97 5.13 Coercing an unchecked Throwable to a RuntimeException...... 98 5.14 Using Semaphore toboundacollection...... 100 5.15 Coordinating computation in a cellular automaton with Cyclic- Barrier...... 102 5.16 Initial cache attempt using HashMap andsynchronization...... 103 5.17 Replacing HashMap with ConcurrentHashMap...... 105 5.18 Memoizing wrapper using FutureTask...... 106 5.19 Final implementation of Memoizer...... 108 5.20 Factorizing servlet that caches results using Memoizer...... 109 6.1 Sequentialwebserver...... 114 6.2 Webserverthatstartsanewthreadforeachrequest...... 115 6.3 Executor interface...... 117 6.4 Webserverusingathreadpool...... 118 6.5 Executor thatstartsanewthreadforeachtask...... 118 6.6 Executor that executes tasks synchronously in the calling thread. . 119 6.7 Lifecycle methods in ExecutorService...... 121 6.8 Webserverwithshutdownsupport...... 122 6.9 Class illustrating confusing Timer behavior...... 124 6.10 Rendering page elements sequentially...... 125 6.11 Callable and Future interfaces...... 126 xiv Listings

6.12 Default implementation of newTaskFor in ThreadPoolExecutor...126 6.13 Waiting for image download with Future...... 128 6.14 QueueingFuture class used by ExecutorCompletionService.....129 6.15 Using CompletionService to render page elements as they be- comeavailable...... 130 6.16 Fetchinganadvertisementwithatimebudget...... 132 6.17 Requestingtravelquotesunderatimebudget...... 134 7.1 Using a volatile fieldtoholdcancellationstate...... 137 7.2 Generatingasecond’sworthofprimenumbers...... 137 7.3 Unreliable cancellation that can leave producers stuck in a blocking operation. Don’t do this...... 139 7.4 Interruption methods in Thread...... 139 7.5 Using interruption for cancellation...... 141 7.6 Propagating InterruptedException tocallers...... 143 7.7 Noncancelable task that restores interruption before exit...... 144 7.8 Scheduling an interrupt on a borrowed thread. Don’t do this. ....145 7.9 Interrupting a task in a dedicated thread...... 146 7.10 Cancelling a task using Future...... 147 7.11 Encapsulating nonstandard cancellation in a Thread by overriding interrupt...... 149 7.12 Encapsulating nonstandard cancellation in a task with newTaskFor. 151 7.13 Producer-consumer logging service with no shutdown support. . . 152 7.14 Unreliable way to add shutdown support to the logging service. . . 153 7.15 Adding reliable cancellation to LogWriter...... 154 7.16 Logging service that uses an ExecutorService...... 155 7.17 Shutdown with poison pill...... 156 7.18 Producer thread for IndexingService...... 157 7.19 Consumer thread for IndexingService...... 157 7.20 Using a private Executor whose lifetime is bounded by a method call...... 158 7.21 ExecutorService that keeps track of cancelled tasks after shutdown.159 7.22 Using TrackingExecutorService to save unfinished tasks for later execution...... 160 7.23 Typicalthread-poolworkerthreadstructure...... 162 7.24 UncaughtExceptionHandler interface...... 163 7.25 UncaughtExceptionHandler thatlogstheexception...... 163 7.26 Registering a shutdown hook to stop the logging service...... 165 8.1 Task that deadlocks in a single-threaded Executor. Don’t do this. ..169 8.2 General constructor for ThreadPoolExecutor...... 172 8.3 Creating a fixed-sized thread pool with a bounded queue and the caller-runssaturationpolicy...... 175 8.4 Using a Semaphore to throttle task submission...... 176 8.5 ThreadFactory interface...... 176 8.6 Customthreadfactory...... 177 8.7 Customthreadbaseclass...... 178 8.8 Modifying an Executor created with the standard factories. . . . . 179 8.9 Thread pool extended with logging and timing...... 180 Listings xv

8.10 Transforming sequential execution into parallel execution...... 181 8.11 Transforming sequential tail-recursion into parallelized recursion. . 182 8.12 Waitingforresultstobecalculatedinparallel...... 182 8.13 Abstraction for puzzles like the “sliding blocks puzzle”...... 183 8.14 Linknodeforthepuzzlesolverframework...... 184 8.15 Sequentialpuzzlesolver...... 185 8.16 Concurrent version of puzzle solver...... 186 8.17 Result-bearing latch used by ConcurrentPuzzleSolver...... 187 8.18 Solverthatrecognizeswhennosolutionexists...... 188 9.1 Implementing SwingUtilities using an Executor...... 193 9.2 Executor built atop SwingUtilities...... 194 9.3 Simple event listener...... 194 9.4 Bindingalong-runningtasktoavisualcomponent...... 196 9.5 Long-running task with user feedback...... 196 9.6 Cancelling a long-running task...... 197 9.7 Background task class supporting cancellation, completion notiﬁ- cation,andprogressnotiﬁcation...... 199 9.8 Initiating a long-running, cancellable task with BackgroundTask..200 10.1 Simple lock-ordering deadlock. Don’t do this...... 207 10.2 Dynamic lock-ordering deadlock. Don’t do this...... 208 10.3 Inducing a lock ordering to avoid deadlock...... 209 10.4 Driver loop that induces deadlock under typical conditions. . . . . 210 10.5 Lock-ordering deadlock between cooperating objects. Don’t do this. 212 10.6 Using open calls to avoiding deadlock between cooperating objects. 214 10.7 Portionofthreaddumpafterdeadlock...... 217 11.1 Serializedaccesstoataskqueue...... 227 11.2 Synchronization that has no effect. Don’t do this...... 230 11.3 Candidate for lock elision...... 231 11.4 Holdingalocklongerthannecessary...... 233 11.5 Reducinglockduration...... 234 11.6 Candidateforlocksplitting...... 236 11.7 ServerStatus refactoredtousesplitlocks...... 236 11.8 Hash-basedmapusinglockstriping...... 238 12.1 Bounded buffer using Semaphore...... 249 12.2 Basic unit tests for BoundedBuffer...... 250 12.3 Testing blocking and responsiveness to interruption...... 252 12.4 Medium-quality random number generator suitable for testing. . . 253 12.5 Producer-consumer test program for BoundedBuffer...... 255 12.6 Producer and consumer classes used in PutTakeTest...... 256 12.7 Testingforresourceleaks...... 258 12.8 Thread factory for testing ThreadPoolExecutor...... 258 12.9 Testmethodtoverifythreadpoolexpansion...... 259 12.10 Using Thread.yield togeneratemoreinterleavings...... 260 12.11 Barrier-basedtimer...... 261 12.12 Testingwithabarrier-basedtimer...... 262 12.13 Driver program for TimedPutTakeTest...... 262 13.1 Lock interface...... 277 xvi Listings

13.2 Guarding object state using ReentrantLock...... 278 13.3 Avoiding lock-ordering deadlock using tryLock...... 280 13.4 Lockingwithatimebudget...... 281 13.5 Interruptible lock acquisition...... 281 13.6 ReadWriteLock interface...... 286 13.7 Wrapping a Map witharead-writelock...... 288 14.1 Structureofblockingstate-dependentactions...... 292 14.2 Baseclassforboundedbufferimplementations...... 293 14.3 Bounded buffer that balks when preconditions are not met...... 294 14.4 Client logic for calling GrumpyBoundedBuffer...... 294 14.5 Boundedbufferusingcrudeblocking...... 296 14.6 Boundedbufferusingconditionqueues...... 298 14.7 Canonical form for state-dependent methods...... 301 14.8 Using conditional notification in BoundedBuffer.put...... 304 14.9 Recloseablegateusingwait and notifyAll...... 305 14.10 Condition interface...... 307 14.11 Boundedbufferusingexplicitconditionvariables...... 309 14.12 Counting semaphore implemented using Lock...... 310 14.13 Canonical forms for acquisition and release in AQS...... 312 14.14 Binary latch using AbstractQueuedSynchronizer...... 313 14.15 tryAcquire implementation from nonfair ReentrantLock...... 315 14.16 tryAcquireShared and tryReleaseShared from Semaphore.....316 15.1 SimulatedCASoperation...... 322 15.2 NonblockingcounterusingCAS...... 323 15.3 PreservingmultivariableinvariantsusingCAS...... 326 15.4 Random number generator using ReentrantLock...... 327 15.5 Random number generator using AtomicInteger...... 327 15.6 Nonblocking stack using Treiber’s algorithm (Treiber, 1986).....331 15.7 Insertion in the Michael-Scott nonblocking queue algorithm (Michael and Scott, 1996)...... 334 15.8 Using atomic field updaters in ConcurrentLinkedQueue...... 335 16.1 Insufficiently synchronized program that can have surprising results. Don’t do this...... 340 16.2 Inner class of FutureTask illustrating synchronization piggybacking.343 16.3 Unsafe lazy initialization. Don’t do this...... 345 16.4 Thread-safelazyinitialization...... 347 16.5 Eagerinitialization...... 347 16.6 Lazyinitializationholderclassidiom...... 348 16.7 Double-checked-locking antipattern. Don’t do this...... 349 16.8 Initializationsafetyforimmutableobjects...... 350 Preface

At this writing, multicore processors are just now becoming inexpensive enough for midrange desktop systems. Not coincidentally, many development teams are noticing more and more threading-related bug reports in their projects. In a recent post on the NetBeans developer site, one of the core maintainers observed that a single class had been patched over 14 times to ﬁx threading-related problems. Dion Almaer, former editor of TheServerSide, recently blogged (after a painful debugging session that ultimately revealed a threading bug) that most Java programs are so rife with concurrency bugs that they work only “by accident”. Indeed, developing, testing and debugging multithreaded programs can be extremely difﬁcult because concurrency bugs do not manifest themselves pre- dictably. And when they do surface, it is often at the worst possible time—in production, under heavy load. One of the challenges of developing concurrent programs in Java is the mismatch between the concurrency features offered by the platform and how developers need to think about concurrency in their programs. The language provides low-level mechanisms such as synchronization and condition waits, but these mechanisms must be used consistently to implement application-level protocols or policies. Without such policies, it is all too easy to create programs that com- pile and appear to work but are nevertheless broken. Many otherwise excellent books on concurrency fall short of their goal by focusing excessively on low-level mechanisms and APIs rather than design-level policies and patterns. Java 5.0 is a huge step forward for the development of concurrent applications in Java, providing new higher-level components and additional low-level mechanisms that make it easier for novices and experts alike to build concurrent applications. The authors are the primary members of the JCP Expert Group that created these facilities; in addition to describing their behavior and features, we present the underlying design patterns and anticipated usage scenarios that motivated their inclusion in the platform libraries. Our goal is to give readers a set of design rules and mental models that make it easier—and more fun—to build correct, performant concurrent classes and applications in Java. We hope you enjoy Java Concurrency in Practice. Brian Goetz Williston, VT March 2006

xvii xviii Preface

How to use this book

To address the abstraction mismatch between Java’s low-level mechanisms and the necessary design-level policies, we present a simplified set of rules for writing concurrent programs. Experts may look at these rules and say “Hmm, that’s not entirely true: class C is thread-safe even though it violates rule R.” While it is possible to write correct programs that break our rules, doing so requires a deep understanding of the low-level details of the Java Memory Model, and we want developers to be able to write correct concurrent programs without having to master these details. Consistently following our simplified rules will produce correct and maintainable concurrent programs. We assume the reader already has some familiarity with the basic mechanisms for concurrency in Java. Java Concurrency in Practice is not an introduction to concurrency—for that, see the threading chapter of any decent introductory volume, such as The Java Programming Language (Arnold et al., 2005). Nor is it an encyclopedic reference for All Things Concurrency—for that, see Concurrent Programming in Java (Lea, 2000). Rather, it offers practical design rules to assist developers in the difficult process of creating safe and performant concurrent classes. Where appropriate, we cross-reference relevant sections of The Java Pro- gramming Language, Concurrent Programming in Java, The Java Language Specification (Gosling et al., 2005), and Effective Java (Bloch, 2001) using the conventions [JPL n.m], [CPJ n.m], [JLS n.m], and [EJ Item n]. After the introduction (Chapter 1), the book is divided into four parts: Fundamentals. Part I (Chapters 2-5) focuses on the basic concepts of concurrency and thread safety, and how to compose thread-safe classes out of the concurrent building blocks provided by the class library. A “cheat sheet” summa- rizing the most important of the rules presented in Part I appears on page 110. Chapters 2 (Thread Safety) and 3 (Sharing Objects) form the foundation for the book. Nearly all of the rules on avoiding concurrency hazards, constructing thread-safe classes, and verifying thread safety are here. Readers who prefer “practice” to “theory” may be tempted to skip ahead to Part II, but make sure to come back and read Chapters 2 and 3 before writing any concurrent code! Chapter 4 (Composing Objects) covers techniques for composing thread-safe classes into larger thread-safe classes. Chapter 5 (Building Blocks) covers the concurrent building blocks—thread-safe collections and synchronizers—provided by the platform libraries. Structuring Concurrent Applications. Part II (Chapters 6-9) describes how to exploit threads to improve the throughput or responsiveness of concurrent applications. Chapter 6 (Task Execution) covers identifying parallelizable tasks and executing them within the task-execution framework. Chapter 7 (Cancellation and Shutdown) deals with techniques for convincing tasks and threads to terminate before they would normally do so; how programs deal with cancellation and shutdown is often one of the factors that separates truly robust concurrent applications from those that merely work. Chapter 8 (Applying Thread Pools) addresses some of the more advanced features of the task-execution framework. Preface xix

Chapter 9 (GUI Applications) focuses on techniques for improving responsiveness in single-threaded subsystems. Liveness, Performance, and Testing. Part III (Chapters 10-12) concerns itself with ensuring that concurrent programs actually do what you want them to do and do so with acceptable performance. Chapter 10 (Avoiding Liveness Hazards) describes how to avoid liveness failures that can prevent programs from making forward progress. Chapter 11 (Performance and Scalability) covers techniques for improving the performance and scalability of concurrent code. Chapter 12 (Testing Concurrent Programs) covers techniques for testing concurrent code for both correctness and performance. Advanced Topics. Part IV (Chapters 13-16) covers topics that are likely to be of interest only to experienced developers: explicit locks, atomic variables, nonblocking algorithms, and developing custom synchronizers.

Code examples

While many of the general concepts in this book are applicable to versions of Java prior to Java 5.0 and even to non-Java environments, most of the code examples (and all the statements about the Java Memory Model) assume Java 5.0 or later. Some of the code examples may use library features added in Java 6. The code examples have been compressed to reduce their size and to high- light the relevant portions. The full versions of the code examples, as well as supplementary examples and errata, are available from the book’s website, http://www.javaconcurrencyinpractice.com. The code examples are of three sorts: “good” examples, “not so good” examples, and “bad” examples. Good examples illustrate techniques that should be emulated. Bad examples illustrate techniques that should deﬁnitely not be em- 1 ulated, and are identiﬁed with a “Mr. Yuk” icon to make it clear that this is “toxic” code (see Listing 1). Not-so-good examples illustrate techniques that are not necessarily wrong but are fragile, risky, or perform poorly, and are decorated with a “Mr. Could Be Happier” icon as in Listing 2. public > void sort(List list) { // Never returns the wrong answer! System.exit(0); } Listing 1. Bad way to sort a list. Don’t do this.

Some readers may question the role of the “bad” examples in this book; after all, a book should show how to do things right, not wrong. The bad examples have two purposes. They illustrate common pitfalls, but more importantly they demonstrate how to analyze a program for thread safety—and the best way to do that is to see the ways in which thread safety is compromised.

1. Mr. Yuk is a registered trademark of the Children’s Hospital of Pittsburgh and appears by permission. xx Preface public > void sort(List list) { for (int i=0; i<1000000; i++) doNothing(); Collections.sort(list); } Listing 2. Less than optimal way to sort a list.

Acknowledgments

This book grew out of the development process for the java.util.concurrent package that was created by the Java Community Process JSR 166 for inclusion in Java 5.0. Many others contributed to JSR 166; in particular we thank Martin Buch- holz for doing all the work related to getting the code into the JDK, and all the readers of the concurrency-interest mailing list who offered their suggestions and feedback on the draft APIs. This book has been tremendously improved by the suggestions and assistance of a small army of reviewers, advisors, cheerleaders, and armchair critics. We would like to thank Dion Almaer, Tracy Bialik, Cindy Bloch, Martin Buchholz, Paul Christmann, Cliff Click, Stuart Halloway, David Hovemeyer, Jason Hunter, Michael Hunter, Jeremy Hylton, Heinz Kabutz, Robert Kuhar, Ramnivas Lad- dad, Jared Levy, Nicole Lewis, Victor Luchangco, Jeremy Manson, Paul Martin, Berna Massingill, Michael Maurer, Ted Neward, Kirk Pepperdine, Bill Pugh, Sam Pullara, Russ Rufer, Bill Scherer, Jeffrey Siegal, Bruce Tate, Gil Tene, Paul Tyma, and members of the Silicon Valley Patterns Group who, through many interesting technical conversations, offered guidance and made suggestions that helped make this book better. We are especially grateful to Cliff Bifﬂe, Barry Hayes, Dawid Kurzyniec, Ange- lika Langer, Doron Rajwan, and Bill Venners, who reviewed the entire manuscript in excruciating detail, found bugs in the code examples, and suggested numerous improvements. We thank Katrina Avery for a great copy-editing job and Rosemary Simpson for producing the index under unreasonable time pressure. We thank Ami Dewar for doing the illustrations. Thanks to the whole team at Addison-Wesley who helped make this book a reality. Ann Sellers got the project launched and Greg Doench shepherded it to a smooth completion; Elizabeth Ryan guided it through the production process. We would also like to thank the thousands of software engineers who contributed indirectly by creating the software used to create this book, including TEX, LATEX, Adobe Acrobat, pic, grap, Adobe Illustrator, Perl, Apache Ant, IntelliJ IDEA, GNU emacs, Subversion, TortoiseSVN, and of course, the Java platform and class libraries. This page intentionally left blank Chapter 6 Task Execution

Most concurrent applications are organized around the execution of tasks:abstract, discrete units of work. Dividing the work of an application into tasks simpliﬁes program organization, facilitates error recovery by providing natural transaction boundaries, and promotes concurrency by providing a natural structure for parallelizing work.

6.1 Executing tasks in threads

The first step in organizing a program around task execution is identifying sensible task boundaries. Ideally, tasks are independent activities: work that doesn’t depend on the state, result, or side effects of other tasks. Independence facilitates concurrency, as independent tasks can be executed in parallel if there are adequate processing resources. For greater flexibility in scheduling and load bal- ancing tasks, each task should also represent a small fraction of your application’s processing capacity. Server applications should exhibit both good throughput and good responsiveness under normal load. Application providers want applications to support as many users as possible, so as to reduce provisioning costs per user; users want to get their response quickly. Further, applications should exhibit graceful degradation as they become overloaded, rather than simply falling over under heavy load. Choosing good task boundaries, coupled with a sensible task execution policy (see Section 6.2.2), can help achieve these goals. Most server applications offer a natural choice of task boundary: individual client requests. Web servers, mail servers, file servers, EJB containers, and database servers all accept requests via network connections from remote clients. Using individual requests as task boundaries usually offers both independence and appropriate task sizing. For example, the result of submitting a message to a mail server is not affected by the other messages being processed at the same time, and handling a single message usually requires a very small percentage of the server’s total capacity.

113 114 Chapter 6. Task Execution

6.1.1 Executing tasks sequentially There are a number of possible policies for scheduling tasks within an application, some of which exploit the potential for concurrency better than others. The simplest is to execute tasks sequentially in a single thread. SingleThreadWeb- Server in Listing 6.1 processes its tasks—HTTP requests arriving on port 80— sequentially. The details of the request processing aren’t important; we’re inter- ested in characterizing the concurrency of various scheduling policies. class SingleThreadWebServer { public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { Socket connection = socket.accept(); handleRequest(connection); } } } Listing 6.1. Sequential web server.

SingleThreadedWebServer is simple and theoretically correct, but would perform poorly in production because it can handle only one request at a time. The main thread alternates between accepting connections and processing the associated request. While the server is handling a request, new connections must wait until it ﬁnishes the current request and calls accept again. This might work if request processing were so fast that handleRequest effectively returned immediately, but this doesn’t describe any web server in the real world. Processing a web request involves a mix of computation and I/O. The server must perform socket I/O to read the request and write the response, which can block due to network congestion or connectivity problems. It may also perform ﬁle I/O or make database requests, which can also block. In a single-threaded server, blocking not only delays completing the current request, but prevents pending requests from being processed at all. If one request blocks for an un- usually long time, users might think the server is unavailable because it appears unresponsive. At the same time, resource utilization is poor, since the CPU sits idle while the single thread waits for its I/O to complete. In server applications, sequential processing rarely provides either good throughput or good responsiveness. There are exceptions—such as when tasks are few and long-lived, or when the server serves a single client that makes only 1 a single request at a time—but most server applications do not work this way.

1. In some situations, sequential processing may offer a simplicity or safety advantage; most GUI frameworks process tasks sequentially using a single thread. We return to the sequential model in Chapter 9. 6.1. Executing tasks in threads 115

6.1.2 Explicitly creating threads for tasks A more responsive approach is to create a new thread for servicing each request, as shown in ThreadPerTaskWebServer in Listing 6.2. class ThreadPerTaskWebServer { public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); } } } Listing 6.2. Web server that starts a new thread for each request.

ThreadPerTaskWebServer is similar in structure to the single-threaded version—the main thread still alternates between accepting an incoming connection and dispatching the request. The difference is that for each connection, the main loop creates a new thread to process the request instead of processing it within the main thread. This has three main consequences:

• Task processing is ofﬂoaded from the main thread, enabling the main loop to resume waiting for the next incoming connection more quickly. This enables new connections to be accepted before previous requests complete, improving responsiveness. • Tasks can be processed in parallel, enabling multiple requests to be serviced simultaneously. This may improve throughput if there are multiple processors, or if tasks need to block for any reason such as I/O completion, lock acquisition, or resource availability. • Task-handling code must be thread-safe, because it may be invoked concurrently for multiple tasks.

Under light to moderate load, the thread-per-task approach is an improvement over sequential execution. As long as the request arrival rate does not exceed the server’s capacity to handle requests, this approach offers better responsiveness and throughput. 116 Chapter 6. Task Execution

6.1.3 Disadvantages of unbounded thread creation For production use, however, the thread-per-task approach has some practical drawbacks, especially when a large number of threads may be created:

Thread lifecycle overhead. Thread creation and teardown are not free. The ac- tual overhead varies across platforms, but thread creation takes time, intro- ducing latency into request processing, and requires some processing activity by the JVM and OS. If requests are frequent and lightweight, as in most server applications, creating a new thread for each request can consume signiﬁcant computing resources. Resource consumption. Active threads consume system resources, especially memory. When there are more runnable threads than available processors, threads sit idle. Having many idle threads can tie up a lot of memory, putting pressure on the garbage collector, and having many threads competing for the CPUs can impose other performance costs as well. If you have enough threads to keep all the CPUs busy, creating more threads won’t help and may even hurt. Stability. There is a limit on how many threads can be created. The limit varies by platform and is affected by factors including JVM invocation parameters, the requested stack size in the Thread constructor, and limits on threads 2 placed by the underlying operating system. When you hit this limit, the most likely result is an OutOfMemoryError. Trying to recover from such an error is very risky; it is far easier to structure your program to avoid hitting this limit.

Up to a certain point, more threads can improve throughput, but beyond that point creating more threads just slows down your application, and creating one thread too many can cause your entire application to crash horribly. The way to stay out of danger is to place some bound on how many threads your application creates, and to test your application thoroughly to ensure that, even when this bound is reached, it does not run out of resources. The problem with the thread-per-task approach is that nothing places any limit on the number of threads created except the rate at which remote users can throw HTTP requests at it. Like other concurrency hazards, unbounded thread creation may appear to work just ﬁne during prototyping and development, with problems surfacing only when the application is deployed and under heavy load. So a malicious user, or enough ordinary users, can make your web server crash if the trafﬁc load ever reaches a certain threshold. For a server application that is supposed to provide high availability and graceful degradation under load, this is a serious failing.

2.On32-bit machines, a major limiting factor is address space for thread stacks. Each thread main- tains two execution stacks, one for Java code and one for native code. Typical JVM defaults yield a combined stack size of around half a megabyte. (You can change this with the -Xss JVM ﬂag or through the Thread constructor.) If you divide the per-thread stack size into 232, you get a limit of a few thousands or tens of thousands of threads. Other factors, such as OS limitations, may impose stricter limits. 6.2.TheExecutor framework 117

6.2 The Executor framework

Tasks are logical units of work, and threads are a mechanism by which tasks can run asynchronously. We’ve examined two policies for executing tasks using threads—execute tasks sequentially in a single thread, and execute each task in its own thread. Both have serious limitations: the sequential approach suffers from poor responsiveness and throughput, and the thread-per-task approach suffers from poor resource management. In Chapter 5,wesawhowtousebounded queues to prevent an overloaded application from running out of memory. Thread pools offer the same beneﬁt for thread management, and java.util.concurrent provides a ﬂexible thread pool implementation as part of the Executor framework. The primary abstraction for task execution in the Java class libraries is not Thread,butExecutor,shownin Listing 6.3. public interface Executor { void execute(Runnable command); } Listing 6.3. Executor interface.

Executor may be a simple interface, but it forms the basis for a ﬂexible and powerful framework for asynchronous task execution that supports a wide variety of task execution policies. It provides a standard means of decoupling task submission from task execution, describing tasks with Runnable.TheExecutor implementations also provide lifecycle support and hooks for adding statistics gathering, application management, and monitoring. Executor is based on the producer-consumer pattern, where activities that submit tasks are the producers (producing units of work to be done) and the threads that execute tasks are the consumers (consuming those units of work). Using an Executor is usually the easiest path to implementing a producer-consumer design in your application.

6.2.1 Example: web server using Executor Building a web server with an Executor is easy. TaskExecutionWebServer in Listing 6.4 replaces the hard-coded thread creation with an Executor.Inthis case, we use one of the standard Executor implementations, a fixed-size thread pool with 100 threads. In TaskExecutionWebServer, submission of the request-handling task is de- coupled from its execution using an Executor, and its behavior can be changed merely by substituting a different Executor implementation. Changing Executor implementations or configuration is far less invasive than changing the way tasks are submitted; Executor configuration is generally a one-time event and can easily be exposed for deployment-time configuration, whereas task submission code tends to be strewn throughout the program and harder to expose. 118 Chapter 6. Task Execution class TaskExecutionWebServer { private static final int NTHREADS = 100; private static final Executor exec = Executors.newFixedThreadPool(NTHREADS);

public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; exec.execute(task); } } }

Listing 6.4. Web server using a thread pool.

We can easily modify TaskExecutionWebServer to behave like ThreadPer- TaskWebServer by substituting an Executor that creates a new thread for each request. Writing such an Executor is trivial, as shown in ThreadPerTaskExecut- or in Listing 6.5. public class ThreadPerTaskExecutor implements Executor { public void execute(Runnable r) { new Thread(r).start(); }; } Listing 6.5. Executor that starts a new thread for each task.

Similarly, it is also easy to write an Executor that would make TaskExecut- ionWebServer behave like the single-threaded version, executing each task synchronously before returning from execute,asshowninWithinThreadExecutor in Listing 6.6.

6.2.2 Execution policies The value of decoupling submission from execution is that it lets you easily specify, and subsequently change without great difﬁculty, the execution policy for a given class of tasks. An execution policy speciﬁes the “what, where, when, and how” of task execution, including: 6.2.TheExecutor framework 119 public class WithinThreadExecutor implements Executor { public void execute(Runnable r) { r.run(); }; } Listing 6.6. Executor that executes tasks synchronously in the calling thread.

• In what thread will tasks be executed? • In what order should tasks be executed (FIFO, LIFO, priority order)? • How many tasks may execute concurrently? • How many tasks may be queued pending execution? • If a task has to be rejected because the system is overloaded, which task should be selected as the victim, and how should the application be noti- ﬁed? • What actions should be taken before or after executing a task?

Execution policies are a resource management tool, and the optimal policy depends on the available computing resources and your quality-of-service requirements. By limiting the number of concurrent tasks, you can ensure that the application does not fail due to resource exhaustion or suffer performance 3 problems due to contention for scarce resources. Separating the speciﬁcation of execution policy from task submission makes it practical to select an execution policy at deployment time that is matched to the available hardware.

Whenever you see code of the form: new Thread(runnable).start() and you think you might at some point want a more ﬂexible execution policy, seriously consider replacing it with the use of an Executor.

6.2.3 Thread pools A thread pool, as its name suggests, manages a homogeneous pool of worker threads. A thread pool is tightly bound to a work queue holding tasks waiting to be executed. Worker threads have a simple life: request the next task from the work queue, execute it, and go back to waiting for another task.

3. This is analogous to one of the roles of a transaction monitor in an enterprise application: it can throttle the rate at which transactions are allowed to proceed so as not to exhaust or overstress limited resources. 120 Chapter 6. Task Execution

Executing tasks in pool threads has a number of advantages over the thread- per-task approach. Reusing an existing thread instead of creating a new one amortizes thread creation and teardown costs over multiple requests. As an added bonus, since the worker thread often already exists at the time the request arrives, the latency associated with thread creation does not delay task execution, thus improving responsiveness. By properly tuning the size of the thread pool, you can have enough threads to keep the processors busy while not having so many that your application runs out of memory or thrashes due to competition among threads for resources. The class library provides a flexible thread pool implementation along with some useful predefined configurations. You can create a thread pool by calling one of the static factory methods in Executors: newFixedThreadPool. A fixed-size thread pool creates threads as tasks are submitted, up to the maximum pool size, and then attempts to keep the pool size constant (adding new threads if a thread dies due to an unexpected Exception). newCachedThreadPool. A cached thread pool has more flexibility to reap idle threads when the current size of the pool exceeds the demand for processing, and to add new threads when demand increases, but places no bounds on the size of the pool. newSingleThreadExecutor. A single-threaded executor creates a single worker thread to process tasks, replacing it if it dies unexpectedly. Tasks are guaranteed to be processed sequentially according to the order imposed by the 4 task queue (FIFO, LIFO, priority order). newScheduledThreadPool. A fixed-size thread pool that supports delayed and periodic task execution, similar to Timer.(SeeSection6.2.5.)

The newFixedThreadPool and newCachedThreadPool factories return in- stances of the general-purpose ThreadPoolExecutor, which can also be used directly to construct more specialized executors. We discuss thread pool conﬁgu- ration options in depth in Chapter 8. The web server in TaskExecutionWebServer uses an Executor with a bounded pool of worker threads. Submitting a task with execute adds the task to the work queue, and the worker threads repeatedly dequeue tasks from the work queue and execute them. Switching from a thread-per-task policy to a pool-based policy has a big effect 5 on application stability: the web server will no longer fail under heavy load.

4. Single-threaded executors also provide sufﬁcient internal synchronization to guarantee that any memory writes made by tasks are visible to subsequent tasks; this means that objects can be safely conﬁned to the “task thread” even though that thread may be replaced with another from time to time. 5. While the server may not fail due to the creation of too many threads, if the task arrival rate exceeds the task service rate for long enough it is still possible (just harder) to run out of memory because of the growing queue of Runnables awaiting execution. This can be addressed within the Executor framework by using a bounded work queue—see Section 8.3.2. 6.2.TheExecutor framework 121

It also degrades more gracefully, since it does not create thousands of threads that compete for limited CPU and memory resources. And using an Executor opens the door to all sorts of additional opportunities for tuning, management, monitoring, logging, error reporting, and other possibilities that would have been far more difﬁcult to add without a task execution framework.

6.2.4 Executor lifecycle We’ve seen how to create an Executor but not how to shut one down. An Exec- utor implementation is likely to create threads for processing tasks. But the JVM can’t exit until all the (nondaemon) threads have terminated, so failing to shut down an Executor could prevent the JVM from exiting. Because an Executor processes tasks asynchronously, at any given time the state of previously submitted tasks is not immediately obvious. Some may have completed, some may be currently running, and others may be queued awaiting execution. In shutting down an application, there is a spectrum from graceful shutdown (ﬁnish what you’ve started but don’t accept any new work) to abrupt shutdown (turn off the power to the machine room), and various points in between. Since Executors provide a service to applications, they should be able to be shut down as well, both gracefully and abruptly, and feed back information to the application about the status of tasks that were affected by the shutdown. To address the issue of execution service lifecycle, the ExecutorService interface extends Executor, adding a number of methods for lifecycle management (as well as some convenience methods for task submission). The lifecycle management methods of ExecutorService are shown in Listing 6.7. public interface ExecutorService extends Executor { void shutdown(); List shutdownNow(); boolean isShutdown(); boolean isTerminated(); boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException; // ... additional convenience methods for task submission } Listing 6.7. Lifecycle methods in ExecutorService.

The lifecycle implied by ExecutorService has three states—running, shutting down,andterminated. ExecutorServices are initially created in the running state. The shutdown method initiates a graceful shutdown: no new tasks are accepted but previously submitted tasks are allowed to complete—including those that have not yet begun execution. The shutdownNow method initiates an abrupt shutdown: it attempts to cancel outstanding tasks and does not start any tasks that are queued but not begun. Tasks submitted to an ExecutorService after it has been shut down are han- dled by the rejected execution handler (see Section 8.3.3), which might silently dis- 122 Chapter 6. Task Execution card the task or might cause execute to throw the unchecked RejectedExecu- tionException. Once all tasks have completed, the ExecutorService transitions to the terminated state. You can wait for an ExecutorService to reach the terminated state with awaitTermination, or poll for whether it has yet terminated with isTerminated. It is common to follow shutdown immediately by awaitTermina- tion, creating the effect of synchronously shutting down the ExecutorService. (Executor shutdown and task cancellation are covered in more detail in Chapter 7.) LifecycleWebServer in Listing 6.8 extends our web server with lifecycle support. It can be shut down in two ways: programmatically by calling stop,and through a client request by sending the web server a specially formatted HTTP request. class LifecycleWebServer { private final ExecutorService exec = ...;

public void start() throws IOException { ServerSocket socket = new ServerSocket(80); while (!exec.isShutdown()) { try { final Socket conn = socket.accept(); exec.execute(new Runnable() { public void run() { handleRequest(conn); } }); } catch (RejectedExecutionException e) { if (!exec.isShutdown()) log("task submission rejected", e); } } }

public void stop() { exec.shutdown(); }

void handleRequest(Socket connection) { Request req = readRequest(connection); if (isShutdownRequest(req)) stop(); else dispatchRequest(req); } }

Listing 6.8. Web server with shutdown support. 6.3. Finding exploitable parallelism 123

6.2.5 Delayed and periodic tasks The Timer facility manages the execution of deferred (“run this task in 100 ms”) and periodic (“run this task every 10 ms”) tasks. However, Timer has some drawbacks, and ScheduledThreadPoolExecutor should be thought of as its replace- 6 ment. You can construct a ScheduledThreadPoolExecutor through its constructor or through the newScheduledThreadPool factory. A Timer creates only a single thread for executing timer tasks. If a timer task takes too long to run, the timing accuracy of other TimerTaskscansuffer. If a recurring TimerTask is scheduled to run every 10 ms and another Timer- Task takes 40 ms to run, the recurring task either (depending on whether it was scheduled at ﬁxed rate or ﬁxed delay) gets called four times in rapid succession after the long-running task completes, or “misses” four invocations completely. Scheduled thread pools address this limitation by letting you provide multiple threads for executing deferred and periodic tasks. Another problem with Timer is that it behaves poorly if a TimerTask throws an unchecked exception. The Timer thread doesn’t catch the exception, so an unchecked exception thrown from a TimerTask terminates the timer thread. Timer also doesn’t resurrect the thread in this situation; instead, it erroneously assumes the entire Timer was cancelled. In this case, TimerTasks that are already scheduled but not yet executed are never run, and new tasks cannot be scheduled. (This problem, called “thread leakage” is described in Section 7.3, along with techniques for avoiding it.) OutOfTime in Listing 6.9 illustrates how a Timer can become confused in this manner and, as confusion loves company, how the Timer shares its confusion with the next hapless caller that tries to submit a TimerTask. You might expect the program to run for six seconds and exit, but what actually happens is that it terminates after one second with an IllegalStateException whose message text is “Timer already cancelled”. ScheduledThreadPoolExecutor deals properly with ill-behaved tasks; there is little reason to use Timer in Java 5.0 or later. If you need to build your own scheduling service, you may still be able to take advantage of the library by using a DelayQueue,aBlockingQueue implementation that provides the scheduling functionality of ScheduledThreadPoolExecutor.A DelayQueue manages a collection of Delayed objects. A Delayed has a delay time associated with it: DelayQueue lets you take an element only if its delay has expired. Objects are returned from a DelayQueue ordered by the time associated with their delay.

6.3 Finding exploitable parallelism

The Executor framework makes it easy to specify an execution policy, but in order to use an Executor, you have to be able to describe your task as a Runn- able. In most server applications, there is an obvious task boundary: a single client request. But sometimes good task boundaries are not quite so obvious, as

6. Timer does have support for scheduling based on absolute, not relative time, so that tasks can be sensitive to changes in the system clock; ScheduledThreadPoolExecutor supports only relative time. 124 Chapter 6. Task Execution public class OutOfTime { public static void main(String[] args) throws Exception { Timer timer = new Timer(); timer.schedule(new ThrowTask(), 1); SECONDS.sleep(1); timer.schedule(new ThrowTask(), 1); SECONDS.sleep(5); }

static class ThrowTask extends TimerTask { public void run() { throw new RuntimeException(); } } } Listing 6.9. Class illustrating confusing Timer behavior.

in many desktop applications. There may also be exploitable parallelism within a single client request in server applications, as is sometimes the case in database servers. (For a further discussion of the competing design forces in choosing task boundaries, see [CPJ 4.4.1.1].) In this section we develop several versions of a component that admit varying degrees of concurrency. Our sample component is the page-rendering portion of a browser application, which takes a page of HTML and renders it into an image buffer. To keep it simple, we assume that the HTML consists only of marked up text interspersed with image elements with pre-speciﬁed dimensions and URLs.

6.3.1 Example: sequential page renderer The simplest approach is to process the HTML document sequentially. As text markup is encountered, render it into the image buffer; as image references are encountered, fetch the image over the network and draw it into the image buffer as well. This is easy to implement and requires touching each element of the input only once (it doesn’t even require buffering the document), but is likely to annoy the user, who may have to wait a long time before all the text is rendered. A less annoying but still sequential approach involves rendering the text elements ﬁrst, leaving rectangular placeholders for the images, and after completing the initial pass on the document, going back and downloading the images and drawing them into the associated placeholder. This approach is shown in Sin- gleThreadRenderer in Listing 6.10. Downloading an image mostly involves waiting for I/O to complete, and during this time the CPU does little work. So the sequential approach may under- utilize the CPU, and also makes the user wait longer than necessary to see the ﬁnished page. We can achieve better utilization and responsiveness by breaking the problem into independent tasks that can execute concurrently. 6.3. Finding exploitable parallelism 125 public class SingleThreadRenderer { void renderPage(CharSequence source) { renderText(source); List imageData = new ArrayList(); for (ImageInfo imageInfo : scanForImageInfo(source)) imageData.add(imageInfo.downloadImage()); for (ImageData data : imageData) renderImage(data); } } Listing 6.10. Rendering page elements sequentially.

6.3.2 Result-bearing tasks: Callable and Future The Executor framework uses Runnable as its basic task representation. Runn- able is a fairly limiting abstraction; run cannot return a value or throw checked exceptions, although it can have side effects such as writing to a log file or placing a result in a shared data structure. Many tasks are effectively deferred computations—executing a database query, fetching a resource over the network, or computing a complicated func- tion. For these types of tasks, Callable is a better abstraction: it expects that the main entry point, call, will return a value and anticipates that it might throw 7 an exception. Executors includes several utility methods for wrapping other types of tasks, including Runnable and java.security.PrivilegedAction,with a Callable. Runnable and Callable describe abstract computational tasks. Tasks are usually finite: they have a clear starting point and they eventually terminate. The lifecycle of a task executed by an Executor has four phases: created, submitted, started,andcompleted. Since tasks can take a long time to run, we also want to be able to cancel a task. In the Executor framework, tasks that have been submitted but not yet started can always be cancelled, and tasks that have started can sometimes be cancelled if they are responsive to interruption. Cancelling a task that has already completed has no effect. (Cancellation is covered in greater detail in Chapter 7.) Future represents the lifecycle of a task and provides methods to test whether the task has completed or been cancelled, retrieve its result, and cancel the task. Callable and Future are shown in Listing 6.11. Implicit in the specification of Future is that task lifecycle can only move forwards, not backwards—just like the ExecutorService lifecycle. Once a task is completed, it stays in that state forever. The behavior of get varies depending on the task state (not yet started, running, completed). It returns immediately or throws an Exception if the task has already completed, but if not it blocks until the task completes. If the task completes by throwing an exception, get rethrows it wrapped in an Execution-

7. To express a non-value-returning task with Callable,useCallable. 126 Chapter 6. Task Execution public interface Callable { V call() throws Exception; } public interface Future { boolean cancel(boolean mayInterruptIfRunning); boolean isCancelled(); boolean isDone(); V get() throws InterruptedException, ExecutionException, CancellationException; V get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, CancellationException, TimeoutException; } Listing 6.11. Callable and Future interfaces.

Exception; if it was cancelled, get throws CancellationException.Ifget throws ExecutionException, the underlying exception can be retrieved with getCause. There are several ways to create a Future to describe a task. The submit methods in ExecutorService all return a Future, so that you can submit a Runn- able or a Callable to an executor and get back a Future that can be used to retrieve the result or cancel the task. You can also explicitly instantiate a Fut- ureTask for a given Runnable or Callable. (Because FutureTask implements Runnable, it can be submitted to an Executor for execution or executed directly by calling its run method.) As of Java 6, ExecutorService implementations can override newTaskFor in AbstractExecutorService to control instantiation of the Future corresponding to a submitted Callable or Runnable. The default implementation just creates a new FutureTask, as shown in Listing 6.12. protected RunnableFuture newTaskFor(Callable task) { return new FutureTask(task); } Listing 6.12. Default implementation of newTaskFor in ThreadPoolExecutor.

Submitting a Runnable or Callable to an Executor constitutes a safe publication (see Section 3.5)oftheRunnable or Callable from the submitting thread to the thread that will eventually execute the task. Similarly, setting the result value for a Future constitutes a safe publication of the result from the thread in which it was computed to any thread that retrieves it via get. 6.3. Finding exploitable parallelism 127

6.3.3 Example: page renderer with Future As a ﬁrst step towards making the page renderer more concurrent, let’s divide it into two tasks, one that renders the text and one that downloads all the images. (Because one task is largely CPU-bound and the other is largely I/O-bound, this approach may yield improvements even on single-CPU systems.) Callable and Future can help us express the interaction between these cooperating tasks. In FutureRenderer in Listing 6.13,wecreateaCallable to download all the images, and submit it to an ExecutorService. This returns a Future describing the task’s execution; when the main task gets to the point where it needs the images, it waits for the result by calling Future.get. If we’re lucky, the results will already be ready by the time we ask; otherwise, at least we got a head start on downloading the images. The state-dependent nature of get means that the caller need not be aware of the state of the task, and the safe publication properties of task submission and result retrieval make this approach thread-safe. The exception handling code surrounding Future.get deals with two possible problems: that the task encountered an Exception, or the thread calling get was interrupted before the results were available. (See Sections 5.5.2 and 5.4.) FutureRenderer allows the text to be rendered concurrently with downloading the image data. When all the images are downloaded, they are rendered onto the page. This is an improvement in that the user sees a result quickly and it exploits some parallelism, but we can do considerably better. There is no need for users to wait for all the images to be downloaded; they would probably prefer to see individual images drawn as they become available.

6.3.4 Limitations of parallelizing heterogeneous tasks In the last example, we tried to execute two different types of tasks in parallel— downloading the images and rendering the page. But obtaining significant performance improvements by trying to parallelize sequential heterogeneous tasks can be tricky. Two people can divide the work of cleaning the dinner dishes fairly effectively: one person washes while the other dries. However, assigning a different type of task to each worker does not scale well; if several more people show up, it is not obvious how they can help without getting in the way or significantly restructuring the division of labor. Without finding finer-grained parallelism among similar tasks, this approach will yield diminishing returns. A further problem with dividing heterogeneous tasks among multiple workers is that the tasks may have disparate sizes. If you divide tasks A and B between two workers but A takes ten times as long as B, you’ve only speeded up the total process by 9%. Finally, dividing a task among multiple workers always involves some amount of coordination overhead; for the division to be worthwhile, this overhead must be more than compensated by productivity improvements due to parallelism. FutureRenderer uses two tasks: one for rendering text and one for downloading the images. If rendering the text is much faster than downloading the images, 128 Chapter 6. Task Execution

public class FutureRenderer { private final ExecutorService executor = ...;

void renderPage(CharSequence source) { final List imageInfos = scanForImageInfo(source); Callable> task = new Callable>() { public List call() { List result = new ArrayList(); for (ImageInfo imageInfo : imageInfos) result.add(imageInfo.downloadImage()); return result; } };

Future> future = executor.submit(task); renderText(source);

try { List imageData = future.get(); for (ImageData data : imageData) renderImage(data); } catch (InterruptedException e) { // Re-assert the thread’s interrupted status Thread.currentThread().interrupt(); // We don’t need the result, so cancel the task too future.cancel(true); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } } Listing 6.13. Waiting for image download with Future. 6.3. Finding exploitable parallelism 129 as is entirely possible, the resulting performance is not much different from the sequential version, but the code is a lot more complicated. And the best we can do with two threads is speed things up by a factor of two. Thus, trying to increase concurrency by parallelizing heterogeneous activities can be a lot of work, and there is a limit to how much additional concurrency you can get out of it. (See Sections 11.4.2 and 11.4.3 for another example of the same phenomenon.)

The real performance payoﬀ of dividing a program’s workload into tasks comes when there are a large number of independent, homogeneous tasks that can be processed concurrently.

6.3.5 CompletionService: Executor meets BlockingQueue If you have a batch of computations to submit to an Executor and you want to retrieve their results as they become available, you could retain the Future associated with each task and repeatedly poll for completion by calling get with a timeout of zero. This is possible, but tedious. Fortunately there is a better way: a completion service. CompletionService combines the functionality of an Executor and a Block- ingQueue.YoucansubmitCallable tasks to it for execution and use the queue- like methods take and poll to retrieve completed results, packaged as Futures, as they become available. ExecutorCompletionService implements Completion- Service, delegating the computation to an Executor. The implementation of ExecutorCompletionService is quite straightforward. The constructor creates a BlockingQueue to hold the completed results. Future- Task has a done method that is called when the computation completes. When a task is submitted, it is wrapped with a QueueingFuture, a subclass of FutureTask that overrides done to place the result on the BlockingQueue, as shown in Listing 6.14.Thetake and poll methods delegate to the BlockingQueue, blocking if results are not yet available. private class QueueingFuture extends FutureTask { QueueingFuture(Callable c) { super(c); } QueueingFuture(Runnable t, V r) { super(t, r); }

protected void done() { completionQueue.add(this); } } Listing 6.14. QueueingFuture class used by ExecutorCompletionService. 130 Chapter 6. Task Execution

6.3.6 Example: page renderer with CompletionService

We can use a CompletionService to improve the performance of the page renderer in two ways: shorter total runtime and improved responsiveness. We can create a separate task for downloading each image and execute them in a thread pool, turning the sequential download into a parallel one: this reduces the amount of time to download all the images. And by fetching results from the CompletionService and rendering each image as soon as it is available, we can give the user a more dynamic and responsive user interface. This implementation is shown in Renderer in Listing 6.15. public class Renderer { private final ExecutorService executor;

Renderer(ExecutorService executor) { this.executor = executor; }

void renderPage(CharSequence source) { List info = scanForImageInfo(source); CompletionService completionService = new ExecutorCompletionService(executor); for (final ImageInfo imageInfo : info) completionService.submit(new Callable() { public ImageData call() { return imageInfo.downloadImage(); } });

renderText(source);

try { for(intt=0,n=info.size();t f = completionService.take(); ImageData imageData = f.get(); renderImage(imageData); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } }

Listing 6.15.UsingCompletionService to render page elements as they become available.

Multiple ExecutorCompletionServices can share a single Executor,soitis 6.3. Finding exploitable parallelism 131 perfectly sensible to create an ExecutorCompletionService that is private to a particular computation while sharing a common Executor. When used in this way, a CompletionService acts as a handle for a batch of computations in much the same way that a Future acts as a handle for a single computation. By remem- bering how many tasks were submitted to the CompletionService and counting how many completed results are retrieved, you can know when all the results for a given batch have been retrieved, even if you use a shared Executor.

6.3.7 Placing time limits on tasks Sometimes, if an activity does not complete within a certain amount of time, the result is no longer needed and the activity can be abandoned. For example, a web application may fetch its advertisements from an external ad server, but if the ad is not available within two seconds, it instead displays a default advertisement so that ad unavailability does not undermine the site’s responsiveness requirements. Similarly, a portal site may fetch data in parallel from multiple data sources, but may be willing to wait only a certain amount of time for data to be available before rendering the page without it. The primary challenge in executing tasks within a time budget is making sure that you don’t wait longer than the time budget to get an answer or ﬁnd out that one is not forthcoming. The timed version of Future.get supports this requirement: it returns as soon as the result is ready, but throws TimeoutExcep- tion if the result is not ready within the timeout period. A secondary problem when using timed tasks is to stop them when they run out of time, so they do not waste computing resources by continuing to compute a result that will not be used. This can be accomplished by having the task strictly manage its own time budget and abort if it runs out of time, or by cancelling the task if the timeout expires. Again, Future can help; if a timed get completes with a TimeoutException, you can cancel the task through the Future.Ifthetaskis writtentobecancellable(seeChapter7), it can be terminated early so as not to consume excessive resources. This technique is used in Listings 6.13 and 6.16. Listing 6.16 shows a typical application of a timed Future.get. It generates a composite web page that contains the requested content plus an advertisement fetched from an ad server. It submits the ad-fetching task to an executor, com- putes the rest of the page content, and then waits for the ad until its time budget 8 9 runs out. If the get times out, it cancels the ad-fetching task and uses a default advertisement instead.

6.3.8 Example: a travel reservations portal The time-budgeting approach in the previous section can be easily generalized to an arbitrary number of tasks. Consider a travel reservation portal: the user en-

8. The timeout passed to get is computed by subtracting the current time from the deadline; this may in fact yield a negative number, but all the timed methods in java.util.concurrent treat negative timeouts as zero, so no extra code is needed to deal with this case. 9.Thetrue parameter to Future.cancel means that the task thread can be interrupted if the task is currently running; see Chapter 7. 132 Chapter 6. Task Execution

Page renderPageWithAd() throws InterruptedException { long endNanos = System.nanoTime() + TIME_BUDGET; Future f = exec.submit(new FetchAdTask()); // Render the page while waiting for the ad Page page = renderPageBody(); Ad ad; try { // Only wait for the remaining time budget long timeLeft = endNanos - System.nanoTime(); ad = f.get(timeLeft, NANOSECONDS); } catch (ExecutionException e) { ad = DEFAULT_AD; } catch (TimeoutException e) { ad = DEFAULT_AD; f.cancel(true); } page.setAd(ad); return page; } Listing 6.16. Fetching an advertisement with a time budget.

ters travel dates and requirements and the portal fetches and displays bids from a number of airlines, hotels or car rental companies. Depending on the company, fetching a bid might involve invoking a web service, consulting a database, performing an EDI transaction, or some other mechanism. Rather than have the response time for the page be driven by the slowest response, it may be preferable to present only the information available within a given time budget. For providers that do not respond in time, the page could either omit them completely or display a placeholder such as “Did not hear from Air Java in time.” Fetching a bid from one company is independent of fetching bids from another, so fetching a single bid is a sensible task boundary that allows bid retrieval to proceed concurrently. It would be easy enough to create n tasks, submit them to a thread pool, retain the Futures, and use a timed get to fetch each result sequentially via its Future, but there is an even easier way—invokeAll. Listing 6.17 uses the timed version of invokeAll to submit multiple tasks to an ExecutorService and retrieve the results. The invokeAll method takes a collection of tasks and returns a collection of Futures. The two collections have identical structures; invokeAll adds the Futures to the returned collection in the order imposed by the task collection’s iterator, thus allowing the caller to associate a Future with the Callable it represents. The timed version of invokeAll will return when all the tasks have completed, the calling thread is interrupted, or the timeout expires. Any tasks that are not complete when the timeout expires are cancelled. On return from invokeAll, each task will have either completed normally or been cancelled; the client code can call get or isCancelled to ﬁnd 6.3. Finding exploitable parallelism 133 out which.

Summary

Structuring applications around the execution of tasks can simplify development and facilitate concurrency. The Executor framework permits you to decouple task submission from execution policy and supports a rich variety of execution policies; whenever you find yourself creating threads to perform tasks, consider using an Executor instead. To maximize the benefit of decomposing an application into tasks, you must identify sensible task boundaries. In some applications, the obvious task boundaries work well, whereas in others some analysis may be required to uncover finer-grained exploitable parallelism. 134 Chapter 6. Task Execution

private class QuoteTask implements Callable { private final TravelCompany company; private final TravelInfo travelInfo; ... public TravelQuote call() throws Exception { return company.solicitQuote(travelInfo); } } public List getRankedTravelQuotes( TravelInfo travelInfo, Set companies, Comparator ranking, long time, TimeUnit unit) throws InterruptedException { List tasks = new ArrayList(); for (TravelCompany company : companies) tasks.add(new QuoteTask(company, travelInfo));

List> futures = exec.invokeAll(tasks, time, unit);

List quotes = new ArrayList(tasks.size()); Iterator taskIter = tasks.iterator(); for (Future f : futures) { QuoteTask task = taskIter.next(); try { quotes.add(f.get()); } catch (ExecutionException e) { quotes.add(task.getFailureQuote(e.getCause())); } catch (CancellationException e) { quotes.add(task.getTimeoutQuote(e)); } }

Collections.sort(quotes, ranking); return quotes; } Listing 6.17. Requesting travel quotes under a time budget. This page intentionally left blank Index

Symbols acquisition of locks 64-bit operations See locks, acquisition; nonatomic nature of; 36 action(s) See also compound actions; condi- A tion, predicate; control flow; ABA problem; 336 task(s); abnormal thread termination barrier; 99 handling; 161–163 JMM specification; 339–342 abort saturation policy; 174 listener; 195–197 See also lifecycle; termination; activity(s) abrupt shutdown See also task(s); limitations; 158–161 cancellation; 135, 135–150 triggers for; 164 tasks as representation of; 113 vs. graceful shutdown; 153 ad-hoc thread confinement; 43 AbstractExecutorService See also confinement; task representation use; 126 algorithm(s) abstractions See also design patterns; idioms; rep- See models/modeling; representa- resentation; tion; comparing performance; 263–264 AbstractQueuedSynchronizer design role of representation; 104 See AQS framework; lock-free; 329 access Michael-Scott nonblocking queue; See also encapsulation; sharing; visi- 332 bility; nonblocking; 319, 329, 329–336 231 exclusive backoff importance for; fn and concurrent collections; 86 synchronization; 319–336 174 integrity SynchronousQueue; fn nonblocking algorithm use; 319 parallel iterative mutable state barrier use in; 99 importance of coordinating; 110 recursive remote resource parallelizing; 181–188 as long-running GUI task; 195 Treiber’s 331 serialized nonblocking stack; li 227 WorkerThread example; li work stealing 27 vs. object serialization; fn deques and; 92 visibility role in; 33 alien method; 40 AccessControlContext See also untrusted code behavior; custom thread factory handling; 177 deadlock risks posed by; 211 publication risks; 40

359 360 Index allocation ArrayBlockingQueue; 89 advantages vs. synchronization; 242 as bounded buffer example; 292 48 immutable objects and; fn performance advantages over object pool use BoundedBuffer; 263 disadvantages of; 241 ArrayDeque; 92 scalability advantages; 263 arrays Amdahl’s law; 225–229 See also collections; data struc- See also concurrent/concurrency; ture(s); performance; resource(s); atomic variable; 325 throughput; utilization; asymmetric two-party tasks lock scope reduction advantage; 234 Exchanger management of; 101 qualitative application of; 227 asynchrony/asynchronous analysis events, handling; 4 See also instrumentation; measure- I/O, and non-interruptable block- ment; static analysis tools; ing; 148 deadlock sequentiality vs.; 2 thread dump use; 216–217 tasks escape; 230 execution, Executor framework for exploitable parallelism; 123–133 use; 117 lock contention FutureTask handling; 95–98 thread dump use; 240 atomic variables; 319–336 performance; 221–245 and lock contention; 239–240 annotations; 353–354 classes; 324–329 See also documentation; locking vs.; 326–329 for concurrency documentation; 6 strategies for use; 34 @GuardedBy; 28, 354 volatile variables vs.; 39, 324–326 synchronization policy docu- atomic/atomicity; 22 mentation use; 75 See also invariant(s); synchroniza- @Immutable; 353 tion; visibility; @NotThreadSafe; 353 64-bit operations @ThreadSafe; 353 nonatomic nature of; 36 AOP (aspect-oriented programming) and compound actions; 22–23 in testing; 259, 273 and multivariable invariants; 57, application(s) 67–68 See also frameworks(s); service(s); and open call restructuring; 213 tools; and service shutdown; 153 -scoped objects and state transition constraints; 56 thread safety concerns; 10 caching issues; 24–25 frameworks, and ThreadLocal; 46 client-side locking support for; 80 GUI; 189–202 ﬁeld updaters; 335–336 thread safety concerns; 10–11 immutable object use for; 48 parallelizing in cache implementation design; 106 task decomposition; 113 intrinsic lock enforcement of; 25–26 shutdown loss and task cancellation; 136 risk of lock scope reduction; 234 AQS (AbstractQueuedSynchronizer) Map operations; 86 framework; 308, 311–317 put-if-absent; 71–72 exit protocol use; 306 statistics gathering hooks use; 179 FutureTask implementation thread-safety issues piggybacking use; 342 in servlets with state; 19–23 Index 361

AtomicBoolean; 325 methods AtomicInteger; 324 and interruption; 143 nonblocking algorithm use; 319 non-interruptable; 147–150 random number generator using; operations 327 250 252 li testing; – AtomicLong; 325 thread pool size impact; 170 AtomicReference; 325 queues; 87–94 nonblocking algorithm use; 319 See also Semaphore; safe publication use; 52 and thread pool management; AtomicReferenceFieldUpdater; 335 173 audit(ing) cancellation, problems; 138 See also instrumentation; cancellation, solutions; 140 28 audit(ing) tools; fn Executor functionality com- AWT (Abstract Window Toolkit) bined with; 129 See also GUI; producer-consumer pattern and; thread use; 9 87–92 safety concerns and; 10–11 spin-waiting; 232 state-dependent actions; 291–308 B and polling; 295–296 backoff and sleeping; 295–296 231 and nonblocking algorithms; fn condition queues; 296–308 283 292 barging; structure; li See also fairness; ordering; synchro- threads, costs of; 232 nization; waits and read-write locks; 287 timed vs. unbounded; 170 performance advantages of; 284 BlockingQueue; 84–85 barrier(s); 99, 99–101 and state-based preconditions; 57 See also latch(es); semaphores; syn- safe publication use; 52 chronizers; thread pool use of; 173 -based timer; 260–261 bound(ed) action; 99 See also constraints; encapsulation; memory; 230, 338 blocking collections point; 99 semaphore management of; 99 behavior buffers See also activities; task(s); blocking operations; 292 bias scalability testing; 261 See testing, pitfalls; size determination; 261 bibliography; 355–357 queues binary latch; 304 and producer-consumer pattern; AQS-based; 313–314 88 binary semaphore saturation policies; 174–175 mutex use; 99 thread pool use; 172 Bloch, Joshua thread pool use of; 173 (bibliographic reference); 69 resource; 221 block(ing); 92 boundaries bounded collections See also encapsulation; semaphore management of; 99 task; 113 testing; 248 analysis for parallelism; 123–133 context switching impact of; 230 broken multi-threaded programs interruptible methods and; 92–94 strategies for ﬁxing; 16 interruption handling methods; 138 362 Index

BrokenBarrierException long-running GUI tasks; 197–198 parallel iterative algorithm use; 99 non-standard buffer(s) encapsulation of; 148–150 See also cache/caching; reasons and strategies; 147–150 bounded points; 140 blocking state-dependent opera- policy; 136 tions with; 292 and thread interruption policy; scalability testing; 261 141 size determination; 261 interruption advantages as im- 249 140 BoundedBuffer example; li plementation strategy; condition queue use; 297 reasons for; 136 test case development for; 248 shutdown and; 135–166 250 BoundedBufferTest example; li task capacities Executor handling; 125 comparison testing; 261–263 in timed task handling; 131 testing; 248 timed locks use; 279 bug pattern(s); 271, 271 CancellationException See also debugging; design patterns; Callable handling; 98 testing; CAS (compare-and-swap) instructions; detector; 271 321–324 busy-waiting; 295 See also atomic/atomicity, variables; See also spin-waiting; Java class support in Java 5.0; 324 lock-free algorithm use; 329 C nonblocking algorithm use; 319, 329 cache/caching cascading effects See also performance; of thread safety requirements; 28 atomicity issues; 24–25 cellular automata flushing barrier use for computation of; 101 and memory barriers; 230 check-then-act operation implementation issues See also compound actions; atomic/atomicity; 106 as race condition cause; 21 safety; 104 atomic variable handling; 325 misses compound action as cost of context switching; 229 in collection operations; 79 result multivariable invariant issues; 67–68 building; 101–109 service shutdown issue; 153 126 Callable; li checkpoint FutureTask use; 95 state results handling capabilities; 125 shutdown issues; 158 callbacks checksums testing use; 257–259 safety testing use; 253 caller-runs saturation policy; 174 class(es) cancellation; 135–150 as instance confinement context; 59 See also interruption; lifecycle; shut- extension down; strategies and risks; 71 activity; 135 with helper classes; 72–73 as form of completion; 95 synchronized wrapper Future use; 145–147 client-side locking support; 73 interruptible lock acquisition; 279– thread-safe 281 and object composition; 55–78 interruption relationship to; 138 Index 363 cleanup of long-running GUI task; 198 See also lifecycle; service and interruption handling Future; 129 protecting data integrity; 142 task in end-of-lifecycle processing; 135 measuring service time variance; JVMshutdownhooksusefor;164 264–266 client(s) volatile variable use with; 39 See also server; CompletionService requests in page rendering example; 129 as natural task boundary; 113 composition; 73 client-side locking; 72–73, 73 See also delegation; encapsulation; See also lock(ing); as robust functionality extension and compound actions; 79–82 mechanism; 73 and condition queues; 306 of objects; 55–78 class extension relationship to; 73 compound actions; 22 150 stream class management; fn See also atomic/atomicity; concur- coarsening rent/concurrency, collec- See also lock(ing); tions; race conditions; 231 235 286 22 23 lock; , fn, atomicity handling of; – code review concurrency design rules role; 110 as quality assurance strategy; 271 concurrent collection support for; 84 collections examples of See also hashtables; lists; set(s); See check-then-act operation; bounded blocking iteration; navigation; put- semaphore management of; 99 if-absent operation; read- concurrent; 84–98 modify-write; remove-if- building block; 79–110 equal operation; replace-if- copying equal operation; as alternative to locking; 83 in cache implementation; 106 lock striping use; 237 in synchronized collection class use synchronized; 79–84 mechanisms for handling; 79–82 concurrent collections vs.; 84 synchronization requirements; 29 Collections.synchronizedList computation safe publication use; 52 compute-intensive code Collections.synchronizedXxx impact on locking behavior; 34 synchronized collection creation; 79 thread pool size impact; 170 communication deferred mechanisms for; 1 design issues; 125 compare-and-swap (CAS) instructions thread-local See CAS; and performance testing; 268 comparison Concurrent Programming in Java; 42, priority-ordered queue use; 89 57, 59, 87, 94, 95, 98, 99, 101, compilation 124, 201, 211, 279, 282, 304 dynamic concurrent/concurrency and performance testing; 267– See also parallelizing/parallelism; 268 safety; synchroniza- timing and ordering alterations tion/synchronized; thread safety risks; 7 and synchronized collections; 84 completion; 95 and task independence; 113 See also lifecycle; annotations; 353–354 notification brief history; 1–2 364 Index

building blocks; 79–110 blocking state-dependent opera- cache implementation issues; 103 tions use; 296–308 collections; 84–98 explicit; 306–308 ConcurrentHashMap locking strategy intrinsic; 297 advantages; 85 intrinsic, disadvantages of; 306 debugging using; 298 costs vs. performance optimiza- variables tion value; 224 explicit; 306–308 design rules; 110 waits errors and condition predicate; 299 301 See deadlock; livelock; race con- canonical form; li ditions; starvation; interruptible, as feature of Con- fine-grained dition; 307 and thread-safe data models; 201 uninterruptable, as feature of modifying Condition; 307 synchronized collection prob- waking up from, condition lems with; 82 queue handling; 300–301 object pool disadvantages; 241 conditional poor; 30 See also blocking/blocks; prevention notification; 303 See also single-threaded; as optimization; 303 single-threaded executor use; subclassing safety issues; 304 172 177 178 304 , – use; li read-write lock advantages; 286–289 read-modify-writer operations testing; 247–274 atomic variable support for; 325 ConcurrentHashMap; 84–86 configuration performance advantages of; 242 of ThreadPoolExecutor; 171–179 ConcurrentLinkedDeque; 92 thread creation ConcurrentLinkedQueue; 84–85 and thread factories; 175 algorithm; 319–336 thread pool reflection use; 335 post-construction manipulation; safe publication use; 52 177–179 84 87 ConcurrentMap; , li confinement safe publication use; 52 See also encapsulation; single- ConcurrentModificationException thread(ed); avoiding; 85 instance; 59, 58–60 fail-fast iterators use; 82–83 stack; 44, 44–45 ConcurrentSkipListMap; 85 thread; 42, 42–46 ConcurrentSkipListSet; 85 ad-hoc; 43 307 167 Condition; li and execution policy; explicit condition object use; 306 in Swing; 191–192 intrinsic condition queues vs. role, synchronization policy performance considerations; 308 specification; 56 condition serial; 90, 90–92 predicate; 299, 299–300 single-threaded GUI framework lock and condition variable rela- use; 190 tionship; 308 ThreadLocal; 45–46 queues; 297 Connection See also synchronizers; thread confinement use; 43 AQS support for; 312 ThreadLocal variable use with; 45 Index 365 consistent/consistency locking vs. atomic variables; 328 copy timeliness vs. resource as design tradeoff; 62 and task execution policy; 119 data view timeliness vs. deque advantages; 92 as design tradeoff; 66, 70 scalability under lock ordering as AQS advantage; 311 and deadlock avoidance; 206 scope weakly consistent iterators; 85 atomic variable limitation of; 324 constraints synchronization; 230 See also invariant(s); post-conditions; thread pre-conditions; collision detection help with; 321 state transition; 56 latches help with; 95 thread creation throughput impact; 228 importance of; 116 unrealistic degrees of construction/constructors as performance testing pitfall; See also lifecycle; 268–269 object context switching; 229 publication risks; 41–42 See also performance; thread handling issues; 41–42 as cost of thread use; 229–230 partial condition queues advantages; 297 unsafe publication influence; 50 cost(s); 8 private constructor capture idiom; message logging 69 243 244 fn reduction strategies; – starting thread from performance impact of; 221 as concurrency bug pattern; 272 reduction; 243–244 172 308 ThreadPoolExecutor; li signal method reduction in; post-construction customization; throughput impact; 228 177 control flow consumers See also event(s); lifecycle; MVC See also blocking, queues; producer- (model-view-controller) pat- consumer pattern; tern; blocking queues use; 88 coordination producer-consumer pattern in producer-consumer pattern; blocking queues and; 87–92 94 containers event handling 195 See also collections; model-view objects; fg 94 194 blocking queues as; simple; fg scoped latch characteristics; 94 thread safety concerns; 10 model-view-controller pattern contention/contended and inconsistent lock ordering; as performance inhibiting factor; 263 190 intrinsic locks vs. ReentrantLock vehicle tracking example; 61 performance considerations; convenience 282–286 See also responsiveness; lock as concurrency motivation; 2 costs of; 320 conventions measurement; 240–241 annotations reduction impact; 211 concurrency documentation; 6 reduction, strategies; 232–242 Java monitor pattern; 61 scalability impact; 232 signal method reduction in; 308 366 Index cooperation/cooperating cost(s) See also concurrent/concurrency; See also guidelines; performance; synchronization; safety; strategies; tradeoffs; end-of-lifecycle mechanisms thread; 229–232 interruption as; 93, 135 context switching; 8 model, view, and controller objects locality loss; 8 in GUI applications tradeoffs inconsistent lock ordering; 190 in performance optimization objects strategies; 223 212 95 deadlock, lock-ordering; li CountDownLatch; deadlock, possibilities; 211 AQS use; 315–316 livelock possibilities; 218 puzzle-solving framework use; 184 thread TestHarness example use; 96 concurrency mechanisms for; 79 counting semaphores; 98 coordination See also Semaphore; See also synchronization/synchro- permits, thread relationships; 248 310 nized; SemaphoreOnLock example; li control flow coupling producer-consumer pattern, See also dependencies; blocking queues use; 94 behavior in multithreaded environments blocking queue handling; 89 performance impact of; 221 implicit mutable state access between tasks and execution importance of; 110 policies; 167–170 copying CPU utilization collections See also performance; as alternative to locking; 83 and sequential execution; 124 data condition queues advantages; 297 thread safety consequences; 62 impact on performance testing; 261 CopyOnWriteArrayList; 84, 86–87 monitoring; 240–241 safe publication use; 52 optimization versioned data model use as multithreading goal; 222 in GUI applications; 201 spin-waiting impact on; 295 CopyOnWriteArraySet creation safe publication use; 52 See also copying; design; policy(s); synchronized Set replacement; 86 representation; core pool size parameter atomic compound actions; 80 171 172 thread creation impact; , fn class correctly synchronized program; 341 existing thread-safe class reuse correctness; 17 advantages over; 71 See also safety; collection copy testing; 248–260 as immutable object strategy; 86 goals; 247 of immutable objects; 48 thread safety defined in terms of; 17 of state-dependent methods; 57 corruption synchronizer; 94 See also atomic/atomicity; encapsu- thread; 171–172 lation; safety; state; explicitly, for tasks; 115 data thread factory use; 175–177 and interruption handling; 142 unbounded, disadvantages; 116 causes, stale data; 35 thread pools; 120 wrappers Index 367

during memoization; 103 Date customization effectively immutable use; 53 thread configuration dead-code elimination ThreadFactory use; 175 and performance testing; 269–270 thread pool configuration deadline-based waits post-construction; 177–179 as feature of Condition; 307 CyclicBarrier; 99 deadlock(s); 205, 205–217 102 parallel iterative algorithm use; li See also concurrent/concurrency, 255 260 testing use; li, li errors; liveness; safety; analysis D thread dump use; 216–217 daemon threads; 165 as liveness failure; 8 data avoidance 43 See also state; and thread confinement; fn contention avoidance nonblocking algorithm advan- and scalability; 237 tages; 319, 329 hiding strategies for; 215–217 thread-safety use; 16 cooperating objects; 211 nonatomic diagnosis 64-bit operations; 36 strategies for; 215–217 sharing; 33–54 dynamic lock order; 207–210 See also page renderer examples; in GUI framework; 190 access coordination; 277–290, 319 lock splitting as risk factor for; 235 advantages of threads; 2 locking during iteration risk of; 83 shared data models; 198–202 recovery synchronization costs; 8 database capabilities; 206 split data models; 201, 201–202 polled and timed lock acquisi- stale; 35–36 tion use; 279, 280 versioned data model; 201 timed locks use; 215 data race; 341 reentrancy avoidance of; 27 20 race condition vs.; fn resource; 213–215 data structure(s) thread starvation; 169, 168–169, 215 See also collections; object(s); deadly embrace queue(s); stack(s); trees; See deadlock; handling death, thread See atomic/atomicity; confine- abnormal, handling; 161–163 ment; encapsulation; itera- debugging tors/iteration; recursion; See also analysis; design; documenta- protection tion; recovery; testing; and interruption handling; 142 annotation use; 353 shared concurrency as serialization source; 226 costs vs. performance optimiza- testing insertion and removal han- tion value; 224 dling; 248 custom thread factory as aid for; 175 38 database(s) JVM optimization pitfalls; fn 206 216 deadlock recovery capabilities; thread dump use; fn JDBC Connection thread dumps thread confinement use; 43 intrinsic lock advantage over thread pool size impact; 171 ReentrantLock; 285–286 unbounded thread creation risks; 116 368 Index decomposition task freedom from, importance See also composition; delegation; of; 113 encapsulation; task producer-consumer pattern; 89 and execution policy; 167 tasks-related; 113–134 thread starvation deadlock; 168 Decorator pattern task freedom from collection class use for wrapper fac- importance; 113 tories; 60 Deque; 92 decoupling deques of activities See also collections; data structure(s); as producer-consumer pattern queue(s); advantage; 87 work stealing and; 92 task decomposition as represen- design tation of; 113 See also documentation; guidelines; of interrupt notification from han- policies; representation; dling in Thread interruption strategies; handling methods; 140 class task submission from execution state ownership as element of; and Executor framework; 117 57–58 delayed tasks concurrency design rules; 110 See also time/timing; concurrency testing; 250–252 handling of; 123 condition queue encapsulation; 306 DelayQueue condition queues time management; 123 and condition predicate; 299 delegation control flow See also composition; design; safety; latch characteristics; 94 advantages execution policy class extension vs.; 314 influencing factors; 167 for class maintenance safety; 234 GUI single-threaded use thread safety; 234 rationale for; 189–190 failure causes; 67–68 importance management; 62 in thread-safe programs; 16 dependencies of thread-safe classes See also atomic/atomicity; invari- guidelines; 55–58 ant(s); postconditions; pre- parallelism conditions; state; application analysis for; 123–133 code parallelization criteria; 181 as removal, as producer- performance consumer pattern advantage; analysis, monitoring, and im- 87 provement; 221–245 in multiple-variable invariants performance tradeoffs thread safety issues; 24 evaluation of; 223–225 state principles blocking operations; 291–308 simplicity of final fields; 48 classes; 291 producer-consumer pattern classes, building; 291–318 decoupling advantages; 117 managing; 291–298 Executor framework use; 117 operations; 57 program operations, condition queue han- and task decomposition; 113–134 dling; 296–308 result-bearing tasks representation issues; 125 Index 369

strategies driver program for InterruptedException; 93 for TimedPutTakeTest example; 262 thread confinement; 43 dynamic thread pool size See also responsiveness; relevant factors for; 170 compilation timed tasks; 131–133 as performance testing pitfall; tradeoffs 267–268 collection copying vs. locking lock order deadlocks; 207–210 during iteration; 83 concurrent vs. synchronized E collections; 85 EDT (event dispatch thread) copy-on-write collections; 87 GUI frameworks use; 5 synchronized block; 34 single-threaded GUI use; 189 timeliness vs. consistency; 62, thread confinement use; 42 66, 70 Effective Java Programming Language design patterns Guide; 46–48, 73, 166, 257, antipattern example 292, 305, 314, 347 double-checked locking; 348–349 efficiency examples See also performance; See Decorator pattern; MVC responsiveness vs. (model-view-controller) pat- polling frequency; 143 tern; producer-consumer result cache, building; 101–109 pattern; Singleton pattern; elision 231 destruction lock; fn See teardown; JVM optimization; 286 dining philosophers problem; 205 encapsulation See also deadlock; See also access; atomic/atomicity; discard saturation policy; 174 confinement; safety; state; discard-oldest saturation policy; 174 visibility; documentation breaking See also debugging; design; good costs of; 16–17 practices; guidelines; pol- code icy(s); as producer-consumer pattern annotation use; 6, 353 advantage; 87 concurrency design rules role; 110 composition use; 74 critical importance for conditional concurrency design rules role; 110 notification use; 304 implementation importance class extension violation of; 71 for special execution policy re- instance confinement relationship quirements; 168 with; 58–60 stack confinement usage; 45 invariant management with; 44 of synchronization policies; 74–77 locking behavior safe publication requirements; 54 reentrancy facilitation of; 27 double-checked locking (DCL); 348– non-standard cancellation; 148–150 349 of condition queues; 306 as concurrency bug pattern; 272 of lifecycle methods; 155 downgrading of synchronization read-write lock implementation hidden iterator management strategy; 287 through; 83 publication dangers for; 39 state 370 Index

breaking, costs of; 16–17 sequential processing invariant protection use; 83 in GUI applications; 191 ownership relationship with; 58 timing synchronizer role; 94 and liveness failures; 8 thread-safe class use; 23 example classes 327 synchronization policy AtomicPseudoRandom; li 71 233 and client-side locking; AttributeStore; li 150 199 thread ownership; BackgroundTask; li 55 261 thread-safety role; BarrierTimer; li 16 293 thread-safety use; BaseBoundedBuffer; li 234 end-of-lifecycle BetterAttributeStore; li 72 See also thread(s); BetterVector; li 135 166 258 management techniques; – Big; li 248 249 297 298 enforcement BoundedBuffer; , li, , li 28 250 locking policies, lack of; BoundedBufferTest; li entry protocols BoundedExecutor; 175 306 100 state-dependent operations; BoundedHashSet; li 139 Error BrokenPrimeProducer; li 97 31 Callable handling; CachedFactorizer; li 151 error(s) CancellableTask; li 136 323 as cancellation reason; CasCounter; li 326 concurrency CasNumberRange; li 102 See deadlock; livelock; race con- CellularAutomata; li 103 ditions; Computable; li 39 186 escape; ConcurrentPuzzleSolver; li 230 331 analysis; ConcurrentStack; li 308 309 prevention ConditionBoundedBuffer; , li 59 256 in instance confinement; Consumer; li 39 42 56 publication and; – Counter; li 23 risk factors CountingFactorizer; li 60 157 in instance confinement; CrawlerThread; li 65 Ethernet protocol DelegatingVehicleTracker; li, exponential backoff use; 219 201 210 evaluation DemonstrateDeadlock; li 212 214 See also design; measurement; test- Dispatcher; li, li 349 ing; DoubleCheckedLocking; li 223 225 103 of performance tradeoffs; – ExpensiveFunction; li 191 109 event(s); Factorizer; li 136 91 as cancellation reason; FileCrawler; li 128 dispatch thread FutureRenderer; li 5 292 294 GUI frameworks use; GrumpyBoundedBuffer; , li 192 194 handling GuiExecutor; , li 194 84 control flow, simple; fg HiddenIterator; li 195 74 model-view objects; fg ImprovedList; li 4 91 threads benefits for; Indexer; li 99 157 latch handling based on; IndexerThread; li 156 main event loop IndexingService; li 5 21 vs. event dispatch thread; LazyInitRace; li 207 notification LeftRightDeadlock; li 122 copy-on-write collection advan- LifecycleWebServer; li 87 334 tages; LinkedQueue; li Index 371

73 74 26 ListHelper; , li SynchronizedFactorizer; li 153 154 36 LogService; , li SynchronizedInteger; li 152 118 LogWriter; li TaskExecutionWebServer; li 103 108 94 Memoizer; li, li TaskRunnable; li 104 212 214 Memoizer2; li Taxi; li, li 106 96 Memoizer3; li TestHarness; li 63 258 MonitorVehicleTracker; li TestingThreadFactory; li 36 41 MutableInteger; li ThisEscape; li 64 169 MutablePoint; li ThreadDeadlock; li 177 178 305 MyAppThread; , li ThreadGate; li 177 118 MyThreadFactory; li ThreadPerTaskExecutor; li 184 115 Node; li ThreadPerTaskWebServer; li 34 47 NoVisibility; li ThreeStooges; li 67 261 NumberRange; li TimedPutTakeTest; 313 180 OneShotLatch; li TimingThreadPool; li 49 51 159 OneValueCache; li, li TrackingExecutorService; li 124 161 163 OutOfTime; li, UEHLogger; li 59 24 PersonSet; li UnsafeCachingFactorizer; li 64 19 Point; li UnsafeCountingFactorizer; li 340 345 PossibleReordering; li UnsafeLazyInitialization; li 97 40 Preloader; li UnsafeStates; li 137 184 187 PrimeGenerator; li ValueLatch; , li 141 66 PrimeProducer; li VisualComponent; li 61 50 PrivateLock; li VolatileCachedFactorizer; li 256 160 Producer; li WebCrawler; li 255 260 27 PutTakeTest; li, Widget; li 183 119 Puzzle; li WithinThreadExecutor; li 188 227 PuzzleSolver; li WorkerThread; li 129 QueueingFuture; li exceptions 149 ReaderThread; li See also error(s); interruption; lifecy- 288 ReadWriteMap; li cle; 327 292 295 ReentrantLockPseudoRandom; li and precondition failure; – 130 95 Renderer; li as form of completion; 42 97 SafeListener; li Callable handling; 69 SafePoint; li causes 350 35 SafeStates; li stale data; 145 ScheduledExecutorService; li handling 310 125 SemaphoreOnLock; li Runnable limitations; 7 Sequence; li logging 185 163 SequentialPuzzleSolver; li UEHLogger example; li 236 82 ServerStatus; li thread-safe class handling; 322 123 SimulatedCAS; li Timer disadvantages; 125 162 SingleThreadRenderer; li uncaught exception handler; – 114 163 SingleThreadWebServer; li 295 296 SleepyBoundedBuffer; , li unchecked 151 161 SocketUsingTask; li catching, disadvantages; 186 SolverTask; li Exchanger 18 StatelessFactorizer; li See also producer-consumer pattern; 238 101 StripedMap; li as two-party barrier; 191 192 193 53 SwingUtilities; , , li safe publication use; 343 Sync; li 372 Index execute external locking; 73 submit vs., uncaught exception handling; 163 F execution factory(s) policies See also creation; design, influencing factors; 167 methods Executors factory methods; 171 constructor use with; 42 implicit couplings between tasks newTaskFor; 148 and; 167–170 synchronized collections; 79, 171 parallelism analysis for; 123–133 thread pool creation with; 120 task; 113–134 thread; 175, 175–177 policies; 118–119 fail-fast iterators; 82 sequential; 114 See also iteration/iterators; ExecutionException failure Callable handling; 98 See also exceptions; liveness, failure; 117 117 133 Executor framework; li, – recovery; safety; and GUI event processing; 191, 192 causes and long-running GUI tasks; 195 stale data; 35 as producer-consumer pattern; 88 graceful degradation execution policy design; 167 task design importance; 113 FutureTask use; 97 management techniques; 135–166 194 GuiExecutor example; li modes single-threaded testing for; 247–274 169 deadlock example; li precondition ExecutorCompletionService bounded buffer handling of; 292 in page rendering example; 129 propagation to callers; 292–295 Executors thread factory methods uncaught exception handlers; thread pool creation with; 120 162–163 ExecutorService timeout and service shutdown; 153–155 deadlock detection use; 215 cancellation strategy using; 146 fairness checkMail example; 158 See also responsiveness; synchroniza- 121 121 122 lifecycle methods; li, – tion; exhaustion as concurrency motivation; 1 See failure; leakage; resource exhaus- fair lock; 283 tion; nonfair lock; 283 exit protocols nonfair semaphores vs. fair state-dependent operations; 306 performance measurement; 265 explicit locks; 277–290 queuing 297 interruption during acquisition; 148 intrinsic condition queues; fn exponential backoff ReentrantLock options; 283–285 and avoiding livelock; 219 ReentrantReadWriteLock; 287 extending scheduling existing thread-safe classes thread priority manipulation and client-side locking; 73 risks; 218 strategies and risks; 71 ’fast path’ synchronization ThreadPoolExecutor; 179 CAS-based operations vs.; 324 costs of; 230 Index 373 feedback as blocking queue advantage; 88 See also GUI; task design guidelines for; 113 user task design role; 113 196 in long-running GUI tasks; li flow control fields communication networks, thread 335 336 173 atomic updaters; – pool comparison; fn hot fields fragility avoiding; 237 See also debugging; guidelines; ro- updating, atomic variable ad- bustness; safety; scalability; vantages; 239–240 testing; initialization safety issues and causes final field guarantees; 48 as class extension; 71 FIFO queues as client-side locking; 73 BlockingQueue implementations; 89 interruption use for non- files standard purposes; 138 See also data; database(s); issue; 43 as communication mechanism; 1 piggybacking; 342 final state-dependent classes; 304 and immutable objects; 48 volatile variables; 38 concurrency design rules role; 110 solutions immutability not guaranteed by; 47 composition; 73 safe publication use; 52 encapsulation; 17 158 volatile vs.; fn stack confinement vs. ad-hoc finalizers thread confinement; 44 JVM orderly shutdown use; 164 frameworks warnings; 165–166 See also AQS framework; data struc- finally block ture(s); Executor framework; See also interruptions; lock(ing); RMI framework; Servlets importance with explicit locks; 278 framework; FindBugs code auditing tool application See also tools; and ThreadLocal; 46 as static analysis tool example; 271 serialization hidden in; 227 28 9 locking failures detected by; fn thread use; 278 9 unreleased lock detector; fn thread use impact on applications; fire-and-forget event handling strategy threads benefits for; 4 drawbacks of; 195 functionality flag(s) extending for existing thread-safe See mutex; classes cancellation request strategies and risks; 71 as cancellation mechanism; 136 tests interrupted status; 138 vs. performance tests; 260 126 flexibility Future; li See also responsiveness; scalability; cancellation and instance confinement; 60 of long-running GUI task; 197 decoupling task submission from strategy using; 145–147 execution, advantages for; characteristics of; 95 119 encapsulation of non-standard can- immutable object design for; 47 cellation use; 148 in CAS-based algorithms; 322 results handling capabilities; 125 interruption policy; 142 safe publication use; 53 resource management task lifecycle representation by; 125 374 Index

task representation @GuardedBy; 353–354 implementation strategies; 126 and documenting synchronization 95 7 75 FutureTask; policy; fn, AQS use; 316 GUI (Graphical User Interface) as latch; 95–98 See also event(s); single-thread(ed); completion notification Swing; of long-running GUI task; 198 applications; 189–202 efficient and scalable cache imple- thread safety concerns; 10–11 mentation with; 105 frameworks 97 108 151 199 example use; li, li, li, li as single-threaded task execu- 126 114 task representation use; tion example; fn long-running task handling; 195–198 G MVC pattern use garbage collection in vehicle tracking example; 61 as performance testing pitfall; 266 response-time sensitivity gate and execution policy; 168 See also barrier(s); conditional; single-threaded use latch(es); rationale for; 189–190 as latch role; 94 threads benefits for; 5 ThreadGate example; 304 guidelines global variables See also design; documentation; pol- ThreadLocal variables use with; 45 icy(s); strategies; good practices allocation vs. synchronization; 242 See design; documentation; encap- atomicity sulation; guidelines; perfor- definitions; 22 mance; strategies; concurrency design rules; 110 graceful Condition methods degradation potential confusions; 307 and execution policy; 121 condition predicate and saturation policy; 175 documentation; 299 limiting task count; 119 lock and condition queue rela- task design importance; 113 tionship; 300 shutdown condition wait usage; 301 vs. abrupt shutdown; 153 confinement; 60 granularity deadlock avoidance See also atomic/atomicity; scope; alien method risks; 211 atomic variable advantages; 239–240 lock ordering; 206 lock open call advantages; 213 Amdahl’s law insights; 229 thread starvation; 169 reduction of; 235–237 documentation nonblocking algorithm advantages; value for safety; 16 319 encapsulation; 59, 83 serialization value for safety; 16 throughput impact; 228 exception handling; 163 timer execution policy measurement impact; 264 design; 119 guarded special case implications; 168 objects; 28, 54 final field use; 48 state finalizer precautions; 166 locks use for; 27–29 happens-before use; 346 immutability Index 375

effectively immutable objects; 53 sharing objects; 46 safety strategies; 16 requirements for; 47 sharing objects; 54 value for safety; 16 simplicity initialization safety; 349, 350 performance vs.; 32 interleaving diagrams; 6 starvation avoidance interruption handling thread priority precautions; 218 cancellation relationship; 138 state importance of interruption pol- consistency preservation; 25 icy knowledge; 142, 145 managing; 23 interrupt swallowing precau- variables, independent; 68 tions; 143 stateless objects intrinsic locks vs. ReentrantLock; thread-safety of; 19 285 synchronization invariants immutable objects as replace- locking requirements for; 29 ment for; 52 thread safety importance; 57 shared state requirements for; 28 value for safety; 16 task cancellation lock criteria for; 147 contention, reduction; 233 testing contention, scalability impact; effective performance tests; 270 231 timing-sensitive data races; 254 holding; 32 this reference ordering, deadlock avoidance; publication risks; 41 206 threads measurement daemon thread precautions; 165 importance; 224 handling encapsulation; 150 notification; 303 lifecycle methods; 150 objects pools; 174 stateless, thread-safety of; 19 safety; 18, 55 operation ordering volatile variables; 38 synchronization role; 35 optimization H lock contention impact; 231 hand-over-hand locking; 282 premature, avoidance of; 223 happens-before parallelism analysis; 123–133 JMM definition; 340–342 performance piggybacking; 342–344 optimization questions; 224 publication consequences; 244–249 simplicity vs.; 32 hardware postconditions; 57 See also CPU utilization; private field use; 48 concurrency support; 321–324 publication; 52, 54 JVM interaction safety reordering; 34 definition; 18 platform memory models; 338 testing; 252 timing and ordering alterations by scalability; 84 thread safety risks; 7 attributes; 222 hashcodes/hashtables locking impact on; 232 See also collections; sequential loops ConcurrentHashMap; 84–86 parallelization criteria; 181 performance advantages of; 242 serialization sources; 227 Hashtable; 79 376 Index

safe publication use; 52 HttpSession 208 58 inducing lock ordering with; thread-safety requirements; fn lock striping use; 237 heap inspection tools I See also tools; I/O measuring memory usage; 257 See also resource(s); 247 Heisenbugs; fn asynchronous helper classes non-interruptable blocking; 148 and extending class functionality; message logging 72–73 reduction strategies; 243–244 heterogeneous tasks operations parallelization limitations; 127–129 thread pool size impact; 170 hijacked signal sequential execution limitations; 124 See missed signals; server applications Hoare, C. A. R. task execution implications; 114 Java monitor pattern inspired by synchronous 60 148 (bibliographic reference); fn non-interruptable blocking; hoisting threads use to simulate; 4 variables utilization measurement tools; 240 38 as JVM optimization pitfall; fn idempotence homogeneous tasks and race condition mitigation; 161 parallelism advantages; 129 idioms hooks See also algorithm(s); conventions; See also extending; design patterns; documen- completion tation; policy(s); protocols; in FutureTask; 198 strategies; shutdown; 164 double-checked locking (DCL) JVM orderly shutdown; 164–165 as bad practice; 348–349 single shutdown lazy initialization holder class; 347– orderly shutdown strategy; 164 348 69 ThreadPoolExecutor extension; 179 private constructor capture; fn hot fields safe initialization; 346–348 avoidance safe publication; 52–53 scalability advantages; 237 IllegalStateException updating Callable handling; 98 atomic variable advantages; 239– @Immutable; 7, 353 240 immutable/immutability; 46–49 HotSpot JVM See also atomic/atomicity; safety; dynamic compilation use; 267 concurrency design rules role; 110 ’how fast’; 222 effectively immutable objects; 53 See also GUI; latency; responsive- initialization safety guarantees; 51 ness; initialization safety limitation; 350 vs. ’how much’; 222 objects; 46 ’how much’; 222 publication with volatile; 48–49 See also capacity; scalability; requirements for; 47 throughput; role in synchronization policy; 56 importance for server applications; thread-safety use; 16 223 implicit coupling vs. ’how fast’; 222 See also dependencies; between tasks and execution policies; 167–170 Index 377 improper publication; 51 interleaving See also safety; diagram interpretations; 6 increment operation (++) generating as non-atomic operation; 19 testing use; 259 independent/independence; 25 logging output 150 See also dependencies; encapsula- and client-side locking; fn 81 tion; invariant(s); state; operation; fg multiple-variable invariant lack of ordering impact; 339 thread safety issues; 24 thread parallelization use; 183 dangers of; 5–8 state variables; 66, 66–67 timing dependencies impact on lock splitting use with; 235 race conditions; 20 task thread execution concurrency advantages; 113 in thread safety definition; 18 inducing lock ordering interrupted (Thread) for deadlock avoidance; 208–210 usage precautions; 140 initialization InterruptedException See also construction/constructors; flexible interruption policy advan- lazy; 21 tages; 142 as race condition cause; 21–22 interruption API; 138 348 143 safe idiom for; li propagation of; li unsafe publication risks; 345 strategies for handling; 93 safety task cancellation and immutable objects; 51 criteria for; 147 final field guarantees; 48 interruption(s); 93, 135, 138–150 idioms for; 346–348 See also completion; errors; lifecycle; JMM support; 349–350 notification; termination; inner classes triggering; publication risks; 41 and condition waits; 307 instance confinement; 59, 58–60 blocking and; 92–94 See also confinement; encapsulation; blocking test use; 251 instrumentation interruption response strategy See also analysis; logging; monitor- exception propagation; 142 ing; resource(s), manage- status restoration; 142 ment; statistics; testing; lock acquisition use; 279–281 of thread creation non-cancellation uses for; 143 thread pool testing use; 258 non-interruptable blocking potential handling; 147–150 as execution policy advantage; reasons for; 148 121 policies; 141, 141–142 service shutdown use; 158 preemptive 135 support deprecation reasons; fn Executor framework use; 117 request thread pool size requirements deter- strategies for handling; 140 mination use of; 170 responding to; 142–150 ThreadPoolExecutor hooks for; 179 swallowing interfaces as discouraged practice; 93 user bad consequences of; 140 threads benefits for; 5 when permitted; 143 thread; 138 volatile variable use with; 39 378 Index intransitivity iterators/iteration encapsulation characterized by; 150 See also concurrent/concurrency; intrinsic condition queues; 297 control flow; recursion; disadvantages of; 306 as compound action intrinsic locks; 25, 25–26 in collection operations; 79 See also encapsulation; lock(ing); atomicity requirements during; 80 safety; synchronization; fail-fast; 82 thread(s); ConcurrentModificationExcep- acquisition, non-interruptable block- tion exception with; 82–83 ing reason; 148 hidden; 83–84 advantages of; 285 locking explicit locks vs.; 277–278 concurrent collection elimination intrinsic condition queue relation- of need for; 85 ship to; 297 disadvantages of; 83 limitations of; 28 parallel iterative algorithms 237 99 recursion use; fn barrier management of; ReentrantLock vs.; 282–286 parallelization of; 181 visibility management with; 36 unreliable invariant(s) and client-side locking; 81 See also atomic/atomicity; post- weakly consistent; 85 conditions; pre-conditions; state; J and state variable publication; 68 Java Language Specification, The; 53, 218 259 358 BoundedBuffer example; 250 fn, , callback testing; 257 Java Memory Model (JMM); 337–352 concurrency design rules role; 110 See also design; safety; synchroniza- encapsulation tion; visibility; state, protection of; 83 initialization safety guarantees for value for; 44 immutable objects; 51 immutable object use; 49 Java monitor pattern; 60, 60–61 independent state variables require- composition use; 74 ments; 66–67 vehicle tracking example; 61–71 multivariable Java Programming Language, The; 346 and atomic variables; 325–326 java.nio package atomicity requirements; 57, 67– synchronous I/O 68 non-interruptable blocking; 148 locking requirements for; 29 JDBC (Java Database Connectivity) preservation of, as thread safety Connection requirement; 24 thread confinement use; 43 thread safety issues; 24 JMM (Java Memory Model) preservation of See Java Memory Model (JMM); immutable object use; 46 join (Thread) mechanisms and synchroniza- timed tion policy role; 55–56 problems with; 145 publication dangers for; 39 JSPs (JavaServer Pages) specification of thread safety requirements; 10 thread-safety use; 16 JVM (Java Virtual Machine) thread safety role; 17 See also optimization; iostat application deadlock handling limitations; 206 See also measurement; tools; escape analysis; 230–231 320 I/O measurement; 240 lock contention handling; fn Index 379

nonblocking algorithm use; 319 Life cellular automata game 38 101 optimization pitfalls; fn barrier use for computation of; optimizations; 286 lifecycle service shutdown issues; 152–153 See also cancellation; completion; shutdown; 164–166 construction/constructors; and daemon threads; 165 Executor; interruption; shut- orderly shutdown; 164 down; termination; thread(s); synchronization optimization by; time/timing; 230 encapsulation; 155 thread timeout interaction Executor 172 121 122 and core pool size; fn implementations; – thread use; 9 management strategies; 135–166 162 uncaught exception handling; fn support Executor framework use; 117 K task keep-alive time and Future; 125 thread termination impact; 172 Executor phases; 125 thread L performance impact; 116 latch(es); 94, 94–95 thread-based service manage- See also barriers; blocking; ment; 150 semaphores; synchroniz- lightweight processes ers; See threads; barriers vs.; 99 linked lists 304 binary; LinkedBlockingDeque; 92 313 314 AQS-based; – LinkedBlockingQueue; 89 FutureTask; 95–98 performance advantages; 263 184 puzzle-solving framework use; thread pool use of; 173–174 304 ThreadGate example; LinkedList; 85 layering Michael-Scott nonblocking queue; three-tier application 332–335 as performance vs. scalability nonblocking; 330 illustration; 223 List 21 lazy initialization; CopyOnWriteArrayList as concur- 21 22 as race condition cause; – rent collection for; 84, 86 348 safe idiom for; li listeners 345 unsafe publication risks; See also event(s); leakage action; 195–197 See also performance; Swing resource single-thread rule exceptions; testing for; 257 190 thread; 161 Swing event handling; 194 Timer problems with; 123 lists UncaughtExceptionHandler See also collections; prevention of; 162–163 CopyOnWriteArrayList lexical scope safe publication use; 52 59 as instance conﬁnement context; versioned data model use; 201 library LinkedList; 85 thread-safe collections List 52 safe publication guarantees; CopyOnWriteArrayList as concurrent replacement; 84, 86 380 Index

Little’s law reentrant lock count; 26 232 279 lock contention corollary; fn timed; livelock; 219, 219 and instance confinement; 59 See also concurrent/concurrency, atomic variables vs.; 326–329 errors; liveness; avoidance as liveness failure; 8 immutable objects use; 49 liveness building See also performance; responsiveness AQS use; 311 failure; client-side; 72–73, 73 causes and compound actions; 79–82 See deadlock; livelock; missed condition queue encapsulation signals; starvation; impact on; 306 150 failure stream class management; fn avoidance; 205–220 vs. class extension; 73 improper lock acquisition risk of; 61 coarsening; 231 nonblocking algorithm advantages; as JVM optimization; 286 319–336 impact on splitting synchronized 235 performance and blocks; fn in servlets with state; 29–32 concurrency design rules role; 110 safety vs. ConcurrentHashMap strategy; 85 See safety; ConcurrentModificationException term definition; 8 avoidance with; 82 testing condition variable and condition criteria; 248 predicate relationship; 308 thread safety hazards for; 8 contention local variables measurement; 240–241 See also encapsulation; state; vari- reduction, guidelines; 233 ables; reduction, impact; 211 for thread confinement; 43 reduction, strategies; 232–242 stack confinement use; 44 scalability impact of; 232 locality, loss of coupling; 282 as cost of thread use; 8 cyclic locking dependencies 277 277 282 205 Lock; li, – as deadlock cause; and Condition; 307 disadvantages of; 319–321 interruptible acquisition; 148 double-checked timed acquisition; 215 as concurrency bug pattern; 272 85 231 lock(ing); elision; fn See also confinement; encapsulation; as JVM optimization; 286 @GuardedBy; safety; synchro- encapsulation of nization; reentrancy facilitation; 27 acquisition exclusive AQS-based synchronizer opera- alternative to; 239–240 tions; 311–313 alternatives to; 321 improper, liveness risk; 61 inability to use, as Concurrent- interruptible; 279–281 HashMap disadvantage; 86 intrinsic, non-interruptable timed lock use; 279 blocking reason; 148 explicit; 277–290 nested, as deadlock risk; 208 interruption during lock acquisi- polled; 279 tion use; 148 protocols, instance confinement granularity use; 60 Amdahl’s law insights; 229 Index 381

reduction of; 235–237 scope hand-over-hand; 282 See also lock(ing), granularity; in blocking actions; 292 narrowing, as lock contention intrinsic; 25, 25–26 reduction strategy; 233–235 acquisition, non-interruptable splitting; 235 blocking reason; 148 Amdahl’s law insights; 229 advantages of; 285 as lock granularity reduction explicit locks vs.; 277–278 strategy; 235 236 intrinsic condition queue rela- ServerStatus examples; li tionship to; 297 state guarding with; 27–29 limitations of; 28 striping; 237 private locks vs.; 61 Amdahl’s law insights; 229 237 85 recursion use; fn ConcurrentHashMap use; ReentrantLock vs., performance stripping; 237 considerations; 282–286 thread dump information about; 216 iteration thread-safety issues concurrent collection elimination in servlets with state; 23–29 of need for; 85 timed; 215–216 disadvantages of; 83 unreleased monitor as concurrency bug pattern; 272 See intrinsic locks; visibility and; 36–37 non-block-structured; 281–282 volatile variables vs.; 39 nonblocking algorithms vs.; 319 wait open calls and condition predicate; 299 for deadlock avoidance; 211–213 lock-free algorithms; 329 ordering logging deadlock risks; 206–213 See also instrumentation; dynamic, deadlocks resulting exceptions 207 210 163 from; – UEHLogger example; li inconsistent, as multithreaded service GUI framework problem; 190 as example of stopping a thread- private based service; 150–155 intrinsic locks vs.; 61 thread customization example; 177 protocols ThreadPoolExecutor hooks for; 179 shared state requirements for; 28 logical state; 58 read-write; 286–289 loops/looping implementation strategies; 287 and interruption; 143 reentrant semantics; 26–27 M semantics, ReentrantLock capa- main event loop bilities; 278 vs. event dispatch thread; 5 ReentrantLock fairness options; Map 283–285 ConcurrentHashMap as concurrent release replacement; 84 in hand-over-hand locking; 282 performance advantages; 242 intrinsic locking disadvantages; atomic operations; 86 278 maximum pool size parameter; 172 preference, in read-write lock measurement implementation; 287 importance for effective optimiza- role tion; 224 synchronization policy; 56 performance; 222 382 Index

profiling tools; 225 simplicity lock contention; 240 threads benefit for; 3 responsiveness; 264–266 split data models; 201, 201–202 strategies and tools Swing event handling; 194 profiling tools; 225 three-tier application ThreadPoolExecutor hooks for; 179 performance vs. scalability; 223 memoization; 103 versioned data model; 201 See also cache/caching; modification memory concurrent See also resource(s); synchronized collection prob- barriers; 230, 338 lems with; 82 depletion frequent need for avoiding request overload; 173 copy-on-write collection not testing for; 257 suited for; 87 thread-per-task policy issue; 116 monitor(s) models See also Java monitor pattern; hardware architecture; 338 locks JMM; 337–352 See intrinsic locks; reordering monitoring operations; 339 See also instrumentation; perfor- shared memory multiprocessors; mance; scalability; testing; 338–339 tools; synchronization CPU utilization; 240–241 performance impact of; 230–231 performance; 221–245 thread pool size impact; 171 ThreadPoolExecutor hooks for; 179 visibility; 33–39 tools ReentrantLock effect; 277 for quality assurance; 273 synchronized effect; 33 monomorphic call transformation 268 Michael-Scott nonblocking queue; JVM use; fn 332–335 mpstat application; 240 missed signals; 301, 301 See also measurement; tools; See also liveness; multiple-reader, single-writer locking as single notification risk; 302 and lock contention reduction; 239 model(s)/modeling read-write locks; 286–289 See also Java Memory Model multiprocessor systems (JMM); MVC (model-view- See also concurrent/concurrency; controller) design pattern; shared memory representation; views; memory models; 338–339 event handling threads use of; 3 195 model-view objects; fg multithreaded memory See also safety; single-thread(ed); hardware architecture; 338 thread(s); JMM; 337–352 GUI frameworks model-view-controller pattern issues with; 189–190 deadlock risk; 190 multivariable invariants vehicle tracking example; 61 and atomic variables; 325–326 programming atomicity requirements; 57, 67–68 sequential; 2 dependencies, thread safety issues; shared data 24 See also page renderer examples; locking requirements for; 29 in GUI applications; 198–202 Index 383

preservation of, as thread safety copy-on-write collection advan- requirement; 24 tages; 87 mutable; 15 notify objects as optimization; 303 54 298 safe publication of; efficiency of; fn state missed signal risk; 302 managing access to, as thread notifyAll vs.; 302 safety goal; 15 subclassing safety issues mutexes (mutual exclusion locks); 25 documentation importance; 304 binary semaphore use as; 99 usage guidelines; 303 intrinsic locks as; 25 notifyAll ReentrantLock capabilities; 277 notify vs.; 302 MVC (model-view-controller) pattern @NotThreadSafe; 6, 353 deadlock risks; 190 NPTL threads package 61 4 vehicle tracking example use of; Linux use; fn nulling out memory references N testing use; 257 narrowing lock scope O as lock contention reduction object(s) strategy; 233–235 See also resource(s); native code composing; 55–78 finalizer use and limitations; 165 condition navigation explicit; 306–308 as compound action effectively immutable; 53 in collection operations; 79 guarded; 54 126 46 newTaskFor; li immutable; encapsulating non-standard cancel- initialization safety; 51 lation; 148 publication using volatile; 48–49 nonatomic 64-bit operations; 36 mutable nonblocking algorithms; 319, 329, 329– safe publication of; 54 336 pools 231 241 backoff importance for; fn appropriate uses; fn synchronization; 319–336 bounded, semaphore manage- 174 99 SynchronousQueue; fn ment of; thread-safe counter use; 322–324 disadvantages of; 241 nonfair semaphores serial thread confinement use; 90 advantages of; 265 references notification; 302–304 and stack confinement; 44 See also blocking; condition, queues; sharing; 33–54 event(s); listeners; notify; state; 55 notifyAll; sleeping; wait(s); components of; 55 waking up; Swing completion thread-confinement; 191–192 of long-running GUI task; 198 objects conditional; 303 guarded; 28 as optimization; 303 open calls; 211, 211–213 304 use; li See also encapsulation; errors operating systems as concurrency bug pattern; 272 concurrency use event notification systems historical role; 1 384 Index operations total 64-bit, nonatomic nature of; 36 synchronization actions; 341 state-dependent; 57 orderly shutdown; 164 optimistic concurrency management OutOfMemoryError See atomic variables; CAS; nonblock- unbounded thread creation risk; 116 ing algorithms; overhead optimization See also CPU utilization; measure- compiler ment; performance; as performance testing pitfall; impact of 268–270 See performance; throughput; JVM reduction 38 pitfalls; fn See nonblocking algorithms; op- strategies; 286 timization; thread(s), pools; lock contention sources impact; 231 See blocking/blocks; contention; reduction strategies; 232–242 context switching; multi- performance threaded environments; Amdahl’s law; 225–229 safety; suspension; synchro- premature, avoidance of; 223 nization; thread(s), lifecycle; questions about; 224 ownership scalability requirements vs.; 222 shared; 58 techniques split; 58 See also atomic variabless; non- state blocking synchronization; class design issues; 57–58 condition queues use; 297 thread; 150 conditional notification; 303 order(ing) P See also reordering; synchronization; page renderer examples acquisition, in ReentrantRead- See also model(s)/modeling, shared 317 WriteLock; fn data; checksums heterogenous task partitioning; 127– safety testing use; 253 129 FIFO parallelism analysis; 124–133 impact of caller state depen- sequential execution; 124–127 294 dence handling on; fn parallelizing/parallelism lock See also concurrent/concurrency; deadlock risks; 206–213 Decorator pattern; dynamic deadlock risks; 207–210 application analysis; 123–133 inconsistent, as multithreaded heterogeneous tasks; 127–129 GUI framework problem; 190 iterative algorithms operation barrier management of; 99 synchronization role; 35 puzzle-solving framework; 183–188 340 181 188 partial; fn recursive algorithms; – happens-before, JMM definition; serialization vs. 340–342 Amdahl’s law; 225–229 happens-before, piggybacking; task-related decomposition; 113 342–344 thread-per-task policy; 115 340 happens-before, publication con- partial ordering; fn sequences; 244–249 happens-before performance-based alterations in and publication; 244–249 thread safety risks; 7 JMM definition; 340 Index 385

piggybacking; 342–344 CAS-based operations; 323 partitioning reduction strategies; 232–242 as parallelizing strategy; 101 page renderer example with Com- passivation pletionService impact on HttpSession thread- improvements; 130 58 safety requirements; fn producer-consumer pattern advan- perfbar application tages; 90 See also measurement; tools; read-write lock advantages; 286–289 CPU performance measure; 261 ReentrantLock vs. intrinsic locks; performance measurement use; 225 282–286 perfmon application; 240 requirements See also measurement; tools; thread-safety impact; 16 I/O measurement; 240 scalability vs.; 222–223 performance measurement use; 230 issues, three-tier application performance; 8, 221, 221–245 model as illustration; 223 See also concurrent/concurrency; lock granularity reduction; 239 liveness; scalability; through- object pooling issues; 241 put; utilization; sequential event processing; 191 and heterogeneous tasks; 127 simplicity vs. 48 and immutable objects; fn in refactoring synchronized and resource management; 119 blocks; 34 atomic variables synchronized block scope; 30 326 329 174 locking vs.; – SynchronousQueue; fn cache implementation issues; 103 techniques for improving composition functionality extension atomic variables; 319–336 74 319 336 mechanism; fn nonblocking algorithms; – costs testing; 247–274 thread-per-task policy; 116 criteria; 248 fair vs. nonfair locking; 284 goals; 260 hazards pitfalls, avoiding; 266–270 See also overhead; priority(s), thread pool inversion; size impact; 170 JVM interaction with hardware tuning; 171–179 reordering; 34 thread safety hazards for; 8 liveness timing and ordering alterations for in servlets with state; 29–32 thread safety risks; 7 locking tradeoffs during iteration impact on; 83 evaluation of; 223–225 measurement of; 222 permission See also capacity; efficiency; la- codebase tency; scalability; service and custom thread factory; 177 time; throughput; permits; 98 locks vs. atomic variables; 326– See also semaphores; 329 pessimistic concurrency management memory barrier impact on; 230 See lock(ing), exclusive; notifyAll impact on; 303 piggybacking; 344 optimization on synchronization; 342–344 See also CPU utilization; piggy- point(s) backing; barrier; 99 Amdahl’s law; 225–229 cancellation; 140 bad practices; 348–349 386 Index poison thread confinement; 43 message; 219 polling See also livelock; blocking state-dependent actions; pill; 155, 155–156 295–296 See also lifecycle; shutdown; for interruption; 143 157 279 CrawlerThread; li lock acquisition; 157 IndexerThread; li pool(s) 156 IndexingService; li See also resource(s); unbounded queue shutdown object 155 241 with; appropriate uses; fn policy(s) bounded, semaphore use; 99 See also design; documentation; disadvantages of; 241 guidelines; protocol(s); serial thread confinement use; 90 strategies; resource application semaphore use; 98–99 thread pool advantages; 120 thread pool size impact; 171 cancellation; 136 size 171 172 for tasks, thread interruption core; , fn policy relationship to; 141 maximum; 172 interruption advantages as im- thread; 119–121 plementation strategy; 140 adding statistics to; 179 execution application; 167–188 design, influencing factors; 167 as producer-consumer design; 88 Executors,forThreadPoolExec- as thread resource management utor configuration; 171 mechanism; 117 implicit couplings between tasks callback use in testing; 258 and; 167–170 combined with work queues, in parallelism analysis for; 123–133 Executor framework; 119 task; 118–119 configuration post-construction task, application performance manipulation; 177–179 importance; 113 configuring task queue; 172–174 interruption; 141, 141–142 creating; 120 saturation; 174–175 deadlock risks; 215 security factory methods for; 171 custom thread factory handling; sizing; 170–171 177 uncaught exception handling; sequential 163 task execution; 114 portal sharing objects; 54 timed task example; 131–133 synchronization; 55 postconditions requirements, impact on class See also invariant(s); extension; 71 preservation of requirements, impact on class mechanisms and synchroniza- modification; 71 tion policy role; 55–56 shared state requirements for; 28 thread safety role; 17 task scheduling precondition(s) sequential; 114 See also dependencies, state; invari- thread pools; 117 ant(s); thread pools advantages over condition predicate as; 299 thread-per-task; 121 failure thread-per-task; 115 bounded buffer handling of; 292 Index 387

propagation to callers; 292–295 safety testing; 252 state-based work stealing vs.; 92 in state-dependent classes; 291 profiling management; 57 See also measurement; 320 predictability JVM use; fn See also responsiveness; tools measuring; 264–266 lock contention detection; 240 preemptive interruption performance measurement; 225 135 273 deprecation reasons; fn quality assurance; presentation programming See GUI; models primitive sequential; 2 local variables, safety of; 44 progress indication wrapper classes See also GUI; atomic scalar classes vs.; 325 in long-running GUI task; 198 priority(s) propagation inversion; 320 of interruption exception; 142 avoidance, nonblocking algo- protocol(s) rithm advantages; 329 See also documentation; policy(s); thread strategies; manipulation, liveness hazards; entry and exit 218 state-dependent operations; 306 when to use; 219 lock acquisition PriorityBlockingQueue; 89 instance confinement use; 60 thread pool use of; 173–174 locking PriorityQueue; 85 shared state requirements for; 28 private race condition handling; 21 69 constructor capture idiom; fn thread confinement locks atomicity preservation with Java monitor pattern vs.; 61 open calls; 213 probability pthreads (POSIX threads) 26 deadlock avoidance use with timed default locking behavior; fn and polled locks; 279 publication; 39 determinism vs. See also confinement; documenta- in concurrent programs; 247 tion; encapsulation; sharing; process(es); 1 escape and; 39–42 communication mechanisms; 1 improper; 51, 50–51 lightweight JMM support; 244–249 See threads; of immutable objects threads vs.; 2 volatile use; 48–49 producer-consumer pattern safe; 346 and Executor functionality idioms for; 52–53 in CompletionService; 129 in task creation; 126 blocking queues and; 87–92 of mutable objects; 54 bounded buffer use; 292 serial thread confinement use; 90 control flow coordination safety guidelines; 49–54 blocking queues use; 94 state variables Executor framework use; 117 safety, requirements for; 68–69 pathological waiting conditions; unsafe; 344–346 300 fn performance testing; 261 388 Index put-if-absent operation R See also compound actions; race conditions; 7, 20–22 as compound action See also concurrent/concurrency, atomicity requirements; 71 errors; data, race; time/tim- concurrent collection support for; 84 ing; puzzle solving framework avoidance as parallelization example; 183–188 immutable object use; 48 in thread-based service shut- Q down; 153 quality assurance in GUI frameworks; 189 See also testing; in web crawler example strategies; 270–274 idempotence as mitigating cir- quality of service cumstance; 161 measuring; 264 random(ness) requirements livelock resolution use; 219 and task execution policy; 119 pseudorandom number generation Queue; 84–85 scalability; 326–329 queue(s) test data generation use; 253 See also data structures; reachability blocking; 87–94 publication affected by; 40 cancellation, problems; 138 read-modify-write operation cancellation, solutions; 140 See also compound actions; CompletionService as; 129 as non-atomic operation; 20 producer-consumer pattern and; read-write locks; 286–289 87 92 286 – ReadWriteLock; li bounded exclusive locking vs.; 239 saturation policies; 174–175 reaping condition; 297 See termination; blocking state-dependent opera- reclosable thread gate; 304 tions use; 296–308 recovery, deadlock intrinsic; 297 See deadlock, recovery; intrinsic, disadvantages of; 306 recursion FIFO; 89 See also control flow; iterators/itera- implementations tion; 227 237 serialization differences; intrinsic lock acquisition; fn priority-ordered; 89 parallelizing; 181–188 synchronous See also Decorator pattern; design constraints; 89 reentrant/reentrancy; 26 thread pool use of; 173 and read-write locks; 287 task locking semantics; 26–27 thread pool use of; 172–174 ReentrantLock capabilities; 278 unbounded per-thread lock acquisition; 26–27 poison pill shutdown; 156 ReentrantLock; 277–282 using; 298 ReentrantLock work AQS use; 314–315 in thread pools; 88, 119 intrinsic locks vs. performance; 282–286 Lock implementation; 277–282 random number generator using; 327 li Semaphore relationship with; 308 Index 389

ReentrantReadWriteLock lifecycle, Future use for; 125 AQS use; 316–317 Runnable use for; 125 reentrant locking semantics; 287 with Future; 126 references thread; 150 stack confinement precautions; 44 request reflection interrupt atomic field updater use; 335 strategies for handling; 140 rejected execution handler requirements ExecutorService post-termination See also constraints; design; docu- task handling; 121 mentation; performance; puzzle-solving framework; 187 concrete RejectedExecutionException importance for effective perfor- abort saturation policy use; 174 mance optimization; 224 post-termination task handling; 122 concurrency testing puzzle-solving framework use; 187 TCK example; 250 RejectedExecutionHandler determination and saturation policy; 174 importance of; 223 release independent state variables; 66–67 AQS synchronizer operation; 311 performance lock Amdahl’s law insights; 229 in hand-over-hand locking; 282 thread-safety impact; 16 intrinsic locking disadvantages; synchronization 278 synchronization policy compo- preferences in read-write lock nent; 56–57 implementation; 287 synchronization policy documenta- unreleased lock bug pattern; 271 tion; 74–77 permit resource exhaustion, preventing semaphore management; 98 bounded queue use; 173 remote objects execution policy as tool for; 119 thread safety concerns; 10 testing strategies; 257 remove-if-equal operation thread pool sizing risks; 170 as atomic collection operation; 86 resource(s) reordering; 34 See also CPU; instrumentation; mem- See also deadlock; optimization; or- ory; object(s); pool(s); utiliza- der(ing); ordering; synchro- tion; nization; time/timing; accessing initialization safety limitation; 350 as long-running GUI task; 195 memory bound; 221 barrier impact on; 230 consumption operations; 339 thread safety hazards for; 8 volatile variables warning; 38 deadlocks; 213–215 replace-if-equal operation depletion as atomic collection operation; 86 thread-per-task policy issue; 116 representation increase See also algorithm(s); design; docu- scalability relationship to; 222 mentation; state(s); leakage activities testing for; 257 tasks use for; 113 management algorithm design role; 104 See also instrumentation; testing; result-bearing tasks; 125 dining philosophers prob- task lem; 390 Index

blocking queue advantages; 88 safety vs. execution policy as tool for; 119 graceful vs. abrupt shutdown; Executor framework use; 117 153 finalizer use and limitations; 165 sequential execution limitations; 124 graceful degradation, saturation server applications policy advantages; 175 importance of; 113 long-running task handling; 170 single-threaded execution disad- saturation policies; 174–175 vantages; 114 single-threaded task execution sleeping impact on; 295 disadvantages; 114 thread testing; 257 pool tuning, ThreadPoolExecut- thread pools; 117 or use; 171–179 thread pools, advantages; 121 request overload impact; 173 thread pools, tuning; 171–179 safety hazards for; 8 thread-per-task policy disadvan- restoring interruption status; 142 tages; 116 result(s) threads, keep-alive time impact -bearing latches on; 172 puzzle framework use; 184 timed task handling; 131 cache performance building; 101–109 analysis, monitoring, and im- Callable handling of; 125 provement; 221–245 Callable use instead of Runnable; pools 95 semaphore use; 98–99 dependencies thread pool size impact; 171 task freedom from, importance utilization of; 113 Amdahl’s law; 225 Future handling of; 125 as concurrency motivation; 1 handling response-time-senstive tasks as serialization source; 226 execution policy implications; 168 irrelevancy responsiveness as cancellation reason; 136, 147 See also deadlock; GUI; livelock; live- non-value-returning tasks; 125 ness; performance; Runnable limitations; 125 as performance testing criteria; 248 retry condition queues advantages; 297 randomness, in livelock resolution; efficiency vs. 219 polling frequency; 143 return values interruption policy Runnable limitations; 125 InterruptedException advan- reuse tages; 142 existing thread-safe classes long-running tasks strategies and risks; 71 handling; 170 RMI (Remote Method Invocation) measuring; 264–266 thread use; 9, 10 page renderer example with Com- safety concerns and; 10 pletionService threads benefits for; 4 improvements; 130 robustness performance See also fragility; safety; analysis, monitoring, and im- blocking queue advantages; 88 provement; 221–245 InterruptedException advantages; poor 142 causes and resolution of; 219 thread pool advantages; 120 Index 391 rules scalability; 222, 221–245 See also guidelines; policy(s); strate- algorithm gies; comparison testing; 263–264 happens-before; 341 Amdahl’s law insights; 229 Runnable as performance testing criteria; 248 handling exceptions in; 143 client-side locking impact on; 81 task representation limitations; 125 concurrent collections vs. synchro- running nized collections; 84 ExecutorService state; 121 ConcurrentHashMap advantages; 85, FutureTask state; 95 242 runtime CPU utilization monitoring; 240–241 timing and ordering alterations by enhancement thread safety risks; 7 reducing lock contention; 232– RuntimeException 242 as thread death cause; 161 heterogeneous task issues; 127 Callable handling; 98 hotfieldimpacton;237 catching intrinsic locks vs. ReentrantLock disadvantages of; 161 performance; 282–286 lock scope impact on; 233 S locking during iteration risk of; 83 safety open call strategy impact on; 213 See also encapsulation; immutable performance vs.; 222–223 objects; synchronization; lock granularity reduction; 239 thread(s), confinement; object pooling issues; 241 cache implementation issues; 104 three-tier application model as initialization illustration; 223 guarantees for immutable ob- queue implementations jects; 51 serialization differences; 227 idioms for; 346–348 result cache JMM support; 349–350 building; 101–109 liveness vs.; 205–220 serialization impact on; 228 publication techniques for improving idioms for; 52–53 atomic variables; 319–336 in task creation; 126 nonblocking algorithms; 319–336 of mutable objects; 54 testing; 261 responsiveness vs. thread safety hazards for; 8 as graceful vs. abrupt shutdown; under contention 153 as AQS advantage; 311 split ownership concerns; 58 ScheduledThreadPoolExecutor subclassing issues; 304 as Timer replacement; 123 testing; 252–257 scheduling goals; 247 overhead tradeoffs performance impact of; 222 in performance optimization priority manipulation risks; 218 strategies; 223–224 tasks untrusted code behavior sequential policy; 114 protection mechanisms; 161 thread-per-task policy; 115 saturation threads as basic unit of; 3 policies; 174–175 work stealing deques and; 92 392 Index scope/scoped counting; 98 See also granularity; permits, thread relationships; 248 containers fn 10 310 thread safety concerns; SemaphoreOnLock example; li contention fair vs. nonfair atomic variable limitation of; 324 performance comparison; 265 escaping nonfair publication as mechanism for; 39 advantages of; 265 281 lock sendOnSharedLine example; li narrowing, as lock contention sequential/sequentiality reduction strategy; 233–235 See also concurrent/concurrency; synchronized block; 30 asynchrony vs.; 2 search consistency; 338 depth-first event processing breadth-first search vs.; 184 in GUI applications; 191 parallelization of; 181–182 execution security policies of tasks; 114 and custom thread factory; 177 parallelization of; 181 Selector orderly shutdown strategy; 164 non-interruptable blocking; 148 page renderer example; 124–127 semantics programming model; 2 See also documentation; representa- task execution policy; 114 tion; tests, value in concurrency testing; atomic arrays; 325 250 binary semaphores; 99 threads simulation of; 4 final fields; 48 serialized/serialization of interruption; 93 access 27 of multithreaded environments object serialization vs.; fn ThreadLocal variable considera- timed lock use; 279 46 227 tions; WorkerThread; li reentrant locking; 26–27 granularity ReentrantLock capabilities; 278 throughput impact; 228 ReentrantReadWriteLock capa- impact on HttpSession thread- 287 58 bilities; safety requirements; fn undefined parallelization vs. of Thread.yield; 218 Amdahl’s law; 225–229 volatile; 39 scalability impact; 228 weakly consistent iteration; 85 serial thread confinement; 90, 90–92 within-thread-as-if-serial; 337 sources Semaphore; 98 identification of, performance AQS use; 315–316 impact; 225 100 176 249 example use; li, li, li server in BoundedBuffer example; 248 See also client; saturation policy use; 175 applications similarities to ReentrantLock; 308 context switch reduction; 243– state-based precondition manage- 244 ment with; 57 design issues; 113 semaphores; 98, 98–99 service(s) as coordination mechanism; 1 See also applications; frameworks; binary logging mutex use; 99 Index 393

as thread-based service example; guidelines; 54 150–155 objects; 33–54 shutdown split data models; 201–202 as cancellation reason; 136 state thread-based managing access to, as thread stopping; 150–161 safety goal; 15 servlets strategies framework ExecutorCompletionService thread safety requirements; 10 use; 130 threads beneﬁts for; 4 thread stateful, thread-safety issues necessities and dangers in GUI atomicity; 19–23 applications; 189–190 liveness and performance; 29–32 volatile variables as mechanism for; locking; 23–29 38 stateless shutdown as thread-safety example; 18–19 See also lifecycle; session-scoped objects abrupt thread safety concerns; 10 JVM, triggers for; 164 set(s) limitations; 158–161 See also collection(s); as cancellation reason; 136 100 135 166 BoundedHashSet example; li cancellation and; – CopyOnWriteArraySet ExecutorService state; 121 as synchronized Set replace- graceful vs. abrupt tradeoffs; 153 ment; 86 hooks; 164 safe publication use; 52 in orderly shutdown; 164–165 59 164 166 PersonSet example; li JVM; – SortedSet and daemon threads; 165 ConcurrentSkipListSet as con- of thread-based services; 150–161 current replacement; 85 orderly; 164 TreeSet strategies ConcurrentSkipListSet as con- lifecycle method encapsulation; current replacement; 85 155 shared/sharing; 15 logging service example; 150– See also concurrent/concurrency; 155 publication; one-shot execution service exam- data ple; 156–158 See also page renderer examples; support access coordination, explicit lock LifecycleWebServer example; 277 290 122 use; – li models, GUI application han- shutdown; 121 dling; 198–202 logging service shutdown alterna- synchronization costs; 8 tives; 153 threads advantages vs. pro- shutdownNow; 121 cesses; 2 limitations; 158–161 data structures logging service shutdown alterna- as serialization source; 226 tives; 153 memory side-effects as coordination mechanism; 1 as serialization source; 226 memory multiprocessors freedom from memory models; 338–339 importance for task indepen- mutable objects dence; 113 394 Index

synchronized Map implementations sleeping not available from Concurrent- blocking state-dependent actions HashMap; 86 blocking state-dependent ac- signal tions; 295–296 ConditionBoundedBuffer example; sockets 308 as coordination mechanism; 1 signal handlers synchronous I/O as coordination mechanism; 1 non-interruptable blocking rea- simplicity son; 148 See also design; solutions Java monitor pattern advantage; 61 See also interruption; results; search; of modeling termination; threads benefit for; 3 SortedMap performance vs. ConcurrentSkipListMap as concur- in refactoring synchronized rent replacement; 85 blocks; 34 SortedSet simulations ConcurrentSkipListSet as concur- barrier use in; 101 rent replacement; 85 single notification space See notify; signal; state; 56 single shutdown hook specification See also hook(s); See also documentation; orderly shutdown strategy; 164 correctness defined in terms of; 17 single-thread(ed) spell checking See also thread(s); thread(s), confine- as long-running GUI task; 195 ment; spin-waiting; 232, 295 as Timer restriction; 123 See also blocking/blocks; busy- as synchronization alternative; 42–46 waiting; 43 273 deadlock avoidance advantages; fn as concurrency bug pattern; subsystems split(ing) GUI implementation as; 189–190 data models; 201, 201–202 task execution lock; 235 disadvantages of; 114 Amdahl’s law insights; 229 executor use, concurrency pre- as lock granularity reduction vention; 172, 177–178 strategy; 235 236 Singleton pattern ServerStatus examples; li ThreadLocal variables use with; 45 ownership; 58 size(ing) stack(s) See also configuration; instrumenta- address space 116 tion; thread creation constraint; fn as performance testing goal; 260 confinement; 44, 44–45 bounded buffers See also confinement; encapsula- determination of; 261 tion; heterogeneous tasks; 127 nonblocking; 330 pool size 171 172 184 core; , fn search strategy impact; maximum; 172 trace task thread dump use; 216 appropriate; 113 stale data; 35–36 thread pools; 170–171 improper publication risk; 51 20 race condition cause; fn Index 395 starvation; 218, 218 thread-safety issues, atomicity; See also deadlock; livelock; liveness; 19–23 performance; thread-safety issues, liveness as liveness failure; 8 and performance concerns; locking during iteration risk of; 83 29–32 thread starvation deadlock; 169, thread-safety issues, locking; 168–169 23–29 thread starvation deadlocks; 215 space; 56 state(s); 15 stateless servlet See also atomic/atomicity; encapsu- as thread-safety example; 18–19 lation; lifecycle; representa- task tion; safety; visibility; impact on Future.get; 95 application intermediate, shutdown issues; framework threads impact on; 9 158–161 code vs. transformations thread-safety focus; 17 in puzzle-solving framework dependent example; 183–188 classes; 291 transition constraints; 56 classes, building; 291–318 variables operations; 57 condition predicate use; 299 operations, blocking strategies; independent; 66, 66–67 291–308 independent, lock splitting; 235 operations, condition queue han- safe publication requirements; dling; 296–308 68–69 291 301 operations, managing; stateDependentMethod example; li task freedom from, importance static of; 113 initializer encapsulation safe publication mechanism; 53, breaking, costs of; 16–17 347 invariant protection use; 83 static analysis tools; 271–273 synchronizer role; 94 statistics gathering thread-safe class use; 23 See also instrumentation; lifecyle adding to thread pools; 179 ExecutorService methods; 121 ThreadPoolExecutor hooks for; 179 locks control of; 27–29 status logical; 58 flag management volatile variable use with; 38 AQS-based synchronizer opera- interrupted; 138 tions; 311 thread managing access to shutdown issues; 158 as thread safety goal; 15 strategies modification See also design; documentation; visibility role; 33 guidelines; policy(s); rep- mutable resentation; coordinating access to; 110 atomic variable use; 34 object; 55 cancellation components of; 55 Future use; 145–147 remote and thread safety; 10 deadlock avoidance; 208, 215–217 ownership delegation class design issues; 57–58 vehicle tracking example; 64 servlets with design 396 Index

interruption policy; 93 suspension, thread documentation use costs of; 232, 320 annotations value; 6 elimination by CAS-based concur- end-of-lifecycle management; 135– rency mechanisms; 321 166 Thread.suspend, deprecation rea- 93 135 InterruptedException handling; sons; fn interruption handling; 140, 142–150 swallowing interrupts Future use; 146 as discouraged practice; 93 lock splitting; 235 bad consequences of; 140 locking when permitted; 143 ConcurrentHashMap advantages; Swing 85 See also GUI; monitor listeners vehicle tracking example; 61 single-thread rule exceptions; parallelization 192 partitioning; 101 methods performance improvement; 30 single-thread rule exceptions; program design order 191–192 correctness then performance; 16 thread search conﬁnement; 42 stack size impact on; 184 conﬁnement in; 191–192 shutdown use; 9 lifecycle method encapsulation; use, safety concerns and; 10–11 155 untrusted code protection mecha- logging service example; 150– nisms in; 162 155 SwingWorker one-shot execution service exam- long-running GUI task support; 198 ple; 156–158 synchronization/synchronized; 15 poison pill; 155–156 See also access; concurrent/concur- split ownership safety; 58 rency; lock(ing); safety;; thread safety delegation; 234–235 allocation advantages vs.; 242 thread-safe class extension; 71 bad practices stream classes double-checked locking; 348–349 150 25 client-side locking with; fn blocks; thread safety; 150 Java objects as; 25 String cache implementation issues; 103 47 79 84 immutability characteristics; fn collections; – striping concurrent collections vs.; 84 See also contention; problems with; 79–82 lock; 237, 237 concurrent building blocks; 79–110 Amdahl’s law insights; 229 contended; 230 ConcurrentHashMap use; 85 correctly synchronized program; 341 structuring data sharing requirements for; 33–39 thread-safe classes encapsulation object composition use; 55–78 hidden iterator management subclassing through; 83 safety issues; 304 requirement for thread-safe submit, execute vs. classes; 18 uncaught exception handling; 163 ’fast path’ CAS-based operations vs.; 324 costs of; 230 Index 397

immutable objects as replacement; T 52 task(s); 113 inconsistent See also activities; event(s); lifecycle; as concurrency bug pattern; 271 asynchronous memory FutureTask handling; 95–98 performance impact of; 230–231 boundaries; 113 memory visibility use of; 33–39 parallelism analysis; 123–133 operation ordering role; 35 using ThreadLocal in; 168 piggybacking; 342–344 cancellation; 135–150 policy; 55 policy; 136 documentation requirements; thread interruption policy rela- 74–77 tionship to; 141 encapsulation, client-side lock- completion ing violation of; 71 as cancellation reason; 136 race condition prevention with; 7 service time variance relation- requirements, impact on class ship to; 264–266 extension; 71 dependencies requirements, impact on class execution policy implications; modiﬁcation; 71 167 shared state requirements for; 28 thread starvation deadlock risks; ReentrantLock capabilities; 277 168 requirements execution; 113–134 synchronization policy compo- in threads; 113–115 nent; 56–57 policies; 118–119 thread safety need for; 5 policies and, implicit couplings types between; 167–170 See barriers; blocking, queues; policies, application perfor- FutureTask; latches; mance importance; 113 semaphores; sequential; 114 uncontended; 230 explicit thread creation for; 115 volatile variables vs.; 38 GUI wrapper long-running tasks; 195–198 client-side locking support; 73 short-running tasks; 192–195 synchronizedList (Collections) heterogeneous tasks safe publication use; 52 parallelization limitations; 127– synchronizer(s); 94, 94–101 129 See also Semaphore; CyclicBarrier; homogeneous tasks FutureTask; Exchanger; parallelism advantages; 129 CountDownLatch; lifecycle behavior and interface; 308–311 Executor phases; 125 building ExecutorService methods; 121 with AQS; 311 representing with Future; 125 with condition queues; 291–318 long-running synchronous I/O responsiveness problems; 170 non-interruptable blocking; 148 parallelization of SynchronousQueue; 89 homogeneous vs. heteroge- 174 129 performance advantages; fn neous; thread pool use of; 173, 174 post-termination handling; 121 queues management, thread pool con- ﬁguration issues; 172–174 398 Index

thread pool use of; 172–174 vs. performance tests; 260 representation liveness Runnable use for; 125 criteria; 248 with Future; 126 performance; 260–266 response-time sensitivity criteria; 248 andexecution policy; 168 goals; 260 scheduling pitfalls thread-per-task policy; 115 avoiding; 266–270 serialization sources dead code elimination; 269 identifying; 225 dynamic compilation; 267–268 state garbage collection; 266 effect on Future.get; 95 progress quantiﬁcation; 248 intermediate, shutdown issues; proving a negative; 248 158–161 timing and synchronization arti- thread(s) vs. facts; 247 interruption handling; 141 unrealistic code path sampling; timed 268 handling of; 123 unrealistic contention; 268–269 two-party program correctness; 248–260 Exchanger management of; 101 safety; 252–257 TCK (Technology Compatibility Kit) criteria; 247 concurrency testing requirements; strategies; 270–274 250 258 testPoolExample example; li teardown testTakeBlocksWhenEmpty example; 171 172 252 thread; – li techniques this reference See also design; guidelines; strate- publication risks; 41 gies; Thread temporary objects join and ThreadLocal variables; 45 timed, problems with; 145 terminated getState ExecutorService state; 121 use precautions; 251 138 139 termination interruption methods; , li See also cancellation; interruption; usage precautions; 140 lifecycle; thread safety; 18, 15–32 puzzle-solving framework; 187 and mutable data; 35 safety test and shutdown hooks; 164 criteria for; 254, 257 characteristics of; 17–19 thread data models, GUI application han- abnormal, handling; 161–163 dling; 201 keep-alive time impact on; 172 delegation; 62 135 234 reasons for deprecation of; fn delegation of; timed locks use; 279 in puzzle-solving framework; 183 262 19 23 test example method; li issues, atomicity; – testing issues, liveness and performance; See also instrumentation; logging; 29–32 measurement; monitoring; mechanisms, locking; 23–29 quality assurance; statistics; risks; 5–8 concurrent programs; 247–274 thread(s); 2 210 deadlock risks; fn See also concurrent/concurrency; functionality safety; synchronization; Index 399

abnormal termination of; 161–163 ownership; 150 as instance confinement context; 59 pools; 119–121 benefits of; 3–5 adding statistics to; 179 blocking; 92 and work queues; 119 confinement; 42, 42–46 application; 167–188 See also confinement; encapsula- as producer-consumer design; 88 tion; as thread resource management ad-hoc; 43 mechanism; 117 and execution policy; 167 callback use in testing; 258 in GUI frameworks; 190 creating; 120 in Swing; 191–192 deadlock risks; 215 role, synchronization policy factory methods for; 171 specification; 56 post-construction configuration; stack; 44, 44–45 177–179 ThreadLocal; 45–46 sizing; 170–171 cost task queue configuration; 172– context locality loss; 8 174 context switching; 8 priorities costs; 229–232 manipulation, liveness risks; 218 creation; 171–172 priority explicit creation for tasks; 115 when to use; 219 unbounded, disadvantages; 116 processes vs.; 2 daemon; 165 queued dumps; 216 SynchronousQueue management deadlock analysis use; 216–217 of; 89 intrinsic lock advantage over risks of; 5–8 ReentrantLock; 285 serial thread confinement; 90, 90–92 lock contention analysis use; 240 services that own factories; 175, 175–177 stopping; 150–161 failure sharing uncaught exception handlers; necessities and dangers in GUI 162–163 applications; 189–190 forced termination single 135 114 reasons for deprecation of; fn sequential task execution; interleaving sources of; 9–11 dangers of; 5–8 starvation deadlock; 169, 168–169 interruption; 138 suspension shutdown issues; 158 costs of; 232, 320 status flag; 138 Thread.suspend, deprecation 161 135 leakage; reasons; fn testing for; 257 task Timer problems with; 123 execution in; 113–115 UncaughtExceptionHandler scheduling, thread-per-task pol- prevention of; 162–163 icy; 115 lifecycle scheduling, thread-per-task pol- performance impact; 116 icy disadvantages; 116 thread-based service manage- vs. interruption handling; 141 ment; 150 teardown; 171–172 overhead termination in safety testing, strategies for keep-alive time impact on; 172 mitigating; 254 thread starvation deadlocks; 215 400 Index thread-local -based task See also stack, confinement; handling; 123 computation management design issues; 131– role in accurate performance 133 testing; 268 barrier handling based on; 99 Thread.stop constraints 135 136 deprecation reasons; fn as cancellation reason; Thread.suspend in puzzle-solving framework; 135 187 deprecation reasons; fn 176 144 145 ThreadFactory; li interruption handling; – customizing thread pool with; 175 deadline-based waits ThreadInfo as feature of Condition; 307 and testing; 273 deferred computations ThreadLocal; 45–46 design issues; 125 and execution policy; 168 dynamic compilation for thread confinement; 43 as performance testing pitfall; risks of; 46 267 ThreadPoolExecutor granularity and untrusted code; 162 measurement impact; 264 configuration of; 171–179 keep-alive 172 172 constructor; li thread termination impact; 179 207 extension hooks; LeftRightDeadlock example; fg 126 148 279 newTaskFor; li, lock acquisition; @ThreadSafe; 7, 353 lock scope throttling narrowing, as lock contention as overload management mecha- reduction strategy; 233–235 nism; 88, 173 long-running GUI tasks; 195–198 saturation policy use; 174 long-running tasks Semaphore use in BoundedExecutor responsiveness problem han- 176 170 example; li dling; throughput measuring See also performance; in performance testing; 260–263 as performance testing criteria; 248 ThreadPoolExecutor hooks for; locking vs. atomic variables; 328 179 producer-consumer handoff performance-based alterations in testing; 261 thread safety risks; 7 queue implementations periodic tasks serialization differences; 227 handling of; 123 server application progress indication importance of; 113 for long-running GUI tasks; 198 server applications relative vs. absolute 123 single-threaded task execution class choices based on; fn disadvantages; 114 response thread safety hazards for; 8 task sensitivity to, execution threads benefit for; 3 policy implications; 168 Throwable short-running GUI tasks; 192–195 FutureTask handling; 98 thread timeout time/timing core pool size parameter impact 172 See also deadlock; lifecycle; or- on; fn der/ordering; race condi- timed locks; 215–216 tions; Index 401

weakly consistent iteration seman- TreeMap tics; 86 ConcurrentSkipListMap as concur- TimeoutException rent replacement; 85 in timed tasks; 131 TreeSet task cancellation criteria; 147 ConcurrentSkipListSet as concur- Timer rent replacement; 85 task-handling issues; 123 Treiber’s nonblocking stack algorithm; 9 331 thread use; li timesharing systems trigger(ing) as concurrency mechanism; 2 See also interruption; tools JVM abrupt shutdown; 164 See also instrumentation; measure- thread dumps; 216 ment; try-catch block annotation use; 353 See also exceptions; code auditing as protection against untrusted code 28 161 locking failures detected by; fn behavior; heap inspection; 257 try-finally block measurement See also exceptions; I/O utilization; 240 and uncaught exceptions; 163 importance for effective perfor- as protection against untrusted code mance optimization; 224 behavior; 161 performance; 230 tryLock 283 monitoring barging use; fn 273 280 quality assurance use; deadlock avoidance; li 281 profiling trySendOnSharedLine example; li lock contention detection; 240 tuning performance measurement; 225 See also optimization; quality assurance use; 273 thread pools; 171–179 static analysis; 271–273 transactions U See also events; unbounded concurrent atomicity similar to; 25 See also bounded; constraints; transformations queue(s); state blocking waits in puzzle-solving framework timed vs., in long-running task example; 183–188 management; 170 transition queues See also state; nonblocking characteristics; 87 state transition constraints; 56 poison pill shutdown use; 155 impact on safe state variable thread pool use of; 173 publication; 69 thread creation travel reservations portal example disadvantages of; 116 as timed task example; 131–133 uncaught exception handlers; 162–163 tree(s) See also exceptions; 163 See also collections; UncaughtExceptionHandler; li models custom thread class use; 175 GUI application handling; 200 thread leakage detection; 162–163 traversal unchecked exceptions parallelization of; 181–182 See also exceptions; catching disadvantages of; 161 402 Index uncontended explicit; 306–308 synchronization; 230 hoisting 38 unit tests as JVM optimization pitfall; fn for BoundedBuffer example; 250 local issues; 248 stack confinement use; 44 untrusted code behavior multivariable invariant requirements See also safety; for atomicity; 57 ExecutorService code protection state strategies; 179 condition predicate use; 299 protection mechanisms; 161 independent; 66, 66–67 updating independent, lock splitting use See also lifecycle; with; 235 atomic fields; 335–336 object data stored in; 15 immutable objects; 47 safe publication requirements; views 68–69 in GUI tasks; 201 ThreadLocal; 45–46 upgrading volatile; 38, 37–39 read-write locks; 287 atomic variable class use; 319 usage scenarios atomic variable vs.; 39, 325–326 performance testing use; 260 multivariable invariants prohib- user ited from; 68 See also GUI; variance cancellation request service time; 264 as cancellation reason; 136 Vector feedback as safe publication use; 52 196 79 in long-running GUI tasks; li as synchronized collection; 80 79 interfaces check-then-act operations; li, – threads benefits for; 5 80 utilization; 225 client-side locking management of 81 See also performance; resource(s); compound actions; li CPU vehicle tracking example 225 226 64 Amdahl’s law; , fg delegation strategy; optimization, as multithreading monitor strategy; 61 goal; 222 state variable publication strategy; sequential execution limitations; 69–71 124 thread-safe object composition de- hardware sign; 61–71 improvement strategies; 222 versioned data model; 201 views V event handling 195 value(s) model-view objects; fg See result(s); model-view-controller pattern variables deadlock risks; 190 See also encapsulation; state; vehicle tracking example; 61 atomic reflection-based classes; 324–329 by atomic field updaters; 335 locking vs.; 326–329 timeliness vs. consistency; 66, 70 nonblocking algorithms and; updating 319–336 in long-running GUI task han- volatile variables vs.; 39, 325–326 dling; 201 condition with split data models; 201 Index 403 visibility multiple, as feature of Condi- See also encapsulation; safety; scope; tion; 307 condition queue spin-waiting; 232 control, explicit Condition and as concurrency bug pattern; 273 Lock use; 306 waiting to run guarantees FutureTask state; 95 JMM specification of; 338 waking up lock management of; 36–37 See also blocking/blocks; condition, memory; 33–39 queues; notify; sleep; wait; ReentrantLock capabilities; 277 condition queue handling; 300–301 synchronization role; 33 weakly consistent iterators; 85 volatile reference use; 49 See also iterators/iteration; vmstat application web crawler example; 159–161 See also measurement; tools; within-thread usage CPU utilization measurement; 240 See stack, confinement; performance measurement; 230 within-thread-as-if-serial semantics; thread utilization measurement; 241 337 Void work non-value-returning tasks use; 125 queues volatile and thread pools, as producer- cancellation flag use; 136 consumer design; 88 158 119 final vs.; fn in Executor framework use; publishing immutable objects with; thread pool interaction, size tun- 48–49 ing requirements; 173 safe publication use; 52 sharing variables; 38, 37–39 deques advantages for; 92 atomic variable class use; 319 stealing scheduling algorithm; 92 atomic variable vs.; 39, 325–326 deques and; 92 atomicity disadvantages; 320 tasks as representation of; 113 multivariable invariants prohib- wrapper(s) ited from; 68 factories thread confinement use with; 43 Decorator pattern; 60 synchronized wrapper classes W as synchronized collection wait(s) classes; 79 blocking client-side locking support; 73 timed vs. unbounded; 170 busy-waiting; 295 condition and condition predicate; 299 301 canonical form; li errors, as concurrency bug pattern; 272 interruptible, as feature of Con- dition; 307 uninterruptable, as feature of Condition; 307 waking up from, condition queue handling; 300–301 sets; 297