C 2010 Robert L. Bocchino Jr. Chapters 3 and 5 Are Derived from Work Published in ACM Conference Proceedings (OOPSLA 2009 and POPL 2011)
Total Page:16
File Type:pdf, Size:1020Kb
c 2010 Robert L. Bocchino Jr. Chapters 3 and 5 are derived from work published in ACM conference proceedings (OOPSLA 2009 and POPL 2011). As to that work only, the following notice applies: Copyright c 2009, 2011 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept., ACM, Inc., fax +1 (212) 869-0481, or [email protected]. AN EFFECT SYSTEM AND LANGUAGE FOR DETERMINISTIC-BY-DEFAULT PARALLEL PROGRAMMING BY ROBERT L. BOCCHINO DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate College of the University of Illinois at Urbana-Champaign, 2010 Urbana, Illinois Doctoral Committee: Associate Professor Vikram Adve, Chair and Director of Research Professor David Padua Associate Professor Grigore Ros¸u Professor Marc Snir Bradford Chamberlain, Cray Inc. Associate Professor Dan Grossman, University of Washington Abstract This thesis presents a new, Java-based object-oriented parallel language called Deterministic Parallel Java (DPJ). DPJ uses a novel effect system to guarantee determinism by default. That means that parallel programs are guaranteed to execute deterministically unless nondeterminism is explicitly requested. This is in contrast to the shared-memory models in widespread use today, such as threads and locks (including threads in ordinary Java). Those models are inherently nondeterministic, do not provide any way to check or enforce that a computation is deterministic, and can even have unintended data races, which can lead to strange and unexpected behaviors. Because deterministic programs are much easier to reason about than arbitrary parallel code, determinism by default simplifies parallel programming. This thesis makes several broad contributions to the state of the art in programming languages and effect systems. First, it presents a comprehensive research agenda for achieving determinism by default in parallel languages with reference aliasing and shared mutable state. It argues that an object-oriented effect system is a good approach to managing shared memory conflicts. It also raises several technical challenges, many of which are taken up in the rest of the thesis. Second, this thesis presents an effect system and language for deterministic parallel programming using a fork-join model of parallel control. With simple modular checking, and with no runtime checking overhead, the effect system guarantees at compile time that there are no conflicting memory accesses between any pairs of parallel tasks. The effect system supports several important patterns of deterministic parallelism that previous systems cannot express. We describe the effect system and language both formally and informally, and prove soundness for the formal language. We also describe our evaluation showing that the language can express a range of parallel programming patterns with good performance. Third, this thesis extends the effect system and language for determinism to support a controlled form of nondeterminism. Conflicting accesses are allowed only for an explicitly identified nondeterministic parallel construct, so the language is deterministic by default. A transactional runtime provides isolation for atomic ii statements, while the extended effect system provides stronger compile-time safety guarantees than any system we know of. In addition to determinism by default, the language guarantees race freedom; strong isolation for atomic statements even if the runtime guarantees only weak isolation; and an elegant way of composing deterministic and nondeterministic operations that preserves local reasoning about deterministic operations. Again we give an informal treatment, a formal treatment, and soundness proofs. We describe an evaluation showing that the extended language can express realistic nondeterministic algorithms in a natural way, with reasonable performance given the transactional runtime we used. Further, by eliminat- ing unnecessary synchronization, the effect system enables a significant reduction in the software runtime overhead. Fourth, this thesis describes programming techniques and further extensions to the effect system for supporting object-oriented parallel frameworks. Frameworks represent an important tool for parallel pro- gramming in their own right. They can also express some operations that the language and effect system alone cannot, for example pipeline parallelism. We show how to write a framework API using the DPJ effect system so that the framework writer can guarantee correctness properties to the user, assuming the user’s code passes the DPJ type checker. We also show how to extend the DPJ effect system to add generic types and effects, making the frameworks more general and useful. Finally, we state the requirements for a correct framework implementation. These requirements may be checked with a combination of DPJ’s effect system and external reasoning. Again we give an informal treatment, a formal treatment, and soundness proofs. We also describe the results of an evaluation showing that the techniques described can express realistic frameworks and parallel algorithms. iii Acknowledgments I am grateful to many people for supporting and contributing to the content of this thesis: • My advisor, Vikram Adve, unwaveringly encouraged, supported, and guided this work. He also contributed many ideas to the work. • Sarita Adve and Marc Snir contributed many useful discussions about the benefits of determinis- tic programming, the interaction between deterministic and nondeterministic code, and the various mechanisms for expressing determinism and nondeterminism. Their work on our paper in HotPar 2009, together with Vikram and myself, defined the agenda for this thesis. • Danny Dig was extremely helpful in evaluating DPJ, and in figuring out how to explain the type system details to a general audience. He also encouraged the use of ForkJoinTask for the DPJ runtime. • Dan Grossman, Grigore Ros¸u, Chris Rodrigues, and Madhusudan Parthasarathy gave invaluable feed- back on the formal aspects of the work. • Brad Chamberlain, David Padua, and Ralph Johnson gave invaluable feedback on the language as- pects of the work, including usability. Brad also worked closely with me on a related project involving software transactional memory (STM) for large-scale clusters. While not directly appearing in this thesis, that work gave me lots of insight into both STMs and parallel programming that I brought to bear on this work. • Adam Welc, Tatiana Shpeisman, and Yang Ni contributed insights into the performance character- istics and semantic issues raised by software transactional memory. Adam also helped build the transactional runtime for DPJ. iv • Rakesh Komuravelli, Stephen Heumann, Nima Honarmand, Patrick Simmons, Hyojin Sung, and Mohsen Vakilian were my “users.” They learned how to program in DPJ, and they wrote, tuned, and measured the performance of many DPJ programs. Their efforts contributed greatly to the evalu- ation of DPJ as a language. Patrick and Stephen also helped implement the compiler code generation. Finally, Mohsen caught and forced me to fix many tricky bugs in the compiler. • Jeff Overbey got us on the right path using javac for the DPJ compiler. He also helped get the compiler implementation started. • Maurice Herlihy suggested that we use the Deuce STM for the DPJ transactional runtime. I am also grateful to the following institutions for directly or indirectly supporting the work: • Microsoft Corporation and Intel Corporation directly funded my Research Assistantship (RA), under the auspices of the Universal Parallel Computing Research Center (UPCRC) at the University of Illinois. • NSF also directly funded my RA. • My internship at Cray Inc. supported me for a summer and helped me learn a lot about STM, parallel computing, and language design. • Motorola supported me as an RA for several semesters and as a summer intern. While the work supported by Motorola does not directly appear in this thesis, it helped me learn a lot about parallel computing (specifically vector computing). Finally, thanks to Sonia, my family, and my friends for their caring and support. v Table of Contents ListofFigures ...................................... ........ ix ListofTables....................................... ........ xi Chapter1 Introduction ............................... ......... 1 1.1 The Need for Determinism by Default . ........... 2 1.2 Technical Challenges . ......... 6 1.3 Deterministic Parallel Java . ............ 9 1.4 Thesis Contributions and Outline . ............. 10 Chapter 2 A Research Agenda for Determinism by Default . ............... 15 2.1 Guaranteeing Determinism . .......... 16 2.1.1 Patterns of Determinism . ....... 16 2.1.2 Approaches for Checking Effects . ......... 18 2.1.3 EffectSystems ................................. 19 2.2 Encapsulating Complex Behaviors . ............ 20 2.2.1 LocalNondeterminism . ...... 21 2.2.2 Unsoundness ..................................