THEDESIGN,IMPLEMENTATION, ANDEVALUATIONOFAGRADUAL TYPESYSTEMFORDYNAMIC CLASSCOMPOSITION asumu takikawa Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy College of Computer and Information Science Northeastern University Boston, Massachusetts April 2016 Asumu Takikawa: The Design, Implementation, and Evaluation of a Gradual Type System for Dynamic Class Composition, Doctor of Philosophy, Northeastern University, Boston, Massachusetts © April 2016 ABSTRACTDynamically-typed programming languages are popular and widely- used, but their lack of static type guidance may hamper program maintenance. As a result, language designers have sought to augment dynamically-typed languages with static type-checking for several decades. Gradual typing has emerged at the forefront of these efforts, allowing programmers to add type annotations to existing untyped programs in a piecemeal fashion. These annotations are checked stat- ically and translated to dynamic checks at the boundaries between typed and untyped code to ensure sound interoperation. Despite the abundance of publications on gradual typing, several roadblocks still remain that prevent its adoption in practice. First, gradual type systems lack the expressivity to support the flexible object-oriented patterns, especially mixins expressed with first-class classes, that are used in dynamically-typed languages. Second, the dynamic checks that are installed for sound interoperation often slow down the program. Worse yet, no systematic evaluation method ex- ists for these gradual typing overheads. I present an expressive design of a gradual type system for object- oriented programming and introduce an effective method for the sys- tematic evaluation of its performance characteristics. v 謝辞 It has been a long and eventful six-year journey towards a PhD, and I ACKNOWLEDGMENTS/am deeply indebted to everyone who helped me along the way. Thank you to Kris De Volder and Gregor Kiczales at UBC for first exposing me to Racket (PLT Scheme at the time), and thus setting me on the path to Northeastern University. Thank you very much to Matthias Felleisen, who has been an ex- cellent advisor through and through. Even before I formally joined the program, Matthias had already started advising me1. Under his tutelage, I have grown substantially as a writer, programmer, and re- searcher. I cannot overstate my gratitude for his mentorship. I wish to thank Kathleen Fisher, Sam Tobin-Hochstadt, Jan Vitek, and Mitch Wand for serving on my thesis committee. Their advice has certainly improved this dissertation. In particular, I owe Sam a great deal for his help in learning the design and implementation of Typed Racket. Thanks to several grants from the National Science Foundation for funding my PhD education. This document would not have been possible without the help of my many wonderful co-authors. Many thanks to Earl Dean, Chris- tos Dimoulas, Dan Feltey, Robby Findler, Matthew Flatt, Ben Green- man, Max New, and Stevie Strickland. I also wish to thank my past and present partners in crime in PLT: Claire Alvis, Leif Andersen, Eli Barzilay, Sam Caldwell, Stephen Chang, Ryan Culpepper, Eric Dobson, Carl Eastlund, Spencer Florence, Tony Garnock-Jones, Phúc Nguyên,˜ Vincent St-Amour, and David Van Horn. I was aided by the members of the Programming Research Lab who have been great friends, colleagues, and mentors over the years. My heartfelt thanks to Amal Ahmed, William Bowman, Dan Brown, Ben Chung, Andrew Cobb, Jose Falcon, Oli Flückiger, Dee Glaze, Ben Lerner, Brian LaChance, Paley Li, Alex Marquez, Phillip Mates, Jamie Perconti, Hari Prashanth, Joe Sackett, Jonathan Schuster, Erik Silkensen, Gabriel Scherer, Justin Slepak, Paul Stansifer, Jesse Tov, Aaron Turon, and Dimitris Vardoulakis. I am eternally grateful to my partner Katrina Yupangco, who has kept me grounded and has inspired me to grow as a person. 最後に、本論文を書き終えるまで長年支えてきてくださった両親と兄 弟に深く感謝申し上げます。本当にありがとうございました。 ２０１６年４月 瀧川明日夢 1 Though I was not quite able to read the ninety papers he requested. vii CONTENTSi 1 1 3 1.practical1 A Brief gradual History of typing the Relationship Between Typed and introductionUntyped Languages 4 1.2 A Short History of Gradual Typing 6 1.3 Thesis 6 1.4 Approach 8 1.5 The Structure of this Thesis 8 ii 11 2 13 2.a1 designProgramming & model with for First-Class first-class Classes classes13 design2.1. by1 exampleA First Design for Typed First-Class Classes 15 2.1.2 Protecting Typed Code from Untyped Code 18 2.1.3 Same-component restrictions for method invo- cation 23 3 27 3.1 Formalizing Typed First-Class Classes 27 the model3.1.1 Syntax 27 3.1.2 Type System 33 3.1.3 Operational Semantics 36 3.1.4 An Example Reduction Sequence 39 3.2 Type Soundness for Mixed Programs 45 3.2.1 Type soundness 45 3.2.2 Complete monitoring 46 3.2.3 The Blame Theorem 50 4 53 4.1 By Principle 53 4scaling.2 Examples the design from Case Studies 56 4.3 Implementation challenges 62 4.3.1 Macrology 62 4.3.2 Types to contracts 64 4.4 Limitations 65 iii 69 5 73 5.evaluating1 Cases gradual73 typing 5effectiveness.2 The Process evaluation74 5.3 Quantitative Results 74 5.4 Qualitative Results 76 5.5 Problems and Fixes 77 ix 5.6 Discussion 78 6 79 6.1 Methodology 79 6formative.2 Results performance and Preliminary evaluation Interpretation 80 6.3 Problems and Fixes 83 6.3.1 A Soundness Bug 85 6.4 Threats to Validity 86 7 87 7.1 Evaluation Method 87 summative7.1.1 performanceBenchmarking Software evaluation Evolution 87 7.2 The Benchmark Programs 90 7.2.1 Overview 90 7.2.2 Adaptor Modules 94 7.2.3 Program Descriptions 95 7.3 Evaluating Typed Racket 97 7.3.1 Suffixtree in Depth 97 7.3.2 Reading the Figures 100 7.3.3 Interpretation 103 7.4 Quo Vadis Sound Gradual Typing? 105 7.4.1 Threats to Validity 106 7.4.2 What are the Bottlenecks? 107 7.5 Long Live Sound Gradual Typing 110 iv 111 8 113 8.related1 Other work Approaches to Interoperation 113 8related.2 Type work Systems for Object-Oriented Programming 114 8.3 Sealing and Polymorphic Contracts 115 8.4 Implementations & Evaluations of Gradual Typing 115 8.4.1 Gradualtalk 115 8.4.2 Reticulated Python 116 8.4.3 Thorn and StrongScript 117 8.4.4 Safe TypeScript and StrongScript 118 8.4.5 Evaluations of Optional Typing 119 v 121 9 123 9.final1 Future remarks Directions 124 conclusion9.1.1 Typed Racket with Object-Oriented Programming 124 9.1.2 Performance of Gradual Type Systems 125 vi 127 a 129 Bibliographyappendix 153 appendix x xi Part I PRACTICALGRADUALTYPING As scripts grow into full-fledged applications, programmers should want to port portions of their programs from scripting languages to languages with sound and rich type systems. – Tobin-Hochstadt and Felleisen (2006) 1 ProgrammingINTRODUCTION language designers have created dynamically-typed languages for over five decades. For the past twenty years, work- ing programmers have flocked to dynamically-typed programming languages such as Ruby, Python, and JavaScript. In response to this The reasons for the trend, language designers have pursued the incremental augmenta- popularity of tion of such languages with static type annotations. The unifying dynamically-typed languages are out of goal of this effort is to mitigate the drawbacks of dynamic types—the scope for this increased cost of documentation, maintenance, and refactoring pro- dissertation. cesses due to the lack of static types—without giving up the allure of dynamic typing. These efforts have culminated in two schools of thought: gradual type systems and optional type systems. Implementations of both kinds allow a programmer to add type annotations to an existing untyped code base. After adding such annotations, the resulting code base may contain both typed and untyped code. Where the two schools of thought differ is in whether and how these annotations are enforced and the guarantees that they provide. A gradual type system allows sound interaction between the typed and untyped portions of the program. At the interfaces between these typed and untyped fragments, the programmer writes down type an- notations. In order to ensure sound interoperation, the language must make sure that even untyped fragments uphold the type interfaces. To this end, gradual type systems translate the types at the interfaces into dynamic checks. There is an abundance of theoretical work in gradual type sys- tems (Ina and Igarashi 2011; Siek and Taha 2006, 2007; Takikawa et al. 2012; Tobin-Hochstadt and Felleisen 2006; Wolff et al. 2011), and several burgeoning research implementations have appeared: Typed Racket (Tobin-Hochstadt and Felleisen 2008), Gradualtalk (Allende et al. 2013), Reticulated Python (Vitousek et al. 2014), Safe TypeScript (Ras- togi et al. 2015), and most recently StrongScript (Richards et al. 2015). Meanwhile, an optional type system (Bracha 2004) does not guar- antee sound interoperation. The type system catches type errors in typed portions, but such errors in untyped portions are not checked either statically or dynamically. Instead, the programmer depends on the runtime system of the language to catch all remaining errors. This style of interoperation has been adopted by several industry projects such as Strongtalk (Bracha and Griswold 1993), Hack for PHP,1 MyPy 1 See hacklang.org and Verlaguet, Commercial Users of Functional Programming, Boston, MA 2013. 3 for Python,2 TypeScript,3 and Flow for JavaScript4. Additionally, sev- eral optional type systems have been presented in the academic lit- erature, such as optional typing for Clojure (Bonnaire-Sergeant 2012; Bonnaire-Sergeant et al. 2016) and for Lua (Maidl et al. 2014). This dissertation focuses on sound gradual typing instead of op- tional typing. The rationale for this focus is that as an academic work, this dissertation has the opportunity to tackle the more principled version of the problem of adding type annotations to an untyped lan- guage.