Dynamic Extension of Typed Functional Languages
Total Page：16
File Type：pdf, Size：1020Kb
Dynamic Extension of Typed Functional Languages Don Stewart PhD Dissertation School of Computer Science and Engineering University of New South Wales 2010 Supervisor: Assoc. Prof. Manuel M. T. Chakravarty Co-supervisor: Dr. Gabriele Keller Abstract We present a solution to the problem of dynamic extension in statically typed functional languages with type erasure. The presented solution re- tains the benefits of static checking, including type safety, aggressive op- timizations, and native code compilation of components, while allowing extensibility of programs at runtime. Our approach is based on a framework for dynamic extension in a stat- ically typed setting, combining dynamic linking, runtime type checking, first class modules and code hot swapping. We show that this framework is sufficient to allow a broad class of dynamic extension capabilities in any statically typed functional language with type erasure semantics. Uniquely, we employ the full compile-time type system to perform run- time type checking of dynamic components, and emphasize the use of na- tive code extension to ensure that the performance benefits of static typing are retained in a dynamic environment. We also develop the concept of fully dynamic software architectures, where the static core is minimal and all code is hot swappable. Benefits of the approach include hot swappable code and sophisticated application extension via embedded domain specific languages. We instantiate the concepts of the framework via a full implementation in the Haskell programming language: providing rich mechanisms for dy- namic linking, loading, hot swapping, and runtime type checking in Haskell for the first time. We demonstrate the feasibility of this architecture through a number of novel applications: an extensible text editor; a plugin-based network chat bot; a simulator for polymer chemistry; and xmonad, an ex- tensible window manager. In doing so, we demonstrate that static typing is no barrier to dynamic extension. Acknowledgments This thesis describes work carried out between 2004 and 2008 at the School of Computer Science, University of New South Wales in Sydney. I am deeply indebted to my supervisor, Manuel Chakravarty, who drew me into the func- tional programming world, and encouraged the exploration of its uncharted corners. I am grateful to my co-supervisor, Gabriele Keller, for ongoing feedback and insight. Three groups of people influenced my work and thinking over this time, and I would like to thank them directly. Firstly, the members of the Programming Languages and Systems (PLS) group at the University of New South Wales – Roman Leshchinskiy, André Pang, Sean Seefried, Stefan Wehr, Simon Winwood, Mark Wotton, and Patryk Zadarnowski – who, along with my supervisors, built an energetic culture of innovation and exploration in functional programming that shaped the direction of this work from the beginning. Secondly, my colleagues at Galois, Inc. in Portland, for giving me the resources and mo- tivation to complete my research, and for the opportunity to apply functional programming techniques to solve difficult problems, all in an environment of talent and fun. In particular, I wish to thank John Launchbury, for his mentorship, Adam Wick, for support and motivation, and Jason Dagit, for constructive feedback. Thirdly, the Haskell community provided willing feedback and suggestions for many of the projects described in this thesis, motivating me to pursue ideas I might have passed by. Shae Erisson, in particular, encouraged and inspired this work in its early days, helping to ensure the success of several of the projects. I am glad to be a citizen of such a community of artists and hackers. Finally, this work would not have been possible without Suzie Allen, and her patience, encouragement, and love. Portions of the text of this thesis were originally published in [122, 146, 147, 148], and due acknowledgment is due to André Pang, Sean Seefried, Manuel Chakravarty, Gabriele Keller, Hugh Chaffey-Millar and Christopher Barner-Kowollik, for their contributions. Don Stewart. Portland Oregon, June 2010. Contents 1 Introduction1 1.1 Motivation..................................... 1 1.1.1 Safety via types .............................. 3 1.1.2 Flexibility via runtime code loading .................. 4 1.1.3 The middle way .............................. 5 1.2 Approach ...................................... 6 1.2.1 Dynamic extension ............................ 6 1.2.2 Static typing ................................ 9 1.2.3 A framework approach ......................... 10 1.3 Contribution .................................... 11 1.4 Structure ...................................... 12 2 A framework for dynamic extension 13 2.1 Components of the framework ......................... 13 2.1.1 Dynamic linking ............................. 14 2.1.2 Runtime type checking ......................... 16 2.1.3 First class modules ............................ 18 2.1.4 Runtime code generation ........................ 18 2.1.5 Module hot swapping .......................... 20 2.1.6 Embedded extension languages .................... 20 2.2 Implementation .................................. 22 2.3 Dynamic linking ................................. 24 viii CONTENTS 2.3.1 Defining a plugin interface ....................... 25 2.3.2 Implementing an interface ....................... 26 2.3.3 Using a plugin .............................. 27 2.3.4 Loading plugins from other languages . 28 2.4 Runtime type checking .............................. 29 2.4.1 Dynamically checked plugins ...................... 30 2.4.2 Safety and flexibility ........................... 31 2.4.3 Improving runtime type checking ................... 33 2.5 First class modules ................................ 36 2.5.1 A type class for first class modules . 36 2.5.2 Existential types ............................. 37 2.6 Runtime compilation ............................... 38 2.6.1 Invoking the compiler .......................... 39 2.6.2 Generating new code at runtime .................... 39 2.6.3 Cross-language extension ........................ 41 2.7 Hot swapping ................................... 42 2.7.1 Dynamic architectures .......................... 44 2.7.2 State preservation ............................ 45 2.7.3 State preservation as types change ................... 46 2.7.4 Persistent state .............................. 47 2.8 Embedded languages ............................... 47 2.9 Summary of the framework ........................... 49 3 Dynamic linking 51 3.1 Overview ...................................... 51 3.2 Dynamic linking in Haskell ........................... 53 3.2.1 Runtime loading ............................. 53 3.2.2 Basic plugin loading ........................... 54 3.2.3 Dependency chasing ........................... 56 3.3 Typing dynamic linking ............................. 59 CONTENTS ix 3.3.1 Limitations ................................ 61 3.4 Polymorphic dynamics .............................. 62 3.5 Comparing approaches .............................. 65 3.5.1 Performance of dynamic type checking . 65 3.5.2 Type safety and source code plugins . 66 3.6 Applications .................................... 67 3.6.1 Lambdabot and Riot ........................... 67 3.6.2 Haskell Server Pages ........................... 68 3.6.3 Specializing simulators for computational chemistry . 69 3.7 Discussion ..................................... 71 3.7.1 A standalone type checker ....................... 71 3.7.2 Loading packages and archives ..................... 72 3.7.3 Loading C objects into Haskell ..................... 72 3.7.4 Loading bytecode objects ........................ 73 3.7.5 Executable size .............................. 73 3.8 Related work .................................... 74 3.8.1 Type safe linking ............................. 74 3.8.2 Dynamic typing .............................. 75 3.8.3 Clean .................................... 77 3.8.4 ML ..................................... 77 3.8.5 Java and .NET ............................... 78 4 Runtime compilation 81 4.1 A compilation manager ............................. 81 4.1.1 Invoking the compiler .......................... 82 4.1.2 Manipulating abstract syntax ...................... 84 4.2 An eval for Haskell ................................ 87 4.2.1 Evaluating Haskell from other languages . 89 4.3 Runtime meta-programming .......................... 90 4.3.1 The heterogeneous symbol table problem . 92 x CONTENTS 4.3.2 Type safe printf .............................. 95 4.4 Applications .................................... 96 4.4.1 A Haskell interactive environment ................... 96 4.4.2 Source plugins for compiled programs . 98 4.4.3 Type-based sandboxing of untrusted code . 98 4.4.4 Dynamic server pages, revisited . 100 4.4.5 Optimizing embedded DSLs . 100 4.5 Related work ....................................101 4.5.1 Template Haskell and staged type inference . 102 4.5.2 Multi-stage programming . 103 5 Hot swapping 105 5.1 Overview ......................................105 5.2 A dynamic architecture .............................107 5.2.1 Dynamic bootstrapping . 108 5.3 Hot swapping ...................................110 5.3.1 Dynamic reconfiguration . 111 5.3.2 State preservation ............................114 5.3.3 Reloading the application . 115 5.3.4 Upgrading state types . 117 5.3.5 Persistent state ..............................118 5.4 Performance ....................................118 5.4.1 Static applications ............................121