Gabriël Ditmar Primo Konat was born in The Language-Parametric Methods for Developing Interactive Programming Systems Language-Parametric Methods for Developing Interactive Programming Hague, the Netherlands. In 2009, he received his BSc in Computer Science from the Institute of Ap- Invitation plied Sciences in Rijswijk. In 2012, he received his MSc in Computer Science from Delft University of Technology (TUDelft). From 2012 to 2018, he was Language-Parametric a Ph.D. student with the Programming Languages Methods for Developing group at TUDelft, under supervision of Eelco Viss- Interactive Programming er and Sebastian Erdweg. His work focuses on lan- Systems guage workbenches and incremental build systems. Gabriël Konat [email protected] You are cordially invited to the public defense of my dissertation on Monday, November 18th, 2019 at 3pm. At 2:30pm, I will give a brief presentation summarizing my dissertation. The defense will take place in the Senaatszaal of the Delft University of Technology Auditorium, Mekelweg 5, 2628 CC Delft, the Netherlands Afterwards, there will be a Gabriël Konat Language-Parametric Methods for reception. Developing Interactive Programming Systems Gabriël Konat Propositions accompanying the dissertation Language-Parametric Methods for Developing Interactive Programming Systems by Gabriël Ditmar Primo Konat 1. Language-parametric methods for developing interactive programming sys- tems are feasible and useful. (This dissertation) 2. Compilers of general-purpose languages must be bootstrapped with fixpoint bootstrapping. (This dissertation) 3. Manually implementing an incremental system must be avoided. (This dissertation) 4. Like chemists need lab assistants, computer scientists need software engineers to support them in research, teaching, and application in industry. 5. Programming languages that evolve via public request for comments (RFCs) attract a diverse range of people, and are therefore of higher quality. 6. Critical case studies are a valuable tool for providing evidence in research. 7. Developing an interactive video game is the most effective way to learn a new programming language. 8. The publication process of conferences with a yearly deadline and unidirec- tional feedback is not conducive to innovative and high-quality publications. 9. Rewriting a C or C++ program in Rust always increases code quality. These propositions are regarded as opposable and defendable, and have been approved as such by the promotors prof.dr. E. Visser and prof.dr. S.T. Erdweg. Language-Parametric Methods for Developing Interactive Programming Systems DISSERTATION for the purpose of obtaining the degree of doctor at Delft University of Technology by the authority of the Rector Magnificus Prof.dr.ir. T.H.J.J. van der Hagen; Chair of the Board for Doctorates to be defended publicly on Monday 18 November 2019 at 15:00 o’clock by Gabriël Ditmar Primo KONAT MSc Computer Science, Delft University of Technology, the Netherlands born in The Hague, the Netherlands This dissertation has been approved by the promotors. Composition of the doctoral committee: Rector Magnificus, chairperson Prof.dr. E. Visser Delft University of Technology, promotor Prof.dr. S.T. Erdweg Johannes Gutenberg University Mainz, promotor Independent members: Prof.dr.ir. D.H.J. Epema Delft University of Technology Prof.dr. M. Flatt University of Utah Prof.dr. T. van der Storm University of Groningen / CWI Dr. A. Mokhov Newcastle University Dr. E. Dolstra Tweag I/O Prof.dr. K.G. Langendoen Delft University of Technology, reserve member The work in this dissertation has been carried out at the Delft University of Technology, and was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Specific Languages). Copyright © 2019 Gabriël Ditmar Primo Konat Cover: Museum of Pop Culture - Photo © 2014 Gabriël Ditmar Primo Konat Printed and bound by: Gildeprint - https://www.gildeprint.nl/ ISBN: 978-94-6366-210-9 Contents Samenvatting ix Summary xi Preface xiii 1 Introduction 1 1.1 Programming Systems . 2 1.2 Interactive Programming Systems . 3 1.3 Developing Interactive Programming Systems . 4 1.4 Language-Parametric Methods . 5 1.5 Contributions . 7 1.5.1 NaBL: Declarative Name Binding and Scope Rules . 7 1.5.2 A Task Engine for Incremental Name and Type Analysis 8 1.5.3 Bootstrapping Meta-DSLs in Language Workbenches . 8 1.5.4 PIE: A Framework for Interactive Software Development Pipelines . 9 1.5.5 Scalable Incremental Building with Dynamic Task De- pendencies . 10 1.6 Research Methodology . 11 1.7 Structure . 12 2 NaBL: A Meta-DSL for Declarative Name Binding and Scope Rules 15 2.1 Introduction . 15 2.2 Declarative Name Binding and Scope Rules . 17 2.2.1 Definitions and References . 17 2.2.2 Namespaces . 19 2.2.3 Scopes . 20 2.2.4 Namespaces as Language Concepts . 21 2.2.5 Imports . 21 2.2.6 Types . 22 2.3 Name Binding Patterns . 23 2.3.1 Unscoped Definition Sites . 23 2.3.2 Definition Sites inside their Scopes . 24 2.3.3 Definition Sites outside their Scopes . 25 2.3.4 Contextual Use Sites . 25 2.4 Editor Services . 26 2.4.1 Reference Resolving . 27 2.4.2 Constraint Checking . 27 2.4.3 Code Completion . 28 2.5 Implementation . 28 iii 2.5.1 Persistence of Name Bindings . 28 2.5.2 Resolving Names . 29 2.6 Integration into Spoofax . 30 2.6.1 Index API . 30 2.6.2 Reference resolution . 30 2.6.3 Constraint checking . 30 2.6.4 Code completion . 31 2.7 Evaluation and Discussion . 31 2.7.1 Limitations . 32 2.7.2 Coverage . 32 2.8 Related work . 32 2.8.1 Symbol Tables . 33 2.8.2 Attribute Grammars . 33 2.8.3 Visibility Predicates . 34 2.8.4 Dynamic Rewrite Rules . 34 2.8.5 Textual Language Workbenches . 34 3 A Language Independent Task Engine for Incremental Name and Type Analysis 37 3.1 Introduction . 37 3.2 Name and Type Analysis . 38 3.2.1 Name Analysis . 38 3.2.2 Type Analysis . 39 3.2.3 Incremental Analysis . 40 3.3 Semantic Index . 41 3.3.1 URIs . 41 3.3.2 Index Entries . 41 3.3.3 Initial Collection . 42 3.3.4 Incremental Collection . 43 3.4 Deferred Analysis Tasks . 44 3.4.1 Instructions . 45 3.4.2 Combinators . 47 3.4.3 Initial Evaluation . 48 3.4.4 Incremental Evaluation . 48 3.5 Implementation . 50 3.6 Evaluation . 50 3.6.1 Research method . 51 3.6.2 Results and interpretation . 53 3.6.3 Threats to validity . 54 3.7 Related Work . 54 3.7.1 IDEs and Language Workbenches . 55 3.7.2 Attribute Grammars . 55 3.7.3 Reference Attribute Grammars . 55 3.7.4 Other Approaches . 56 3.8 Conclusion . 56 iv 4 Reflection: Incremental Name and Type Analysis, Bootstrapping, and Spoofax Core 59 5 Bootstrapping Domain-Specific Meta-Languages in Language Work- benches 63 5.1 Introduction . 63 5.2 Problem Analysis . 65 5.2.1 Bootstrapping Example . 65 5.2.2 Requirements . 66 5.3 Sound Bootstrapping . 69 5.3.1 Language Definitions and Products . 69 5.3.2 Compilation . 71 5.3.3 Fixpoint Bootstrapping . 72 5.4 Interactive Bootstrapping . 72 5.5 Bootstrapping Breaking Changes . 73 5.6 Evaluation . 74 5.6.1 Implementation . 74 5.6.2 Meta-languages . 74 5.6.3 Bootstrapping Changes . 76 5.7 Related Work . 77 5.7.1 Bootstrapped General-Purpose Languages . 77 5.7.2 Bootstrapping . 78 5.7.3 Language Workbenches . 78 5.7.4 Staged Metaprogramming . 79 5.8 Conclusion . 80 6 Reflection: Language Workbench Pipelines 81 7 PIE: A DSL, API, and Runtime for Interactive Software Development Pipelines 83 7.1 Introduction . 84 7.2 Problem Analysis . 85 7.2.1 Requirements . 86 7.2.2 State of the Art . 86 7.2.3 Open Problems . 88 7.3 PIE by Example . 90 7.4 PIE API and Runtime . 93 7.4.1 Application Program Interface (API) . 93 7.4.2 Runtime . 95 7.4.3 Reusing the Pluto Runtime . 95 7.5 PIE Language . 96 7.5.1 Syntax . 96 7.5.2 Static Semantics . 96 7.5.3 Compilation . 97 7.6 Case Study: Spoofax Language Workbench . 98 7.6.1 Pipeline Re-Implementation . 99 7.6.2 Analysis . 101 CONTENTS v 7.7 Case Study: Live Performance Testing . 102 7.7.1 Pipeline Re-Implementation . ..