Language-Parametric Methods for Developing

Gabriël Konat 3pm. Systems reception. Mekelweg 5, 2628 CC Delft, my dissertation. the Netherlands [email protected] The defense will take Language-Parametric Language-Parametric Technology Auditorium, Auditorium, Technology place in the Senaatszaal dissertation on Monday, dissertation on Monday, of the Delft University of November 18th, 2019 at the public defense of my Invitation presentation summarizing presentation Afterwards, there will be a there Afterwards, You are cordially invited to cordially are You Methods for Developing Methods Interactive Programming Programming Interactive At 2:30pm, I will give a brief Gabriël Konat Language-Parametric Methods for Language-Parametric Methods for Systems Developing Interactive Programming

Language-Parametric Methods for Developing Interactive Programming Systems Gabriël Konat Gabriël Ditmar Primo Konat was born was Konat Primo Ditmar Gabriël The in his he received 2009, In Netherlands. the Hague, of Ap- Institute the from Science in Computer BSc his he received 2012, In in Rijswijk. Sciences plied of Delft University from Computer Science MSc in 2018, he was 2012 to (TUDelft). From Technology Languages the Programming student with a Ph.D. of Eelco Viss under supervision at TUDelft, - group His work focuses on lan- Erdweg. er and Sebastian build systems. and incremental guage workbenches Propositions

accompanying the dissertation

Language-Parametric Methods for Developing Interactive Programming Systems

Gabriël Ditmar Primo Konat

1. Language-parametric methods for developing interactive programming systems are feasible and useful. (This dissertation) 2. Compilers of general-purpose languages must be bootstrapped with ﬁxpoint bootstrapping. (This dissertation)

3. Manually implementing an incremental system must be avoided. (This dissertation) 4. Like chemists need lab assistants, computer scientists need software engineers to support them in research, teaching, and application in industry.

5. Programming languages that evolve via public request for comments (RFCs) attract a diverse range of people, and are therefore of higher quality. 6. Critical case studies are a valuable tool for providing evidence in research. 7. Developing an interactive video game is the most eﬀective way to learn a new programming language.

8. The publication process of conferences with a yearly deadline and unidirec- tional feedback is not conducive to innovative and high-quality publications. 9. Rewriting a C or C++ program in Rust always increases code quality.

These propositions are regarded as opposable and defendable, and have been approved as such by the promotors prof.dr. E. Visser and prof.dr. S.T. Erdweg. Language-Parametric Methods for Developing Interactive Programming Systems

DISSERTATION

for the purpose of obtaining the degree of doctor at Delft University of Technology by the authority of the Rector Magniﬁcus Prof.dr.ir. T.H.J.J. van der Hagen; Chair of the Board for Doctorates to be defended publicly on Monday 18 November 2019 at 15:00 o’clock by

Gabriël Ditmar Primo KONAT

MSc Computer Science, Delft University of Technology, the Netherlands born in The Hague, the Netherlands This dissertation has been approved by the promotors. Composition of the doctoral committee: Rector Magniﬁcus, chairperson Prof.dr. E. Visser Delft University of Technology, promotor Prof.dr. S.T. Erdweg Johannes Gutenberg University Mainz, promotor

Independent members: Prof.dr.ir. D.H.J. Epema Delft University of Technology Prof.dr. M. Flatt University of Utah Prof.dr. T. van der Storm University of Groningen / CWI Dr. A. Mokhov Newcastle University Dr. E. Dolstra Tweag I/O Prof.dr. K.G. Langendoen Delft University of Technology, reserve member

The work in this dissertation has been carried out at the Delft University of Technology, and was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Speciﬁc Languages).

Copyright © 2019 Gabriël Ditmar Primo Konat Cover: Museum of Pop Culture - Photo © 2014 Gabriël Ditmar Primo Konat Printed and bound by: Gildeprint - https://www.gildeprint.nl/ ISBN: 978-94-6366-210-9 Contents

Samenvatting ix

Summary xi

Preface xiii

1 Introduction 1 1.1 Programming Systems ...... 2 1.2 Interactive Programming Systems ...... 3 1.3 Developing Interactive Programming Systems ...... 4 1.4 Language-Parametric Methods ...... 5 1.5 Contributions ...... 7 1.5.1 NaBL: Declarative Name Binding and Scope Rules . . . 7 1.5.2 A Task Engine for Incremental Name and Type Analysis 8 1.5.3 Bootstrapping Meta-DSLs in Language Workbenches . . 8 1.5.4 PIE: A Framework for Interactive Software Development Pipelines ...... 9 1.5.5 Scalable Incremental Building with Dynamic Task De- pendencies ...... 10 1.6 Research Methodology ...... 11 1.7 Structure ...... 12

2 NaBL: A Meta-DSL for Declarative Name Binding and Scope Rules 15 2.1 Introduction ...... 15 2.2 Declarative Name Binding and Scope Rules ...... 17 2.2.1 Definitions and References ...... 17 2.2.2 Namespaces ...... 19 2.2.3 Scopes ...... 20 2.2.4 Namespaces as Language Concepts ...... 21 2.2.5 Imports ...... 21 2.2.6 Types ...... 22 2.3 Name Binding Patterns ...... 23 2.3.1 Unscoped Definition Sites ...... 23 2.3.2 Definition Sites inside their Scopes ...... 24 2.3.3 Definition Sites outside their Scopes ...... 25 2.3.4 Contextual Use Sites ...... 25 2.4 Editor Services ...... 26 2.4.1 Reference Resolving ...... 27 2.4.2 Constraint Checking ...... 27 2.4.3 Code Completion ...... 28 2.5 Implementation ...... 28

iii 2.5.1 Persistence of Name Bindings ...... 28 2.5.2 Resolving Names ...... 29 2.6 Integration into Spoofax ...... 30 2.6.1 Index API ...... 30 2.6.2 Reference resolution ...... 30 2.6.3 Constraint checking ...... 30 2.6.4 Code completion ...... 31 2.7 Evaluation and Discussion ...... 31 2.7.1 Limitations ...... 32 2.7.2 Coverage ...... 32 2.8 Related work ...... 32 2.8.1 Symbol Tables ...... 33 2.8.2 Attribute Grammars ...... 33 2.8.3 Visibility Predicates ...... 34 2.8.4 Dynamic Rewrite Rules ...... 34 2.8.5 Textual Language Workbenches ...... 34

3 A Language Independent Task Engine for Incremental Name and Type Analysis 37 3.1 Introduction ...... 37 3.2 Name and Type Analysis ...... 38 3.2.1 Name Analysis ...... 38 3.2.2 Type Analysis ...... 39 3.2.3 Incremental Analysis ...... 40 3.3 Semantic Index ...... 41 3.3.1 URIs ...... 41 3.3.2 Index Entries ...... 41 3.3.3 Initial Collection ...... 42 3.3.4 Incremental Collection ...... 43 3.4 Deferred Analysis Tasks ...... 44 3.4.1 Instructions ...... 45 3.4.2 Combinators ...... 47 3.4.3 Initial Evaluation ...... 48 3.4.4 Incremental Evaluation ...... 48 3.5 Implementation ...... 50 3.6 Evaluation ...... 50 3.6.1 Research method ...... 51 3.6.2 Results and interpretation ...... 53 3.6.3 Threats to validity ...... 54 3.7 Related Work ...... 54 3.7.1 IDEs and Language Workbenches ...... 55 3.7.2 Attribute Grammars ...... 55 3.7.3 Reference Attribute Grammars ...... 55 3.7.4 Other Approaches ...... 56 3.8 Conclusion ...... 56 iv 4 Reﬂection: Incremental Name and Type Analysis, Bootstrapping, and Spoofax Core 59

5 Bootstrapping Domain-Speciﬁc Meta-Languages in Language Work- benches 63 5.1 Introduction ...... 63 5.2 Problem Analysis ...... 65 5.2.1 Bootstrapping Example ...... 65 5.2.2 Requirements ...... 66 5.3 Sound Bootstrapping ...... 69 5.3.1 Language Deﬁnitions and Products ...... 69 5.3.2 Compilation ...... 71 5.3.3 Fixpoint Bootstrapping ...... 72 5.4 Interactive Bootstrapping ...... 72 5.5 Bootstrapping Breaking Changes ...... 73 5.6 Evaluation ...... 74 5.6.1 Implementation ...... 74 5.6.2 Meta-languages ...... 74 5.6.3 Bootstrapping Changes ...... 76 5.7 Related Work ...... 77 5.7.1 Bootstrapped General-Purpose Languages ...... 77 5.7.2 Bootstrapping ...... 78 5.7.3 Language Workbenches ...... 78 5.7.4 Staged Metaprogramming ...... 79 5.8 Conclusion ...... 80

6 Reﬂection: Language Workbench Pipelines 81

7 PIE: A DSL, API, and Runtime for Interactive Software Development Pipelines 83 7.1 Introduction ...... 84 7.2 Problem Analysis ...... 85 7.2.1 Requirements ...... 86 7.2.2 State of the Art ...... 86 7.2.3 Open Problems ...... 88 7.3 PIE by Example ...... 90 7.4 PIE API and Runtime ...... 93 7.4.1 Application Program Interface (API) ...... 93 7.4.2 Runtime ...... 95 7.4.3 Reusing the Pluto Runtime ...... 95 7.5 PIE Language ...... 96 7.5.1 Syntax ...... 96 7.5.2 Static Semantics ...... 96 7.5.3 Compilation ...... 97 7.6 Case Study: Spoofax Language Workbench ...... 98 7.6.1 Pipeline Re-Implementation ...... 99 7.6.2 Analysis ...... 101

CONTENTS v 7.7 Case Study: Live Performance Testing ...... 102 7.7.1 Pipeline Re-Implementation ...... 103 7.7.2 Analysis ...... 103 7.8 Related Work ...... 104 7.8.1 Partial Domain-Speciﬁc Build Abstractions ...... 104 7.8.2 Software Development Pipelines as a Library ...... 105 7.8.3 General-Purpose Languages ...... 106 7.8.4 Reactive Programming ...... 107 7.8.5 Workﬂow Languages ...... 107 7.9 Future Work ...... 107 7.9.1 First-Class Functions and Closures ...... 107 7.9.2 Live Pipelines ...... 108 7.10 Conclusion ...... 108

8 Scalable Incremental Building with Dynamic Task Dependencies 109 8.1 Introduction ...... 109 8.2 Background and Problem Statement ...... 111 8.3 Key Idea and Challenges ...... 114 8.3.1 Bottom-Up Traversal ...... 114 8.3.2 Top-Down Initialization ...... 115 8.3.3 Early Cut-Off ...... 115 8.3.4 Order of Recomputation ...... 115 8.3.5 Dynamic Dependencies ...... 116 8.3.6 Dependency Graph Validation ...... 116 8.4 Change-Driven Incremental Building ...... 118 8.4.1 Bottom-Up Building ...... 118 8.4.2 Execution, Requirement, and Validation ...... 119 8.4.3 Properties ...... 120 8.5 Implementation ...... 120 8.6 Evaluation ...... 121 8.6.1 Experimental Setup ...... 121 8.6.2 Results and Interpretation ...... 123 8.6.3 Threats to Validity ...... 126 8.7 Related Work ...... 126 8.8 Conclusion ...... 127

9 Conclusion 129 9.1 Interactive Programming Systems ...... 129 9.2 Language-Parametric Methods ...... 130 9.2.1 Incremental Name and Type Analysis ...... 130 9.2.2 Bootstrapping meta-DSLs of Language Workbenches . . 131 9.2.3 Pipelining of Interactive Programming Systems . . . . . 131 9.3 Future Work ...... 133 9.3.1 Incremental Name and Type Analysis ...... 133 9.3.2 Bootstrapping of Meta-DSLs ...... 134 9.3.3 Pipelining of Interactive Programming Systems . . . . . 134 vi Bibliography 137

Curriculum Vitae 153

List of Publications 155

CONTENTS vii viii Samenvatting

Op alle computers wordt software uitgevoerd, zoals besturingssystemen, web- browsers, en videospellen, die door miljarden mensen over de wereld worden gebruikt. Daarom is het belangrijk om software van hoge kwaliteit te bouwen, wat alleen mogelijk is met interactive programmeringssysteemen die program- meurs betrekken in de uitwisseling van correcte en responsieve feedback. Ge- lukkig maken geïntegreerde software-ontwikkelingsomgevingen dit mogelijk voor vele generieke programmeertalen, door middel van broncodebewerkers met hulpmiddelen zoals syntaxiskleuring en automatische aanvulling. Daarintegen zijn domeinspecifieke talen programmeertalen die gespecialiseerd zijn voor een specifiek probleemdomein, en het daarom mogelijk maken om betere software te schrijven door directe expressie van problemen en oplossingen in termen van het domein. Echter, omdat domeinspecifieke talen gespecialiseerd zijn voor een bepaald domein, en er veel probleemdomeinen zijn, moeten we veel nieuwe domeinspecifieke talen ontwikkelen, inclusief bijbehorende interactieve programmeringssysteemen! Het ontwikkelen van een ad-hoc interactief programmeringssysteem voor een domeinspecifieke taal is ondoenlijk, omdat dit een te grote ontwikkelingsinspanning nodig heeft. Daarom is het onze visie om taalparametrische methodes voor het ontwikkelen van interactieve programmeringssysteemen te gebruiken. Een taalparametrische methode neemt als invoer een beschrijving van een domeinspecifieke taal, en implementeert automatisch (delen van) een interactief programmeringssysteem, waardoor ontwikkelingsinspanning wordt vermin- derd, en domeinspecifieke taalontwikkeling doenlijk wordt. In dit proefschrift ontwikkelen we drie taalparametrische in de vijf kernhoofdstukken. We ontwikkelen een taalparametrische methode voor incrementele naam- en type-analyse, waarbij taalontwikkelaars de naam- en typeregels van hun domeinspecifieke taal specificeren in metatalen (talen die gespecialiseerd zijn in het domein van taalontwikkeling). Uit een dergelijke specificatie leiden we automatisch een incrementele naam- en typeanalyse af, inclusief bewerkings- hulpmiddelen zoals codeaanvulling en inline-foutmeldingen. We ontwikkelen een taalparametrische methode voor het interactief boot- strappen van de metataal compilers van taalwerkbanken. We beheren meerdere versies van metataal compilers, geven expliciet de afhankelijkheden tussen hen aan, en voeren fixpoint bootstrapping uit, waarbij we iteratief metataal compilers op henzelf toepassen om nieuwe versies af te leiden totdat er geen verandering plaatsvindt, of totdat een fout wordt gevonden. Deze bootstrap- pingbewerkingen kunnen worden gestart en teruggedraaid (wanneer een fout is gevonden) in het interactieve programmeringssysteem van de taalwerkbank. Ten slotte ontwikkelen we Pipelines for Interactive Environments (PIE), een parametrische methode voor het ontwikkelen van interactieve pijplijnen voor softwareontwikkeling, een superset van interactieve programmeeromgevingen.

ix Met PIE kunnen pijplijnontwikkelaars bondig pijplijnprogramma’s schrijven in termen van taken en afhankelijkheden tussen taken en bestanden, die PIE vervolgens incrementeel uitvoert. PIE schaalt af naar vele veranderingen met kleine impact, en schaalt op naar grote afhankelijkheidsgraﬁeken via een incrementeel veranderingsgedreven algoritme.

x Summary

All computers run software, such as operating systems, web browsers, and video games, which are used by billions of people around the world. Therefore, it is important to develop high-quality software, which is only possible through interactive programming systems that involve programmers in the exchange of correct and responsive feedback. Fortunately, for many general-purpose programming languages, integrated development environments provide interactive programming systems through code editors and editor services. On the other hand, Domain-Specific Languages (DSLs) are programming languages that are specialized towards a specific problem domain, enabling better software through direct expression of problems and solutions in terms of the domain. However, because DSLs are specialized to a specific domain, and there are many problem domains, we need to develop many new DSLs, including their interactive programming systems! Ad-hoc development of an interactive programming system for a DSL is infeasible, as developing one requires a huge development effort. Therefore, our vision is to create and improve language-parametric methods for developing interactive programming systems. A language-parametric method takes as input a description of a DSL, and automatically implements (parts of) an interactive programming system, reducing development effort, thereby making DSL development feasible. In this dissertation, we develop three language- parametric methods throughout the five core chapters. We develop a language-parametric method for incremental name and type analysis, in which language developers specify the name and type rules of their DSL in meta-languages (languages specialized towards the domain of language development). From such a specification, we automatically derive an incremental name and type analysis, including editor services such as code completion and inline error messages. We develop a language-parametric method for interactively bootstrapping the meta-language compilers of language workbenches. We version meta- language compilers, explicitly denote dependencies between them, and perform fixpoint bootstrapping, where we iteratively self-apply meta-language compilers to derive new versions until no change occurs, or until a defect is found. These bootstrapping operations can be started and rolled back (when defect) in the interactive programming system of the language workbench. Finally, we develop PIE, a parametric method for developing interactive software development pipelines, a superset of interactive programming environments. With PIE, pipeline developers can concisely write pipeline programs in terms of tasks and dependencies between tasks and files, which the PIE runtime then incrementally executes. PIE scales down to many low-impact changes and up to large dependency graphs through a change-driven incremental build algorithm.

xi xii Preface

My journey to this dissertation started when I was two years old. My grandpa, who knew a lot about computers because he was a computer technician, would regularly let me play on his computer when we visited. At first, he’d teach me to play simple point and click games, painting programs, and colouring book programs. Later on he’d teach me DOS commands, how to navigate in Windows, and – more importantly – how to play SimCity 2000. I of course did not fully understand SimCity at the time, but enjoyed it a lot nonetheless, because it would let me create and destroy cities, and hear ’bzzt’ a million times. I would usually run out of money pretty fast and have to take several bonds, which quickly lead to bankruptcy. Eventually, I did manage to complete a scenario which rewarded me the key of the city! My grandpa was also one of the first to have cable internet in the Netherlands, which had a fixed monthly cost in contrast to the by-minute cost of dialup internet, making it a lot more affordable, even though the 64Kb/s speed was horrible. The internet was really a magical experience back then, because it enabled easily chatting with people, finding news, information, games, cheats, jokes, or basically anything. Every day, you’d find something new and exiting. When visiting my grandparents, I would quickly jump behind the computer and go on the internet, usually playing free online games. One online game that stood out is Graal Online, an MMORPG that plays like Zelda: A Link to the Past, which is still alive as of writing this dissertation. While the online mode of Graal Online is a lot of fun, it also features an offline mode with a level editor, allowing you to create your own worlds. The level editor also includes a script editor with a Java-like scripting language, which was my first foray into programming. I was able to take the offline mode home by compressing it and splitting it across 12 floppy disks, allowing me to build worlds and program from my own computer, which I frequently did. Together with a friend, we designed and programmed our own world and submitted it to the creator of Graal Online, asking for it to be hosted as a ’playerworld’. Unfortunately, we never got a response. In 2005, I completed my secondary education at senior (Dutch: HAVO) level. Because of my grandpa’s enthusiasm for computers, teaching me how to use them, letting me regularly use his computer and internet, and learning to program with Graal Online, programming computers became my vocation. However, to be able to go to a university, I’d have to follow two more years of university preparatory education (VWO). Instead I opted to do the more practically-oriented higher professional education (hoger beroepsonder- wijs/HBO) at the Institute of Applied (Computer) Sciences in Rijswijk, as I did not want to spend two more years in secondary education, and instead wanted to immediately specialise in software development. I’m very glad to have chosen this option, as I got to meet many like-minded students, learn about

xiii how computers work, learn to properly program in several real programming languages, and learn how to develop software. In my last year, I was still looking for a project to graduate on. As usual, I was procrastinating to the very end and almost got into trouble for not having a graduation project. Luckily, during one of the robotics labs, Martijn Wisse from Delft University of Technology (TUDelft) visited and advertised one of his graduation projects: building a vision system for their autonomous humanoid football robot, which would compete in RoboCup 2009, Graz, Austria. Together with fellow student Jonathan Staats, we of course agreed to do this awesome graduation project. We’d work with the team in the basement of the TUDelft 3mE building (of course they put a robotics team in the basement) to build the eyes of their robot TUlip, supervised by Boris Lenseigne. The vision system consisted of two small Linux-based boards with cam- eras, connected to the robot’s main computer to provide stereo information. Jonathan worked on writing a Linux device driver for the camera, while I worked on the software that uses the camera data to detect field lines (Hough transform) and the ball. TUlip’s main computer would then integrate this data to determine its position and the position of the ball. TUlip competed in the Robocup and was able to defend the goal, walk short distances, kick the ball, and track the ball through our vision system. However, one of the challenges was to find the ball, dribble it, and then kick it to score, which turned out to be really hard for our big and heavy bipedal robot with a realistic humanoid gait. In any case, RoboCup was an awesome experience, with lots of great football matches with smaller and wheeled robots, lots of like- minded people, and a lot of fun with our team. In the end, despite not winning RoboCup, we were able to graduate and receive the title of engineer (ing. in Dutch). Besides graduating, this project gave me a very high appreciation of TUDelft and the people working there. The university is far more practical than I had imagined, and most students/staff are very kind, open-minded, and motivated, convincing me to study for a master’s degree at TUDelft. While studying for a master’s degree, I would regularly listen to podcasts on my bus trip from The Hague to Delft. One of the podcasts I was listing to at the time was the Software Engineering Radio (SE Radio), founded by Markus Voelter. Episode 118 was on parsing, with an interview of Eelco Visser, which surprised me because I recognised him as the lecturer of Delft’s Compiler Construction course. I found the episode quite interesting, as I really like programming, and building a parser for a (new?) programming language had always interested me, but I never quite understood it or had the time/motivation to dive into it. I thought it was also quite cool that a professor at my university was being interviewed in my favourite podcast. In the first semester of the second year, I did the compiler construction course. I would have done it in the first year, but everyone I talked to at the introductory week (even other teachers!) recommended me to not do it in the first year, because it was a really hard course that required a lot of time investment (a lot more than you’d get ECTS for). While this was definitely true, I would have liked and be motivated by the course so much that it wouldn’t xiv have been a problem at all. In any case, I followed the course, but was slightly discontented. Even though Eelco was the responsible teacher, one of his postdocs, Guido Wachsmuth, was actually teaching the course, so I’d never meet Eelco during the course. Note that this is not a jab at Guido at all, because he did a great job teaching compiler construction. He is very knowledgable, has great-looking slides with a lot of information, and is very supportive of his students. That is, the students that came to the lecture on time at least, as he would usually lock the door when the lecture started to prevent annoyance by late students. I got locked out of half a lecture once because of a flat tire on my bike, oh well. I particularly like Guido’s teaching style of explaining algorithms by showing a lot of examples and actually executing the algorithm step-by-step on the slides, which my brain really appreciates. If you’d want to see the algorithm pseudocode, you can just read the corresponding book or paper. Besides the lectures, there is also a lab in which you build a compiler and IDE from scratch for the MiniJava programming language. To achieve this, we use the Spoofax Language Workbench, which is basically a set of tools to develop programming languages and their interactive programming systems, which is where part of this dissertation’s title comes from. Spoofax was first developed by Karl Trygve Kalleberg as an Eclipse IDE for the Stratego language, but was later developed into a Language Workbench by Lennart Kats, who would sometimes visit the lab (in his cool leather jacket) to help us out. While Spoofax was a bit janky sometimes, as many research tools are, it did enable us to develop a compiler and editor for a programming language – starting without any compiler construction knowledge – in a single semester, how cool is that? This lab got me really enthusiastic about developing programming languages, but also about creating meta-tools (i.e., language-parametric methods) for developing interactive programming systems. In the follow-up course, Model Driven Software Development, which was actually taught by Eelco Visser, we used our knowledge from the compiler construction course to develop our own domain-specific language (DSL). Again together with Jonathan, we constructed a DSL for our 3D virtual world/game engine, Diversia, which we developed during our time at Rijswijk. It was an event-driven DSL that reacted to events in the game world, and compiled down to LUA scripting code which interacted with the game engine. During the course, there was a guest lecture by Markus Voelter, whose podcast sparked my interest for programming languages in the first place, reminding me of how small the world usually is. It was also quite cool to see Markus after hearing him interview so many people. The final programming languages related course was a seminar on metaprogramming, where we dived into program analysis with Datalog. After this course, I was interested in doing my master’s thesis with the programming languages group, and Eelco agreed to supervise me. Lennart and Karl got me up to speed with Spoofax development in their office. I sat next to Karl for a couple of days, which, now that I think of it, was probably quite annoying for him since I took up half of his desk and invaded his privacy. I moved onto a

Chapter 0. Preface xv separate table in their office after a while, which was still quite nice compared to the separate master student room (Dutch: het master hok). Later on, Guido became more heavily involved with my thesis, as the topic shifted to name analysis, and I ended up doing my master’s thesis on "Language-Parametric Incremental and Parallel Name Resolution". Before graduating, Eelco asked if I wanted to do a PhD. I knew that doing a PhD was an option, but never really gave it much thought, as I was planning to take a couple of months off and then go into industry. However, after a couple of days, I accepted his offer, and the work on this dissertation begun. Acknowledgements. Before continuing, I would like to (try to) thank everyone that made this work possible. First, I want to thank my supervisors. I am grateful to Eelco Visser for giving me the opportunity to do a PhD, and his supervision and kindness throughout it. Eelco is extremely good at understanding the bigger picture. When approached with a problem, he will effortlessly break it down into smaller more understandable parts, a path to solving it, and how to present it to a larger audience. This was invaluable during my PhD, as I learned how to structure my research, writing, and presentations. Eelco is also very kind, understanding, and motivating, even when things went wrong in research or real life, which would happen from time to time. Besides that, we would regularly discuss ideas, research, and applications that we would like to do. Since Eelco hired me as a Postdoctoral researcher, we will try to complete more of that. I would like to thank Guido Wachsmuth for his supervision, guidance, and kindness in the first half of my PhD, in which we worked closely together to develop declarative and incremental name and type analysis. Guido has a knack for deeply understanding a topic and explaining it in a very approachable manner, which was extremely useful at the start of my PhD. I learned many things from Guido, including compiler construction, (meta-)domain-driven design, Stratego programming, proper benchmarking, and academic writing. Besides that, we always had a lot of interesting ideas and conversations, and in general just had a lot of fun. Guido has found a new opportunity at Oracle Labs as a member of the PGX team, but we still regularly talk and cooperate, as some of our research is being applied at Oracle Labs. I wish to thank Sebastian Erdweg for his supervision in the latter half of my PhD, in which we worked on bootstrapping and incremental build systems. Without Sebastian’s guidance, the research would have never finished on time, or be of the quality that it is now. Sebastian always gave extremely good feedback on my planning, writing, and research, which helped me to improve those skills a lot, especially academic writing. One of Sebastian’s superpowers is coming up with interesting examples (that may make or break your approach) within seconds, which was extremely useful for developing incremental build systems as it has many nasty corner cases. I am grateful to Michael Steindorfer for his help on our interactive pipeline research, moments after finishing his PhD on data structures. He was able to quickly jump into our research and help out, and came up with the name PIE, which stands for Pipelines for Interactive Environments, which we still use xvi today. Hassan Chafi gave me the opportunity to do three summer traineeships at Oracle Labs in California, for which I am grateful. I was able to learn a great deal about practical application of my research in a more corporate setting, and got to meet many like-minded people. Besides the great work environment, having the opportunity to live in California and explore it during my free time was wonderful. The weather in California, in contrast to the Netherlands, was always sunny (except for the San Francisco fog) and dry, which made exploring cities and nature delightful. I would like to thank the members of the committee, Dick Epema, Matthew Flatt, Tijs van der Storm, Andrey Mokhov, Eelco Dolstra, and Koen Langendoen for reviewing my dissertation. I have many past and current colleagues to thank for their help and hospi- tality. Danny Groenewegen co-manages the servers that run our build farm and artifact server that greatly increase the productivity of our group. He also co-authored the task engine paper by helping us redefine the name binding and type system of WebDSL, which we used as a subject in our evaluation. Besides that, Danny is very open-hearted, always welcoming newcomers by inviting them to activities. I got into Magic the Gathering and Factorio because Danny invited me to his friend group, which I still play with to this day. Vlad Vergu worked on several important parts of our infrastructure. At the start of his PhD, he converted and moved the existing SVN repository to GitHub, which greatly increased our productivity. Vlad bootstrapped the Java version of the Stratego compiler, which was previously bootstrapped with a fixed baseline C version of the Stratego compiler, enabling further development on the Java version. He also built a command-line version of Spoofax, called Sunshine, which was an inspiration for Spoofax Core, a platform-independent version of Spoofax. Finally, he co-authored the task engine paper by performing the majority of the benchmarking, which was invaluable to the evaluation in that paper. Lennart Kats and Karl Trygve Kallenberg got me started with Spoofax and my PhD in general. They built Spoofax and several other related tools that made this line of research possible. Lennart’s dissertation on Building Blocks for Language Workbenches inspired parts of this dissertation. Karl came up with the name Spoofax, which consists of Spoo (a food from Babylon 5) and fax (as in a fax machine), for which the domain-name was free, and which is easy to search for. Maartje de Jonge had just finished her PhD on Language-Parametric Tech- niques for Language-Specific editors, which also inspired parts of this dissertation. Because Maartje just finished when I started, she was leaving Delft and was able to transfer her apartment to me, finally giving me an affordable place to stay in Delft. Hendrik van Antwerpen started out as a programmer in our group, helping immensely in getting Spoofax Core to work. He developed a Maven plugin, implemented cross-language dependencies, bootstrapped the meta-languages, and made several improvements make Spoofax more cross-platform. Later he

Chapter 0. Preface xvii did his master thesis with us, and then started his PhD, taking over the name and type analysis work. I admire Hendrik’s ability to deeply understand and continuously improve large systems. Luis Eduardo de Souza Amorim (Eduardo) started his PhD roughly the same time as I did. He has worked on improved the parsing and editor services side of Spoofax. Eduardo’s hard work, perseverance, and ability to understand things is something I greatly admire. He was able to dive into the existing parser generator and parser code, understand every interaction, and make many very important improvements to it, all while publishing academic papers. Daco Harkes evaluated Spoofax during his PhD by developing his IceDust DSL in it, which drove us to continuously improve Spoofax. We also always had interesting conversations about our research and a whole range of other (probably more geeky) topics. I admire Daco’s ability to go very broad, take in all available information, come to a decision and explain it, and then forge his own path forward. Daniel Pelsmaeker evaluated Spoofax Core in his master thesis by building an IntelliJ plugin for it, showing that it is indeed cross-platform. He made several improvements to Spoofax Core to make it more platform-independent, including a cross-platform configuration framework. He now started a PhD with us on editor services. Daniel is always enthusiastic and optimistic, and has a very fine attention to detail, which I appreciate a lot. Jeff Smits developed FlowSpec, a new meta-DSL for Spoofax for declaratively specifying dataflow analyses, and developed an incremental Stratego compiler, which can speed up the compilation time of a Spoofax project almost by an order of magnitude. He has also made several contributions to Spoofax and PIE. I admire Jeff’s ability to absorb knowledge and to be able to easily explain it to others, and his continuous drive to improve our tools and code. Jasper Denkers engineered JSGLR2, a replacement for the parser of Spoofax, with a modular architecture and better performance. He has also made several improvements to the build system and modularity of Spoofax. In his PhD, he is applying and evaluating Spoofax to real-world DSLs in industry. Jasper’s ability to decompose a system into its constituents and improve it, and his ability to connect our research to industry, is something I admire. Martijn Dwars set up automated feedback and grading for the Compiler Construction lab, which greatly reduced the workload for graders, while giving students the opportunity to get feedback early and improve their grade. He also contributed numerous small improvements to Spoofax. Martijn is one of the best programmer and problem solver I know. I wish to thank Elmer van Chastelet and Stepen van der Laan (along with Danny) for managing our servers and keeping them working. I am grateful to Roniet Sharabi, secretary of our group, who is always supportive, has helped me understand the bureaucracy of the university numerous times, and always made sure the academic process went smoothly. Last but not least, I would like to thank Sven Keidel, Peter Mosses, Casper Bach Poulsen, Robbert Krebbels, Arjen Rouvoet, Tamás Szabó, Paolo Giarrusso, Roelof Sol, xviii Volker Lanting, Oskar van Rest, Taico Aerts, and Chiel Bruin. We had many interesting conversations and it was a pleasure working with you. Finally, I would like to thank my friends and family. This work would not have been possible without my parents, Helma and Rasit, who have always been there for me and supported me my entire life. I would like to thank my grandpa, Loe, for always being there for me and teaching me how to use computers since I was two years old, which led to programming being my vocation. Unfortunately, he passed away in 2013. While he cannot be here in person, he will always be in my heart. I am grateful to Charlotte for always being there for me to talk, keeping me (somewhat) sane through this PhD adventure. Finally, I’d like to thank the rest of my friends and family, of which there are too many to write down, for their support.

Chapter 0. Preface xix xx Introduction

My thesis is that language-parametric methods for developing interactive programming systems are feasible and useful. All computers run software, such as operating systems, web browsers, chat 1 applications, photo editors, and video games. Software is used on many different computer systems by billions of people around the world, and has become such a crucial part of our lives. Therefore, it is important that we develop high-quality software. Software consists of programs that control computers, which are developed using programming languages. Typically, these programming languages are General-Purpose Languages (GPLs), supporting the development of many different kinds of software, applicable across many problem domains. To develop high-quality software, we do not only need good programming languages and programmers, but also need high-quality interactive programming systems that involve programmers in the exchange of correct and responsive feedback. Fortunately, for many GPLs, Integrated Development Environments (IDEs) provide correct and responsive interactive programming systems through code editors, editor services, and inline feedback. On the other hand, Domain-Specific Languages (DSLs) are programming languages that are specialized towards a specific problem domain. DSLs allow us to develop better software through linguistic abstraction for specific problem domains, supporting direct expression of problems and solutions in terms of the domain, and domain-specific constraint checking. However, because DSLs are specialized towards a specific domain, and there are many problem domains, we need to develop many new DSLs, including their interactive programming systems! Manually developing an interactive programming system for a DSL is not feasible, as developing one requires a huge development effort [77]. Therefore, our vision is to create and improve language-parametric methods for developing interactive programming systems. A language-parametric method takes as input a description of a language, and automatically implements (parts of) an interactive programming system, reducing the development effort, thereby making DSL development feasible. We now explore this vision in detail in the rest of this introductory chapter. We cover background information on programming languages, domain-specific languages, and programming systems. We describe interactive programming systems and the challenges in developing them. We describe our vision to tackle these challenges: language-parametric methods for developing interactive programming systems. We summarize our contributions, which show the feasibility and usefulness with concrete instances of these language-parametric methods. Finally, we describe our research methodology and the structure of

1 the rest of the dissertation.

1.1 Programming Systems A programming language is a formal language for writing programs that control computers. Programmers, or software developers, develop software that consist of programs or source code written in one or more programming languages. Therefore, programs are a mechanism to communicate the intent of a programmer to a computer. Programming languages such as C, Java, and Rust, are General-Purpose Languages (GPLs), supporting the development of many different kinds of software, applicable across many problem domains. New general-purpose programming languages appear less often since developing, evolving, and maintaining one is a large undertaking. On the other hand, Domain-Specific Languages (DSLs) are programming languages that are specialized to a specific domain, supporting only the development of solutions to the problem domain [27]. Examples of DSLs are Pic, a language for specifying diagrams in terms of boxes and arrows [11]; Structured Query Language (SQL)[23], a language to declaratively query, modify, and compute data in relational databases; Make [133], a declarative language to specify file-based build systems with incremental execution; and Backus–Naur Form (BNF)[10, 82], a meta-DSL (i.e., a DSL specializing in the domain of languages) to declaratively specify context-free grammars of programming languages. Domain-specific languages provide several advantages over their general- purpose counterparts. Because a DSL is specialized towards a single domain, it can support direct expression of problems and solutions in terms of the domain. Furthermore, because DSL programs relate directly to the problem domain, a DSL can provide domain-specific syntax and perform domain-specific error checking, statically ruling out wrong programs that could not be restricted by a GPL. Finally, because a DSL program is of a higher level than a GPL program, it is possible to derive multiple semantics from a single DSL program by different interpretations or compilation schemata. However, because DSLs are specialized to a particular domain, and there are many problem domains, we need many DSLs, and they need to be developed [102] and maintained [26]. A programming system is needed when developing a new language, consisting at least of a compiler that validates programs and transforms them into an executable form [2]. For example, the Java programming system consists of the javac compiler which checks and transforms Java source files into Java bytecode Intermediate Representation (IR). The Rust programming system has the rustc compiler which compiles rust source files to machine code. Besides a batch compiler, programming systems typically contain more tooling such as package managers, build systems, interpreters, and debuggers. For example, Rust has the cargo package manager. Java has the Java Virtual Machine (JVM) which executes Java bytecode IR. However, we will focus on a compiler. Programmers interact with programming systems through a command-line terminal, manually running the batch compiler after they have made changes

2 to the source code, providing feedback to the programmer in the form of compiler error messages and warnings for invalid programs, or lack thereof for valid programs. While this is a ﬂexible way to develop programs – a programmer can choose to use any source code editor, and run the batch compiler on any operating system in their favorite shell – this development process suffers from several problems. First of all, there is a disconnect between the programming system and the source code editor: error messages from the batch compiler are not displayed inline, and instead need to be manually traced from the text in the terminal back to the source code, introducing a cognitive gap. Furthermore, batch compilers are typically not responsive: the programmer needs to manually run the batch compiler after making changes to the source code, and wait for feedback, inducing a slow and tedious feedback cycle. Finally, there is a lack of feedback: batch compilers only provide error/warning messages. Editor services such as syntax highlighting, structure outlines, or code completion are missing, because the command-line terminal is restricted to text. These problems limit programmer productivity. We need an interactive programming system that increases productivity by providing automatic, continuous, and inline feedback to the programmer.

1.2 Interactive Programming Systems

An interactive programming system is a programming system that is designed to involve the user in the exchange of information [68]. In an interactive programming system, there is a continuous exchange of information between the programmer and the system: a feedback cycle where the programmer edits the source code of a program, the system sends back feedback, and so forth. Integrated Development Environments (IDEs) such as Atom, Eclipse, emacs, IntelliJ, MPS, NetBeans, Notepad++, vim, Visual Studio (Code), and XCode are interactive programming systems for certain programming languages. For example, IntelliJ IDEA is an interactive programming system including built- in support for Java and Kotlin. Interactive programming systems increase productivity by: • Closing the cognitive gap by providing inline error/warning messages and other interactions directly in terms of the source code through a code editor. For example, regions in the source code with errors are highlighted, show the error message when hovered with the mouse, and support application of quick ﬁxes which directly modify the source code. • Automatically providing feedback when changes to the source code are made, and providing this feedback in a timely manner. • Providing better feedback in the form of editor services. For example, syntax coloring provides typographical styling based on the syntactical and semantic meaning of code, structure outlines provide browsable summaries of the source code, reference resolution supports browsing between declarations and references, and code completion provides context-dependent browsing

Chapter 1. Introduction 3 and automatic completion of unambiguous source code sentences [114]. However, for a programming system to be truly interactive, it must be correct and responsive. An interactive programming system only provides feedback that is correct, when the process providing that feedback is precise. A responsive interactive programming system provides feedback automatically: without explicit user interaction where possible, and more importantly, in a timely manner. When an interactive programming system lacks these qualities, productivity is lost. For example, providing an incorrect code completion or quick ﬁx to the programmer, which leads to errors in the code after application, confuses and annoys the programmer. Furthermore, explicitly having to ask for syntax coloring, or waiting for ﬁve seconds to get a new structure outline, after modifying the source code, is tedious. A method to achieve responsiveness in interactive programming systems is incrementality, where the response time is proportional to the impact of a change to the source code. An incremental system achieves this by only recomputing outputs that have been affected by a changed input, while reusing previously computed outputs. For example, typing a character into a code editor (most of the time) does not affect the syntax coloring of the text before and after that new character, requiring only syntax coloring of the newly typed character. However, responsiveness is only achieved when incrementality is scalable, where the programming system can scale down to many low-impact source code changes, while scaling up to large programs. Since most source code changes have a low impact, few outputs should be recomputed, and response times should be fast, even though programs are large. Finally, it is important that correctness is still guaranteed in the presence of incrementality and scalability. Developing correct and responsive interactive programming systems is a challenge, which we now review.

1.3 Developing Interactive Programming Systems Developing interactive programming systems requires the implementation of code editors and editor services for every programming language. Fortunately, IDEs are extensible, supporting reuse of its code editor and editor services by creating a plugin that connects the programming system of a programming language to the editor and editor services of the IDE [114]. Therefore, instead of developing an interactive programming system from scratch for each programming language, we use the code editor and editor services from IDEs. However, developing responsive and correct interactive programming systems is a challenge, as IDEs do not provide support for implementing programming systems with these qualities, requiring manual application of the incrementality and scalability methods. Manually implementing incrementality is a challenge, as it requires the implementation of cross-cutting techniques such as dependency tracking, caching, cache invalidation, change detection, and persistence, which are complicated and error-prone to implement. Making

4 incrementality scale is even more challenging, as incrementality must scale up to tracking large dependency graphs, cache large amounts of data, do cache invalidation through these large graphs, and detect low-impact changes. Finally, the programming system, and the incremental and scalable implementation of these parts, needs to be correct. However, since these methods are complicated and error-prone to implement, they cause subtle incrementality bugs that are hard to detect and reproduce, therefore reducing the correctness of the programming system. In summary, manual implementation of correct incremental and scalable interactive programming environments has a high development and maintenance effort, preventing us from developing interactive programming environments for the many DSLs that need to be developed for many problem domains. What we need is a systematic approach to make interactive programming systems responsive and correct, without having to manually implement the complicated methods to achieve these qualities for every programming language. Therefore, we should use language-parametric methods for developing responsive and correct interactive programming systems. A language-parametric method takes as input an implementation or description of a programming language, and automatically produces an instance of that method, without the developer having to know much about the method at all. For example, a language-parametric incremental name and type analysis framework takes as input a language description, and automatically produces a correct and responsive name and type analysis, without the language developer having to worry about incrementality and scalability, and correctness thereof. This enables us to efﬁciently develop and maintain correct and responsive interactive programming environments for many DSLs. The idea of language-parametric methods is not new. Therefore, we ﬁrst describe several existing language-parametric methods, describe open problems, and then follow up with our contributions: language-parametric methods for developing responsive and correct interactive programming systems.

1.4 Language-Parametric Methods One way of developing a programming language, is to use several disjunct but flexible compiler-compiler tools, which are tools that compile the compiler of a programming language. For example, one can specify a lexical analyzer (lexer) in tools such as Lex [19] or Flex [96], and then a context-free parser in Yacc [19] or Bison [96]. From such specifications, programs implementing a lexer and context-free analyzer are generated. A parser can then be created by feeding the tokens from the lexer into the context-free parser. Therefore, these tools are language-parametric methods for developing programming systems. However, these tools do not support the development of interactive programming systems, and are only available for a limited subset of the language development domain, such as parsing. On the other hand, a Language Workbench (LWB) is a set of unified tools for developing (interactive) programming systems, with the goal of lowering the cost of developing and maintaining the programming system of DSLs. Al-

Chapter 1. Introduction 5 though language workbenches have been around since the 1980s, the term was coined by Martin Fowler only in 2005, in his blogpost "Language Workbenches: The Killer-App for Domain Specific Languages?" [46]. An example of an early language workbench is the Synthesizer Genera- tor [123], which is a tool for generating editors (program synthesizers) from programming language descriptions, using incremental execution and inline error messages for responsive feedback. The ASF + SDF Meta-Environment [80, 14] language workbench included support for specifying program transformations and generation of interactive programming systems. Modern language workbenches have been studied in the Language Work- bench Challenge series of workshops, of which the 2013 edition resulted in a survey comparing language workbench features [39, 38]. For example, MetaEdit+ [78] is a platform-independent graphic language workbench for domain-specific modeling. MPS [159] is a projectional language workbench supporting non-textual notations such as tables and diagrams. Rascal [81] is a metaprogramming language and IDE for source code analysis and transformation. Finally, Spoofax [75] is a language workbench for specification of textual domain-specific languages with full IDE support. Language workbenches provide language-parametric methods through meta- languages, which are programming languages that aid in the development of programming languages. Typically, meta-languages are domain-specific instead of general-purpose. For example, Spoofax provides SDF [153], a meta- DSL for syntax specification, Stratego [18], a meta-DSL for program analysis and transformation, and ESV [75], a meta-DSL for editor service specification. The SDF compiler generates a parse table and pretty-printer from an SDF syntax specification; the Stratego compiler generates an executable program analyzer and transformer; and the ESV specification is interpreted to provide editor services such as syntax highlighting and code completions. Therefore, Spoofax provides language-parametric methods for developing interactive programming systems. There are three open problems with the language-parametric methods of language workbenches that we tackle in this dissertation: missing support for several sub-domains of language development, a lack of responsiveness, and a lack of integration between language-parametric methods. Several subdomains of language development, such as name analysis and bootstrapping, have no domain-specific language-parametric methods, requiring these problems to be solved by manual encoding in a general-purpose method, reducing correctness and increasing the development and maintenance effort. We want to move to more domain-specific language-parametric methods, to benefit from the advantages of domain-specificity, such as direct expression of problems and solutions in terms of the domain, domain-specific consistency checking, and deriving multiple semantics from the same specification. Furthermore, several language-parametric methods are not truly responsive; either requiring manual implementation of complicated methods such as scalable incrementality, reducing correctness and increasing effort; or cannot

6 be made responsive at all, reducing iteration times and increasing tedium. We want language-parametric methods for interactive programming environments to be responsive without much effort by the language developer. Finally, there is a lack of integration between the language-parametric methods of language workbenches. Language workbenches have many components, such as multiple domain-speciﬁc meta-languages, their compilers or interpreters, generated artifacts of these compilers, compilers that may generate compilers, editor services, and so forth. All these components must integrate in a correct and responsive way, for the language workbench to be correct and responsive. However, this integration is typically manually implemented in an ad-hoc way, again increasing effort and reducing correctness and responsiveness. We need a systematic approach to integrate the language-parametric methods and components of language workbenches. This dissertation addresses these open problems with ﬁve contributions.

1.5 Contributions

We now summarize the ﬁve core contributions of this dissertation.

1.5.1 NaBL: A Meta-DSL for Declarative Name Binding and Scope Rules

Every programming language needs to deal with names, their declarations and references, scopes, and importing of names into scopes. This is the name analysis step of the compiler of a programming system, which is often implemented manually with techniques such as nested environments maintained by tree traversals, or imperative lookup operations. However, this requires the language developer to think about how to develop name analysis, distracting from what the name binding rules of their language should be. Therefore, instead of manually implementing the name binding and scope rules of a language, we have developed a language-parametric method for declaratively specifying the name binding rules of a programming language with the Name Binding Language (NaBL), a domain-specific, declarative, meta-language. With NaBL, a language developer declaratively specifies the name binding rules of their programming language in terms of definition and use sites of names, properties of these names associated to language constructs, namespaces for separating categories of names, scopes in which definitions are visible, and imports between scopes. From such a specification, we automatically derive a name analysis, and editor services for inline error checking, reference resolution, and code completion. We evaluate NaBL by specifying many name binding features of C# in NaBL, and by specifying common name binding patterns such as scoped, non-unique, globally visible, and subsequently visible definitions; and overloaded, type-directed, and nested references. In conclusion, NaBL provides a language-parametric method for developing correct name analyses and corresponding editor services.

Chapter 1. Introduction 7 1.5.2 A Language Independent Task Engine for Incremental Name and Type Analysis While NaBL provides a language-parametric method for name analysis, it does not support the definition of typing rules, and is not incremental, and therefore does not truly provide editor services for a responsive interactive programming environment. To mitigate this problem, we have developed a language independent task engine for incremental name and type analysis. In this approach, we specify naming rules in NaBL, and typing rules in TS – a meta-DSL for simple type system specification – from which we automatically derive a traversal that collects naming and typing tasks when given a program. Then, we collect tasks for a program and send them to the task engine, which incrementally executes changed tasks to incrementally execute name and type analysis, updating data structures required for editor services such as code completion, and responsively providing inline name and type error/warning messages. We experimentally evaluate the correctness and responsiveness of our approach by running the incremental name and type analysis against the changes in the source code repository of a real-world application written in a real-world DSL. The evaluation shows that incremental and non-incremental analysis produce the same solution, showing correctness, and that for single-file changes, incremental analysis takes between 0.37 and 1.12 seconds, showing acceptable response times for interactive settings. In conclusion, NaBL, TS, and the incremental task engine provide a language- parametric method for developing correct and responsive name and type analyses with corresponding editor services. 1.5.3 Bootstrapping Domain-Specific Meta-Languages of Language Workbenches A bootstrapped compiler can compile its own source code, because the compiler is written in the compiled language itself. For example, the GCC compiler for the C language is a bootstrapped compiler; its source code is written in C and can compile itself. Bootstrapping yields several advantages: • The compiler is written in the compiled high-level language. • It provides a large-scale test case for detecting defects in the compiler and the compiled language. • It shows that the language’s coverage is sufficient to implement itself. • Compiler improvements such as better static analysis or the generation of faster code applies to all compiled programs, including the compiler itself. A language workbench provides high-level meta-languages (e.g. NaBL and TS) for developing DSLs and their compilers. Thus, users of a language workbench (language developers) develop their DSL in meta-languages, and therefore do not have to bootstrap their own language, which is good since DSLs have limited expressiveness and are often ill-suited for compiler development. What we desire instead, is bootstrapping the meta-language compilers of language workbenches, inheriting the benefits of bootstrapping stated above. However, bootstrapping a language workbench is complicated by the fact

8 that most provide multiple separate domain-specific meta-languages for describing different language aspects such as syntax, name analysis, type analysis, code generation, and so forth. Thus, in order to build a meta-language compiler, multiple meta-language compilers need to be applied, entailing intricate dependencies that sound language workbench bootstrapping needs to handle. Furthermore, most language workbenches provide an interactive programming system for their meta-languages, supporting interactive development of DSLs. Therefore, bootstrapping operations must be available and observable in this interactive environment. Our solution to these problems is to do versioning and dependency tracking between meta-languages, and perform fixpoint bootstrapping, where we iteratively self-apply meta-language compilers to derive new versions until no change occurs. Fixpoint bootstrapping is correct: it either produces a new baseline when it reaches a fixpoint, or stops and displays an error when it finds a defect (i.e., applying a meta-language compiler in an iteration failed), as long as meta-language compilers are deterministic and converge to a fixpoint. Furthermore, bootstrapping operations can be started, cancelled (when diverging), and rolled back (when defect) interactively, supporting the interactive programming system of the language workbench. To evaluate our approach, we have implemented fixpoint bootstrapping for the Spoofax language workbench, and used it to successfully bootstrap eight meta-languages with seven changes. In conclusion, our approach provides a (meta)language-parametric method for correctly and interactively bootstrapping the meta-languages of language workbenches, in an interactive programming environment. 1.5.4 PIE: a DSL, API, and Runtime for Interactive Software Development Pipelines A software development pipeline automates parts of the software engineering process, such as building software via build scripts, continuous integration testing and benchmarking on build farms, and automatic deployment of software artifacts to production servers. An interactive software development pipeline builds software artifacts, but also reacts immediately to changes in input, and provides timely feedback to the user. An interactive programming system is an instance of such a pipeline, where changes to programs are immediately processed to provide timely feedback to programmers. However, interactivity complicates the development of pipelines, if responsiveness and correctness become the responsibility of the pipeline programmer, rather than being supported by the underlying system. Therefore, we need a system that is expressive enough to describe interactive software development pipelines, such as the interactive programming systems of LWBs, while still being correct and responsive. Most build systems are not expressive enough, as they only support static dependencies, where all dependencies to files or build tasks have to be stated statically and up-front in the build script, whereas in a software development pipeline many dependencies only become evident during build execution. One build system that is expressive enough, is Pluto [36], a sound and optimal

Chapter 1. Introduction 9 incremental build system with support for dynamic dependencies, enabling build tasks to create dependencies to files and tasks during build execution, possibly based on (dynamic) values produced by previous build tasks. However, Pluto suffers from four open problems affecting ease of development: requiring a lot of Java boilerplate to define build tasks; semi-automated persistence, requiring pipeline programmers to manually reason about where to cache task outputs; explicit dependency tracking where dependencies could be inferred; and missing domain-specific software development pipeline features such as file system paths and support for lists. To solve these problems, we have developed Pipelines for Interactive Envi- ronments (PIE), a DSL, Application Program Interface (API), and runtime for developing correct and responsive interactive software development pipelines, where ease of development is a focus. The PIE DSL serves as a front-end for developing pipelines with minimal boilerplate in a functional language with support for concepts from the interactive software development pipeline domain such as dependencies, filesystem paths and operations, and list operations. The PIE API is a lower-level front-end for developing foreign pipeline functions which cannot be modeled in the DSL, while having reduced boilerplate compared to Pluto, and also serves as a compilation target for the DSL. Finally, the runtime incrementally executes pipelines implemented in the API using Pluto’s incremental build algorithm, while fully automating persistence and inferring dependencies where possible. We evaluate PIE with two case studies, one being the reimplementation of a significant part of the interactive programming system of the Spoofax language workbench in PIE. The existing pipeline of Spoofax’s interactive programming system was scattered across four different formalisms, decreasing ease of development; overapproximates dependencies, causing loss of incrementality; and underapproximates dependencies, causing loss of correctness. However, with PIE, we can easily integrate the different components of Spoofax; such as its parser, analyzers, transformations, build scripts, editor services, meta- languages, and dynamic language loading; into a single formalism. PIE ensures that the pipeline is correct and responsive, without the pipeline programmer having to implement techniques such as incrementality, or without having to reason about correctness. In conclusion, PIE provides a language-parametric method for developing interactive software development pipelines, a superset of correct and responsive interactive programming environments.

1.5.5 Scalable Incremental Building with Dynamic Task Dependencies Previous work on PIE builds forth on Pluto by improving its ease of use, but essentially uses the same incremental build algorithm. To make a build up-to- date after changes, the Pluto incremental build algorithm traverses the entire dependency graph (produced in a previous build) from top to bottom, while re-executing tasks affected by a change, and possibly executing new tasks. This enables Pluto to detect changes to ﬁles and tasks without the user having to tell Pluto what has actually changed, while also elegantly discovering changes to dynamic dependencies by (re-)executing (new) tasks. However, the downside is

10 that this algorithm does not scale, because the traversal is dependent on the size of the dependency graph, not the impact of the change. This quickly becomes a problem in interactive programming systems, where there are many changes and those changes have a low-impact (e.g., programmer typing characters into an editor), while the program and its induced dependency graph is large. For example, in the Spoofax language workbench pipeline, we observed ∼ 3 second build times even when nothing has changed. Therefore, we need a new incremental build algorithm that scales down to many low-impact changes, while scaling up to large dependency graphs, while still supporting dynamic dependencies. To solve this scalability problem, we have developed a new incremental build algorithm that performs change-driven rebuilding. It takes as input a set of changed ﬁles, starts rebuilding directly affected tasks from the changed leaves of the dependency graph, and rebuilds transitively affected tasks, while also accounting for new task dependencies discovered during rebuilding. Our algorithm scales with the impact of a change, and is independent from the size of the dependency graph, because it only ever visits affected tasks. We experimentally evaluate our change-driven bottom-up algorithm by comparison against Pluto’s top-down algorithm. As a subject, we use the Spoofax-PIE pipeline, a real-world build script for the interactive programming system of the Spoofax language workbench. To measure incremental performance an scalability, we synthesized a chain of 60 realistic changes of varying types and impacts. Results show that for low-impact changes (i.e., changes that only cause a small number of tasks to be actually affected), our change-driven algorithm is several orders of magnitude faster than Pluto’s top-down algorithm, while not slower for high-impact changes. In conclusion, our new algorithm makes PIE scalable, in addition to being correct and responsive. This in turn makes the Spoofax PIE pipeline scalable, providing a language-parametric method for developing truly correct and responsive interactive programming systems.

1.6 Research Methodology We now describe the research methodology used in the core contributions of this dissertation. Mary Shaw identiﬁed ﬁve types of research questions [128] based on the submissions to the International Conference on Software Engineering (ICSE). The type of question we answer in this dissertation is a "method or means of development". That is, what is a better way to develop (or: how do we automate the development of) correct and responsive interactive programming systems? Answering this question requires designing new (language-parametric) methods for developing interactive programming systems. In order to design these methods systematically, we follow the iterative approach of the memorandum on design-oriented information systems research [113], consisting of four phases: analysis, design, evaluation, and diffusion. In the analysis phase, we identify and describe open problems in the development of interactive programming systems, and analyze the relation to existing

Chapter 1. Introduction 11 approaches. In the design phase, we design new methods for developing interactive programming systems that solve open problems, and create tools that implement those methods. These tools come in the form of meta-DSLs and their compilers, APIs that can be programmed against, algorithms that execute instances of a generic model, and systems that combine these artifacts. In the evaluation phase, we evaluate to what degree our methods and tools solve open problems. When the open problem is an ease of development problem, we evaluate coverage of our method by application to real-world case studies, show this application in a research paper or external artifact, and discuss to what degree we improve ease of development. When the open problem is a performance problem, we experimentally evaluate our approach against real-world case studies and subjects, measure performance differences, show the measurement results in a research paper, and discuss to what degree we improve performance. We experimentally evaluate the correctness of our approaches with the same case studies and subjects. In the diffusion phase, we publish our ﬁndings as research papers to conferences or journals and present our research at these conferences. Furthermore, we try to apply our research in industry to gain further insights into applicability and what can be improved, and apply our research to our own systems (dogfooding), possibility spinning up new research projects.

1.7 Structure We now discuss the rest of the structure of this dissertation. The main chapters of this dissertation are based on five peer-reviewed publications. The author of this dissertation is the main contributor of all publications, and the first author for four of the publications. In the publication for chapter 3, Guido Wachsmuth is the first author, since he did most of the writing, whereas the author of this dissertation implemented the task engine, implemented the benchmark subjects, helped with benchmarking, and wrote parts of the introduction and evaluation sections, and the related work section. Since each main chapter is based on a stand-alone publication with distinct contributions, there is some redundancy, especially in the introduction sections. However, we chose not to remove that redundancy, in order to ensure that each chapter can be read independently. The main chapters and their corresponding publication are as follows: • Chapter 2 is an updated version of the SLE 2012 paper Declarative Name Binding and Scope Rules [89]. • Chapter 3 is an updated version of the SLE 2013 paper A Language Indepen- dent Task Engine for Incremental Name and Type Analysis [160]. • Chapter 5 is an updated version of the GPCE 2016 paper Bootstrapping Domain-Specific Meta-Languages in Language Workbenches [86]. • Chapter 7 is an updated version of the Programming 2018 paper PIE: A Domain-Specific Language for Interactive Software Development Pipelines [91]. • Chapter 8 is an updated version of the ASE 2018 paper Scalable Incremental

12 Building with Dynamic Task Dependencies [87]. We reﬂect on incremental name and type analysis, bootstrapping, and our engineering work on Spoofax Core in chapter 4, and reﬂect on language workbench pipelines in chapter 6. Finally, we end with a conclusion in chapter 9 where we summarize our work, discuss the work in relation to the thesis, and discuss future work.

Chapter 1. Introduction 13 14 NaBL: A Meta-DSL for Declarative Name Binding and Scope Rules Abstract 2 In textual programming languages, names are used to reference elements like variables, methods, classes, etc. Name resolution analyses these names in order to establish references between definition and use sites of elements. In this chapter, we identify recurring patterns for name bindings in programming languages and introduce NaBL, a declarative metalanguage for the specification of name bindings in terms of namespaces, definition sites, use sites, and scopes. Based on such declarative name binding specifications, we provide a language-parametric algorithm for static name resolution during compile-time. We discuss the integration of the algorithm into the Spoofax Language Work- bench and show how its results can be employed in semantic editor services like reference resolution, constraint checking, and content completion.

2.1 Introduction Software language engineering is concerned with linguistic abstraction, the formalization of our understanding of domains of computation in higher-level software languages. Such languages allow direct expression in terms of the domain, instead of requiring encoding in a less specific language. They raise the level of abstraction and reduce accidental complexity. One of the key goals in the field of language engineering is to apply these techniques to the discipline itself: high-level languages to specify all aspects of software languages. Declarative languages are of particular interest since they enable language engineers to focus on the What? instead of the How?. Syntax definitions are a prominent example. With declarative formalisms such as Extended Backus–Naur Form (EBNF), we can specify the syntactic concepts of a language without specifying how they can be recognized programmatically. This declarativity is crucial for language engineering. Losing it hampers evolution, maintainability, and compositionality of syntax definitions [76]. Despite the success of declarative syntax formalisms, we tend to programmatic specifications for other language aspects. Instead of specifying languages, we build programmatic language processors, following implementation patterns in rather general specification languages. These languages might still be considered domain-specific, when they provide special means for programmatic language processors. They also might be considered declarative, when they abstract over computation order. However, they enable us only to implement language processors faster, but not to specify language aspects. They lack

15 domain concepts for these aspects and focus on the How?. That is a problem since (1) it entails overhead in encoding concepts in a programming language and (2) the encoding obscures the intention; understanding the definition requires decoding. Our goal is to extend the set of really declarative, domain-specific languages for language specifications. In this paper, we are specifically concerned with name binding and scope rules. Name binding is concerned with the relation between definitions and references of identifiers in textual software languages, including scope rules that govern these relations. In language processors, it is crucial to make information about definitions available at the references. Therefore, traditional language processing approaches provide programmatic abstractions for name binding. These abstractions are centered around tree traversal and information propagation from definitions to references. Typically, they are not specifically addressing name binding, but can also be used for other language processing tasks such as compilation and interpretation. Name binding plays a role in multiple language engineering processes, including editor services such as reference resolution, code completion, refac- torings, type checking, and compilation. The different processes need different information about definitions. For example, name resolution tries to find one definition, while code completion needs to determine all possible references in a certain place. The different requirements lead either to multiple re-implementations of name binding rules for each of these purposes, or to non-trivial, manual weaving into a single implementation supporting all purposes. This results in code duplication with as result errors, inconsistencies, and increased maintenance effort. The traditional paradigm influences not only language processing, but also language specification. For example, the Object Constraint Language (OCL) standard [111] specifies name binding in terms of nested environments, which are maintained in a tree traversal. The C# language specification [134] defines name resolution as a sequence of imperative lookup operations. In this paper, we abstract from the programmatic mechanics of name resolution. Instead, we aim to declare the roles of language constructs in name binding and leave the resolution mechanics to a generator and run-time engine. We introduce the NaBL, a language with linguistic abstractions for declarative definition of name binding and scope rules. NaBL supports the declaration of definition and use sites of names, properties of these names associated with language constructs, namespaces for separating categories of names, scopes in which definitions are visible, and imports between scopes. NaBL is integrated in the Spoofax Language Workbench [75], but can be reused in other language processing environments. From definitions in the name binding language, a compiler generates a language-specific name resolution strategy in the Stratego rewriting language [151] by parametrizing an underlying generic, language independent strategy. Name resolution results in a persistent symbol table for use by semantic editor services such as reference resolution, consistency checking of definitions, type checking, refactoring, and code generation. The implementation supports multiple file analysis by

16 default. We proceed as follows. In sections 2.2 and 2.3 we introduce NaBL by example, using a subset of the C# language. In section 2.4 we discuss the derivation of editor services from a name binding speciﬁcation. In section 2.5 we give a high-level description of the generic name resolution algorithm underlying NaBL. In section 2.6 we discuss the integration of NaBL into the Spoofax Language Workbench. In section 2.7 we discuss NaBL’s applicability to different languages. Finally, section 2.8 discusses related work.

2.2 Declarative Name Binding and Scope Rules In this section we introduce NaBL, illustrated with examples drawn from the specification of name binding for a subset of C# [134]. Listing 2.1 defines the syntax of the subset in Syntax Definition Formalism (SDF)[153]. The subset is by no means complete; it has been selected to model representative features of name binding rules in programming and domain-specific languages. In the following subsections we discuss the following fundamental concepts of name binding: definition and use sites, namespaces, scopes, and imports. For each concept we give a general definition, illustrate it with an example in C#, and then we show how the concept can be modeled in NaBL. 2.2.1 Definitions and References The essence of name binding is establishing relations between a definition that binds a name and a reference that uses that name. Name binding is typically defined programmatically through a name resolution algorithm that connects references to definitions. A definition site is the location of a definition in a program. In many cases, definition sites are required to be unique, that is, there should be exactly one definition site for each name. However, there are cases where definition sites are allowed to be non-unique. Example. Listing 2.2a contains class definitions in C#. Each class definition binds the name of a class. Thus, we have definition sites for A, B, and C. Base class specifications are references to these definition sites. In the example, we have references to A as the base class of B and B as the base class of C. (Thus, B is a sub-class of, or inherits from A.) There is no reference to C. The definition sites for A and B are unique. By contrast, there are two definition sites for C, defining parts of the same class C. Thus, these definition sites are non-unique. This is correct in C#, since regular class definitions are required to be unique, while partial class definitions are allowed to be non-unique. Abstract Syntax Terms. In Spoofax, abstract syntax trees (ASTs) are represented using first-order terms. Terms consist of strings ("x"), lists of terms (["x","y"]), and constructor applications (ClassType("A")) for labelled tree nodes with a fixed number of children. Annotations in grammar productions (listing 2.1) define the constructors to be used in AST construction. For example, Class(Partial(), "C", Base("B"), []) is the representation of the

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 17 Using* NsMem*-> CompUnit{"Unit"}

"using" NsOrTypeName ";"-> Using{"Using"} "using"ID "=" NsOrTypeName-> Using{"Alias"} ID-> NsOrTypeName{"NsOrType"} NsOrTypeName "."ID-> NsOrTypeName{"NsOrType"}

"namespace"ID "{" Using* NsMem* "}"-> NsMem{"Namespace"} Partial "class"ID Base "{" ClassMem* "}"-> NsMem{"Class"}

-> Partial{"NonPartial"} "partial"-> Partial{"Partial"} -> Base{"NoBase"} ":"ID-> Base{"Base"}

TypeID ";"-> ClassMem{"Field"} RetTypeID "("{Param ","}* ")" Block ";"-> ClassMem{"Method"}

ID-> Type{"ClassType"} "int"-> Type{"IntType"} "bool"-> Type{"BoolType"} Type-> RetType "void"-> RetType{"Void"} TypeID-> Param{"Param"}

"{" Stmt* "}"-> Block{"Block"} Decl-> Stmt EmbStmt-> Stmt "return" Exp ";"-> Stmt{"Return"} TypeID ";"-> Decl{"Var"} TypeID "=" Exp ";"-> Decl{"Var"} Block-> EmbStmt StmtExp ";"-> EmbStmt "foreach" "(" TypeID "in" Exp ")" EmbStmt-> EmbStmt{"Foreach"}

INT-> Exp{"IntLit"} "true"-> Exp{"True"} "false"-> Exp{"False"} ID-> Exp{"VarRef"} StmtExp-> Exp Exp "."ID-> StmtExp{"FieldAccess"} Exp "."ID "("{Exp ","}* ")"-> StmtExp{"Call"} ID "("{Exp ","}* ")"-> StmtExp{"Call"}

Listing 2.1: Syntax deﬁnition in SDF2 for a subset of C#. The names in the annotations are abstract syntax tree constructors.

18 classA{} rules classB:A {} Class(NonPartial(),c,_,_): defines unique classc partial classC:B{ Class(Partial(),c,_,_): defines non-unique classc } Base(c): refers to classc partial classC{} ClassType(c): refers to classc

(a) Class declarations (b) NaBL speciﬁcation of deﬁnitions and references for C# class in C#. names.

Listing 2.2: Deﬁnitions and references of names.

classx{ namespaces class field method variable int x; rules Field(_,f): defines unique fieldf voidx() { Method(_,m,_,_): defines unique methodm int x; Call(m,_): refers to methodm x = x +1; } Var(_,v): defines unique variablev VarRef(x): refers to variablex } otherwise to fieldx

(a) Homonym declarations in C#. (b) NaBL speciﬁcation for different C# namespaces.

Listing 2.3: Namespaces of deﬁnitions and use sites.

first partial class in listing 2.2a. A term pattern is a term that may contain variables (x) and wildcards (\_). Model. A specification in NaBL consists of a collection of rules of the form pattern: clause* , where pattern is a term pattern and clause* is a list of name binding declarations about the language construct that matches with pattern. Listing 2.2b shows a declaration of the definitions and references for class names in C#. The first two rules declare class definition sites for class names. Their patterns distinguish regular (non-partial) and partial class declarations. While non-partial class declarations are unique definition sites, partial class declarations are non-unique definition sites. The third rule declares that the term pattern Base(c) is a reference to a class with name c. Thus, the ":A" in listing 2.2a is a reference to class A. Similarly, the second rule declares a class type as a reference to a class. 2.2.2 Namespaces Definitions and references declare relations between named program elements and their uses. Languages typically distinguish several namespaces, i.e. different kinds of names, such that an occurrence of a name in one namespace is not related to an occurrence of that same name in another. Example. Listing 2.3a shows several definitions for the same name x, but of different kinds, namely a class, a field, a method, and a variable. Each of these kinds has its own namespace in C#, and each of these namespaces has its own

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 19 classC{ rules voidm() { Class(NonPartial(),c,_,_): int x; defines unique classc } scopes field, method } Class(Partial(),c,_,_): classD{ defines non-unique classc voidm() { scopes field, method int x; int y; Method(_,m,_,_): { int x; x = y +1;} defines unique methodm x = y +1; scopes variable } } Block(_): scopes variable

(a) Scoped homonym method and variable (b) NaBL speciﬁcation of scopes for different declarations in C#. C# namespaces.

Listing 2.4: Named and anonymous lexical scopes. name x. This enables us to distinguish the definition sites of class x, field x, method x, and variable x, which are all unique. Model. We have declared definitions and references for the namespace class already in the previous example. Listing 2.3b extends that declaration covering also the namespaces field, method, and variable. Note that it is required to declare namespaces to ensure the consistency of name binding rules. Definition sites are bound to a single namespace (defines classc ), but use sites are not. For example, a variable in an expression might either refer to a variable, or to a field, which is modeled in the last rule. In our example, this means that variable declarations hide field declarations, because variables are resolved to variables, if possible. Thus, both x in the assignment in listing 2.3a refer to the variable x. 2.2.3 Scopes Scopes restrict the visibility of definition sites. A named scope is the definition site for a name which scopes other definition sites. By contrast, an anonymous scope does not define a name. Scopes can be nested and name resolution typically looks for definition sites from inner to outer scopes. Example. Listing 2.4a includes two definition sites for a method m. These definition sites are not distinguishable by their namespace method and their name m, but, they are distinguishable by the scope they are in. The first definition site resides in class C, the second one in class D. In C#, class declarations scope method declarations. They introduce named scopes, because class declarations are definition sites for class names. The listing also contains three definition sites for a variable x. Again, these are distinguishable by their scope. In C#, method declarations and blocks scope variable declarations. Method declarations are named scopes, blocks are anonymous scopes. The first definition site resides in method m in class C, the second one in method m in class

20 namespaceN{ namespaces namespace classN{} rules Namespace(n,_): namespaceN{ classN{}} defines namespacen } scopes namespace, class

(a) Nested namespace declarations in C#. (b) NaBL speciﬁcation for C# nested namespace declarations.

Listing 2.5: Nested C# namespaces.

D, and the last one in a nameless block inside method m in class D. In the assignment inside the block (line 9), x refers to the variable declaration in the same block, while the x in the outer assignment (line 10) refers to the variable declaration outside the block. In both assignments, y refers to the variable declaration in the outer scope, because the block does not contain a definition site for y. Model. The scopes ns clause in NaBL declares a construct to be a scope for namespace ns. Listing 2.4b declares scopes for fields, methods, and variables. Named scopes are declared at definition sites. Anonymous scopes are declared similarly, but lack a defines clause. 2.2.4 Namespaces as Language Concepts C# has a notion of ‘namespaces’. It is important to distinguish these namespaces as a language concept from namespaces as a naming concept, which group names of different kinds of declarations. Specifically, in C#, namespace declarations are top-level scopes for class declarations. Namespace declarations can be nested.

Example. Listing 2.5a declares a top-level namespace N, scoping a class declaration N and an inner namespace declaration N. The inner namespace declaration scopes another class declaration N. The definition sites of the namespace name N and the class name N are distinguishable, because they belong to different namespaces (as a naming concept). The two definition sites of namespace name N are distinguishable by scope. The outer namespace declaration scopes the inner one. Also, the definition sites of the class name N are distinguishable by scope. The first one is scoped by the outer namespace declaration, while the second one is scoped by both namespace declarations. Model. The names of C# namespace declarations are distinguishable from names of classes, fields, etc. As declared in listing 2.5b, their names belong to the namespace namespace. The name binding rules for definition sites of names of this namespace models the scoping nature of C# namespace declarations.

2.2.5 Imports An import introduces into the current scope deﬁnitions from another scope, either under the same name or under a new name. An import that imports all deﬁnitions can be transitive.

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 21 usingN; rules Using(qname): namespaceM{ imports class from namespace ns classC{ where qname refers to namespace ns int f; } Alias(alias, qname): } imports namespace ns as alias where qname refers to namespace ns namespaceO{ otherwise imports classc as alias usingD = M.C; where qname refers to classc classE:D { voidm() {} Base(c): } imports field(transitive), classF:E { } method(transitive) } from classc

(a) Various forms of imports in C#. (b) NaBL speciﬁcation of C# import mechanisms.

Listing 2.6: Importing deﬁnitions from another scope.

Example. Listing 2.6a shows different kinds of imports in C#. First, a using directive imports type declarations from namespace N. Second, another using directive imports class C from namespace M into namespace O under a new name D. Finally, classes E and F import fields and methods from their base classes. These imports are transitive, that is, F imports fields and methods from E and D. Model. Listing 2.6b shows name binding rules for import mechanisms in C#. The first rule handles using declarations, which import all classes from the namespace to which the qualified name qname resolves to. The second rule models aliases, which either import a namespace or a class under a new name, depending on the resolution of qname. The last rule models inheritance, where fields and methods are imported transitively from the base classes into the current (parent class) scope. 2.2.6 Types So far, we discussed names, namespaces, and scopes to distinguish definition sites for the same name. Types also play a role in name resolution and can be used to distinguish definition sites for a name or to find corresponding definition sites for a use site. Example. Listing 2.7a shows a number of overloaded method declarations. These share the same name m, namespace method, and scope class C. But we can distinguish them by the types of their parameters. Furthermore, all method calls inside method x can be uniquely resolved to one of these methods by taking the argument types of the calls into account. Model. Listing 2.7b includes type information into name binding rules for fields, methods, and variables. Definition sites might have types. In the simplest case, the type is part of the declaration. In the example, this holds for parameters. For method calls, the type of the definition site for a method name

22 classC{ rules voidm() {} Method(t,m,p*,_): voidm(int x) {} defines unique methodm voidm(bool x) {} of type(t*,t) voidm(int x, int y) {} wherep* has typet* voidm(bool x, bool y) {} voidx() { Call(m,a*): m(); refers to methodm m(42); of type(t*,_) m(true); wherea* has typet* m(21,21); m(true, false); Param(t,p): } defines unique variablep } of typet

(a) Overloaded method declarations in (b) Types in name binding rules for C# overloaded C#. methods.

Listing 2.7: Types of definitions, and type-dependent use sites. depends on the types of the parameters. A type system is needed to connect the type of a single parameter, as declared in the rule for parameters, and the type of a list of parameters, as required in the rule for methods. We will discuss the influence of a type system and the interaction between name and type analysis later. For now, we assume that the type of a list of parameters is a list of types of these parameters. Type information is also needed to resolve method calls to possibly overloaded methods. The refers clause for method calls therefore requires the corresponding definition site to match the type of the arguments. Again, we omit the details how this type can be determined. We also do not consider subtyping here. Method calls and corresponding method declarations need to have the same argument and parameter types.

2.3 Name Binding Patterns We now identify typical name binding patterns. These patterns are formed by scopes, definition sites and their visibility, and use sites referencing these definition sites. We explain each pattern first and give an example in C# next. Afterwards, we show how the example can be modeled with declarative name binding rules. 2.3.1 Unscoped Definition Sites In the simplest case, definition sites are not scoped and globally visible. Example. In C#, namespace and class declarations (as well as any other type declaration) can be unscoped. They are globally visible across file boundaries. For example, the classes C1, C2, and C3 in Listing 2.2a are globally visible. In listing 2.3a, only the outer namespace N is globally visible. In contrast to C#, C++ has file scopes and all top-level declarations are only visible in a file. To share global declarations, each file has to repeat the

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 23 rules CompilationUnit(_,_): scopes namespace, class

(f, CompilationUnit(_,_)): defines filef scopes namespace, class

Listing 2.8: Two ways to model ﬁle scope for top-level syntax tree nodes in NaBL.

rules Namespace(n,_): defines non-unique namespacen in surrounding scope

Var(t,c): defines unique variable of typet in subsequent scope

Listing 2.9: NaBL speciﬁcation of the visibility of deﬁnition sites inside scopes.

declaration and mark it as extern. This is typically achieved by importing a shared header file. Model. We consider any definition site that is not scoped by another definition site or by an anonymous scope to be in global scope. These definition sites are visible over file boundaries. File scope can be modeled with a scoping rule in two different ways. Both are illustrated in listing 2.8. The first rule declares the top-level node of abstract syntax trees as a scope for all namespaces which can have top-level declarations. This scope will be anonymous, because the top-level node cannot be a definition site (otherwise this definition site would be globally visible). The second rule declares a tuple consisting of file name and the abstract syntax tree as a scope. This tuple will be considered a definition site for the file name. Thus, the scope will be named after the file.

2.3.2 Deﬁnition Sites inside their Scopes

Typically, definition sites reside inside the scopes where they are visible. Such definition sites can either be visible only after their declaration, or everywhere in their surrounding scope. Example. In C#, namespace members such as nested namespace declarations and class declarations are visible in their surrounding scope. The same holds for class members. In contrast, variable declarations inside a method scope become visible only after their declaration. Model. Scoped definition sites are by default visible in the complete scope. Optionally, this can be stated explicitly in defines clauses. Listing 2.9 illustrates this for namespace declarations. The second rule in this listing shows how to model definition sites which become visible only after their declaration.

24 classC{ rules voidm(int[] x) { foreach(intx in x) Foreach(t,v, exp, body): System.Console.WriteLine(x); defines unique variablev } of typet } in body

(a) Loop with scoped iterator variable x in C#. (b) NaBL speciﬁcation of C# foreach loops.

Listing 2.10: Deﬁnition sites outside of their scopes.

rules Seq(Var(t,v), stmts): defines unique variablev of typet in stmts

[Var(t,v) | stmts]: defines unique variablev of typet in stmts

Listing 2.11: Two ways to model deﬁnition sites becoming visible after their declaration.

2.3.3 Definition Sites outside their Scopes Some declarations include not only the definition site for a name, but also the scope for this definition site. In such declarations, the definition site resides outside its scope. Example. Let expressions are a classical example for definition sites outside their scopes. In C#, foreach statements declare iterator variables, which are visible in embedded statements. Listing 2.10a shows a method with a parameter x, followed by a foreach statement with an iterator variable of the same name. This is considered incorrect in C#, because definition sites for variable names in inner scopes collide with definition sites of the same name in outer scopes. However, the use sites can still be resolved based on the scopes of the definition sites. The use site for x inside the loop refers to the iterator variable, while the x in the collection expression refers to the parameter. Model. Listing 2.10b shows the name binding rule for foreach loops, stating the scope of the variable explicitly. Note that definition sites which become visible after their declaration are a special case of this pattern. Listing 2.11 illustrates how this can be modeled in the same way as the foreach loop. The first rule assumes a nested representation of statement sequences, while the second rule assumes a list of statements. 2.3.4 Contextual Use Sites Definition sites can be referenced by use sites outside of their scopes. These use sites appear in a context which determines the scope into which they refer. This context can either be a direct reference to this scope, or has a type which determines the scope. Example. In C#, namespace members can be imported into other namespaces. Listing 2.5a shows a class N in a nested namespace. In listing 2.12a, this class is

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 25 using N.N.N; rules NsOrType(n1, n2): refers to namespace n2 in ns namespaceN ' { otherwise to class n2 in ns classC{ where n1 refers to namespace ns C f; voidm(C p) { } FieldAccess(e,f): } refers to fieldf inc classD{ wheree has type ClassType(c) voidm(C p) { p.m(p.f); MethodCall(e,m,p*): } refers to methodm of type(t*,_) inc } wheree has type ClassType(c) } wherep* has typet*

(a) Contextual use sites in C#. (b) NaBL speciﬁcation of contextual use sites.

Listing 2.12: Contextual use sites.

imported. The using directive refers to the class with a qualified name. The first part of this name refers to the outer namespace N. It is the context of the second part, which refers to the inner namespace N. The second part is then the context for the last part of the qualified name, which refers to the class N inside the inner namespace. Listing 2.12a also illustrates use sites in a type-based context. In method m in class D, a field f is accessed. The corresponding definition site is outside the scope of the method in class C. But this scope is given by the type of p, which is the context for the field access. Similarly, the method call is resolved to method m in class C because of the type of p.

Model. Listing 2.12b illustrates how to model contextual use sites. The scope of the declaration site corresponding to a use site can be modeled in refers clauses. This scope needs to be determined from the context of the use site. The first rule resolves the context of a qualified name part to a namespace ns and declares the use site to refer either to a namespace or to a class in ns. The remaining rules declare use sites for field access and method calls. They determine the type of the context, which needs to be a class type. A field access refers to a field in that class. Similarly, a method call refers to a method with the right parameter types in that class.

2.4 Editor Services

Modern IDEs provide a wide range of editor services where name resolution plays a large role. Traditionally, each of these services would be handcrafted for each language supported by the IDE, requiring substantial effort. However, by accurately modeling the relations between names in NaBL, it is possible to generate a name resolution algorithm and editor services that are based on that algorithm.

26 Figure 2.1: Reference resolution of name field reference to name field definition, derived from a NaBL specification

Figure 2.2: Error checking derived from NaBL rules. posterName is declared twice, and nam is not declared at all.

2.4.1 Reference Resolving

Name resolution is exposed directly in the IDE in the form of reference resolving: press and hold Control and hover the mouse cursor over an identifier to reveal a blue hyperlink that leads to its definition side. This behavior is illustrated in fig. 2.1.

2.4.2 Constraint Checking

Modern IDEs statically check programs against a wide range of constraints. Constraint checking is done on the fly while typing and directly displayed in the editor via error markers on the text and in the outline view. Error checking constraints are generated from the NaBL for common name binding errors such as unresolved references, duplicate definitions, use before definition and unused definitions. Figure 2.2 shows an editor with error markers. The message parameter in the post method has a warning marker indicating that it is not used in the method body. On the line that follows it, the posterName variable is assigned but has not yet been declared, violating the visibility rules of listing 2.9. Other errors in the method include a subsequent duplicate definition of posterName, which violates the uniqueness constraint of the variable namespace of listing 2.3b, and referencing a non-existent property nam.

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 27 Figure 2.3: Code completion for ﬁelds and local variables.

2.4.3 Code Completion With code completion, partial (or empty) identifiers can be completed to full identifiers that are valid at the context where code completion is executed. Figure 2.3 shows an example of code completion. In the left program code completion is triggered on a field access expression on the user object. The user object is of type User, so all fields of User are shown as candidates. On the right, completion is triggered on a variable reference, so all variables in the current scope are shown.

2.5 Implementation To implement name resolution based on NaBL, we employ a name resolution algorithm that relies on a symbol table data structure to persist name bindings and lazy evaluation to resolve all references. In this section we give an overview of the data structure, the name resolution algorithm, and their implementation. 2.5.1 Persistence of Name Bindings To persist name bindings, each definition and reference is assigned a qualified name in the form of a Uniform Resource Identifier (URI). The URI identifies the occurrence across a project. Use sites share the URIs of their corresponding definition sites. A URI consists of the namespace, the path, and the name of a definition site. As an example, the URI method://N/C/m is assigned to a method m in a class C in a namespace N. Here, the segments represent the names of the scopes. Anonymous scopes are represented by a special path segment anon(u), where u is a unique string to distinguish different anonymous scopes. For use in analyses and transformations, URIs can be represented in the form of Abstract Syntax Trees (ASTs). For example, [method(),"N","C","m"] is the URI for method m. All name bindings are persisted in an in-memory data structure called the semantic index. It consists of a symbol table that lists all URIs that exist in a project, and can be efficiently implemented as a hash table. It maps each URI to the file and offset of their occurrences in the project. It can also store additional information, such as the type of a definition.

28 2.5.2 Resolving Names Our algorithm is divided into three phases. First, in the annotation phase, all definition and use sites are assigned a preliminary URI, and definition sites are stored in the index. Second, definition sites are analyzed, and their types are stored in the index. And third, any unresolved references are resolved and stored in the index. Annotation Phase. In the first phase, the AST of the input file is traversed in top-down order. The logical nesting hierarchy of programs follows from the AST, and is used to assign URIs to definition sites. For example, as the traversal enters the outer namespace scope n, any definitions inside it are assigned a URI that starts with ‘n.’. As a result of the annotation phase, all definition and use sites are annotated with a URI. In the case of definition sites, this is the definitive URI that identifies the definition across the project. For references, a temporary URI is assigned that indicates its context, but the actual definition it points to has to be resolved in a following phase. For reference by the following phases, all definitions are also stored in the index. Definition Site Analysis Phase. The second phase analyzes each definition site in another top-down traversal. It determines any local information about the definition, such as its type, and stores it in the index so it can be referenced elsewhere. Types and other information that cannot be determined locally are determined and stored in the index in the last phase. Use Site Analysis Phase. When the last phase commences, all local information about definitions has been stored in the index, and non-local information about definitions and uses in other files is available. What remains is to resolve references and to determine types that depend on non-local information (in particular, inferred types). While providing a full description of the use site analysis phase and the implementation of all name binding constructs is outside the scope of this paper, the below steps sketch how each reference is resolved:

1. Determine the temporary URI ns://path/n which was annotated in the first analysis phase. 2. If an import exists in scope, expand the current URI for that import. 3. If the reference corresponds to a name-binding rule that depends on non- local information such as types, retrieve that information. 4. Look for a definition in the index with namespace ns, path path, and name n. If it does not exist, try again with a prefix of path that is one segment shorter. If the no definition is found this way, store an error for the reference. 5. If the definition is an alias, resolve it. An important part to highlight in the algorithm is the interaction between name and type analysis that happens for example with the FieldAccess expression of listing 2.12b. For name binding rules that depend on types or other non-local information, it is possible that determining the type recursively trig-

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 29 gers name resolution. For this reason, we apply lazy evaluation, ensuring that any reference can be resolved lazily as requested in this phase. By traversing through the entire tree, we ensure that all use sites are eventually resolved and persisted to the index.

2.6 Integration into Spoofax NaBL, together with the index, is integrated into the Spoofax Language Work- bench. Stratego rules are generated by the NaBL compiler that use the index API to interface with Spoofax. In this section we will show the index API and how the API is used to integrate the editor services seen in section 2.4. 2.6.1 Index API Once all analysis phases have been completed, the index is filled with a summary of every file. To use the summaries we provide the index API with a number of lookups and queries. Lookups transform annotated identifiers into definitions. Queries transform definitions (retrieved using a lookup) into other data. The API is used for integrating editor services, but is also exposed to Spoofax language developers for specifying additional editor services or other transformations. index-lookup-one looks for a definition of the given identifier in its owning scope. The index-lookup lookup performs a lookup that tries to look for a definition using index-lookup-one. If it cannot be found, the lookup is restarted on the outer scope until the root scope is reached. If no definition is found at the root scope, the lookup fails. There is also an index-lookup-all variant that returns all found definitions instead of stopping at the first found definition. Finally, index-lookup-all-levels is a special version of index-lookup-all that supports partial identifiers. To get data from the index, index-get-data is used. Given a definition and a data kind, it will return all data values of that kind that is attached to the definition. Uses are retrieved in the same way using index-get-uses-all. 2.6.2 Reference resolution Resolving a reference to its definition is very straightforward when using index-lookup, since it does all the work for us. The only thing that has to be done when Spoofax requests a reference lookup is a simple transformation: node-> node . The resulting definition has location information embedded into it which is used to navigate to the reference. If the lookup fails, this is propagated back to Spoofax and no blue hyperlink will appear on the node under the cursor. 2.6.3 Constraint checking Constraint checking rules are called by Spoofax after analysis on every AST node. If a constraint rule succeeds it will return the message and the node where the error marker should be put on. The duplicate definition constraint check that was shown earlier is defined

30 constraint-error: node-> (key, "Duplicate definition") where node; key := node; defs := key; ( defs,1)

Listing 2.13: Duplicate deﬁnitions constraint check written in Stratego, using generated code from the NaBL compiler and the index API.

editor-complete: ast-> identifiers where node@COMPLETION(name) := ast; proposals := node; identifiers := proposals

Listing 2.14: Code completion.

in listing 2.13. First nam-unique (generated for unique definitions by the NaBL compiler) is used to see if the node represents a unique definition; non-unique definition such as partial classes should not get duplicate definition error markers. The identifier is retrieved using nam-key and a lookup in the current scope is done with index-lookup-one. If more than one definition is found, the constraint check succeeds and an error marker is shown on the node.

2.6.4 Code completion

When code completion is requested in Spoofax, a completion node is substituted at the place where the cursor is. For example, if we request code completion on VarRef("a"), it will be substituted by VarRef(COMPLETION("a")) to indicate that the user wants to complete this identifier. See listing 2.14 for the code completion implementation. We first retrieve the completion node and name using collect-one. Completion proposals are gathered by index-lookup-all-levels since it can handle partial identifiers. Finally the retrieved proposals are converted to names by mapping index-uri-name over them.

2.7 Evaluation and Discussion

Our aim with this work has been to design high-level abstractions for name resolution applicable to a wide range of programming languages. In this section we discuss the limitations of our approach and evaluate its applicability to different languages and other language features than those covered in the preceding sections.

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 31 2.7.1 Limitations

There are two areas of possible limitations of NaBL. One is in the provided abstraction, the other is in the implementation algorithm that supports it. As for the provided abstraction, as a definition language, NaBL is inherently limited in the number of features it can support. While the feature space it supports is extensive, ultimately there may always be language features or variations that are not supported. For these cases, the definition of NaBL – written in Stratego – can be extended, or it is possible to escape NaBL and extend a specification using handwritten Stratego rules. As for the implementation algorithm, NaBL’s current implementation strategy relies on laziness, and does not provide much control over the traversal for the computation of names or types. In particular, sophisticated type inference schemes are not supported with the current algorithm. To implement such schemes, the algorithm would have to be extended, preferably in a way that maintains compatibility with the current NaBL definition language.

2.7.2 Coverage

During the design and construction of NaBL, we have performed a number of studies on languages and language features to determine the extent of the feature space that NaBL would support. In this paper we highlighted many of the features by using C# as a running example, but other languages that we studied include a subset of general-purpose programming languages C, Java, and domain-specific languages WebDSL [59], the Hibernate Query Language (HQL), and Mobl [61]. We also applied our approach to the Java Bytecode stack machine language using the Jasmin [104] syntax. For our studies we used earlier prototypes of NaBL, which led to the design as it is now. Notable features that we studied and support in NaBL are partial classes, inheritance, visibility, lexical scoping, imports, type-based name resolution, and overloading; all of which have been discussed in section 2.3. In addition, we studied aspect-oriented programming with intertype declarations and pointcuts, file-based scopes in C, and other features. Our design has also been influenced by past language definitions, such as SDF and Stratego. Altogether, it is fair to say that NaBL supports a wide range of language features and extensive variability, but can only support the full range of possible programming languages by allowing language engineers to escape the abstraction. In future work, we would like to enhance the possibilities of extending NaBL and design a better interface for escapes.

2.8 Related work

We give an overview of other approaches for specifying and implementing name resolution. The main distinguishing feature of our approach is the use of linguistic abstractions for name bindings, thus hiding the low level details of writing name analysis implementations.

32 2.8.1 Symbol Tables In classic compiler construction, symbol tables are used to associate identifiers with information about their definition sites, typically including type information. Symbol tables are commonly implemented using hash tables where the identifiers are indexed for fast lookup. Scoping of identifiers can be implemented in a number of ways. For example, by using qualified identifiers as index, nesting symbol tables, or destructively updating the table during program analysis. The type of symbol table influences the lookup strategy. When using qualified identifiers the entire identifier can be looked up efficiently, but considering outer scopes requires multiple lookups. Nesting symbol tables always requires multiple lookups but is more memory efficient. When destructively updating the symbol table, lookups for visible variables are very efficient, but the symbol table is not available after program analysis. The index we use is a symbol table that uses qualified identifiers. We map qualified identifiers (URIs) to information such as definitions, types and uses. 2.8.2 Attribute Grammars Attribute Grammars (AGs) [83] are a formal way of declaratively specifying and evaluating attributes for productions in formal grammars. Attribute values are associated with nodes and calculated in one or more tree traversals, where the order of computations is determined by dependencies between attributes. Eli [74] provides an attribute grammar specification language for modular and reusable attribute computations. Abstract, language-independent computations can be reused in many languages by letting symbols from a concrete language inherit these computations. For example, computations Range, IdDef, and IdUse would calculate a scope, definitions, and references. A method definition can then inherit from Range and IdDef, because it defines a function and opens a scope. A method call inherits from IdUse because it references a function. These abstract computations are reflected by naming concepts of NaBL and the underlying generic resolution algorithm. However, NaBL is less expressive, but more domain-specific. Where Eli can be used to specify general (and reusable) computations on trees, NaBL is restricted to name binding concepts, helping to understand and specify name bindings more easily. Silver [163] is an extensible attribute grammar specification language which can be extended with general-purpose and domain-specific features. Typical examples are auto-copying, pattern matching, collection attributes, and support for data-flow analysis. However, name analysis is performed the traditional way: an environment with bindings is passed down the tree using inherited properties. Reference Attribute Grammars (RAGs) extend AGs by introducing attributes that can reference nodes, substantially simplifying name resolution implementations. JastAdd [53] is a meta-compilation system for generating language processors relying on RAGs and object orientation. It also supports parametrized

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 33 attributes to act as functions where the value depends on the given parameters. A typical name resolution as seen in [32, 53, 3] is implemented in lookup attributes parameterized by an identifier of use sites, such as variable references. All nodes that can have a variable reference as a child node, such as a method body, then have to provide an equation for performing the lookup. These equations implement scoping and ordering using Java code. JastAdd implementations have much more low level details than NaBL declarations. This provides flexibility, but entails overhead on encoding and requires decoding for understanding. For example, scopes for certain program elements are encoded within a set of equations, usually implemented by early or late returns. 2.8.3 Visibility Predicates CADET [112] is a notation for predicates and functions over abstract syntax tree nodes. Similar to attribute grammar formalisms, it allows to specify general computations in trees but lacks reusable concepts for name binding. Poetsch-Heffter proposes dedicated name binding predicates [119], which can be translated into efficient name resolution functions [118]. In contrast to NaBL, scopes are expressed in terms of start and end points and multi-file analyses are not supported. 2.8.4 Dynamic Rewrite Rules In term rewriting, an environment passing style does not compose well with generic traversals. As an alternative, Stratego allows rewrite rules to create dynamic rewrite rules at run-time [17]. The generated rules can access variables available from their definition context. Rules generated within a rule scope are automatically retracted at the end of that scope. Hemel et al. [60] describe idioms for applying dynamic rules and generic traversals for composing definitions of name analysis, type analysis, and transformations without explicitly staging them into different phases. Our current work builds on the same principles, but applies an external index and provides a specialized language for name binding declarations. Name analysis with scoped dynamic rules is based on consistent renaming, where all names in a program are renamed such that they are unequal to all other names that do not correspond to the same definition site. Instead of changing the names directly in the tree, annotations can be added which ensure uniqueness. This way, the abstract syntax tree remains the same modulo annotations. Furthermore, unscoped dynamic rewrite rules can be used for persistent mappings [75]. 2.8.5 Textual Language Workbenches Xtext [12] is a framework for developing textual software languages. The Xtext Grammar Language is used to specify abstract and concrete syntax, but also name bindings by using cross-references in the grammar. Use sites are then automatically resolved by a simplistic resolution algorithm. Scoping or visibility cannot be defined in the Grammar Language, but have

34 to be implemented in Java with help of a scoping API with some default resolvers. For example field access, method calls, and block scopes would all need custom Java implementations. Only package imports have special support and can be specified directly in the Grammar Language. Common constraint checks such as duplicate definitions, use before definition, and unused definitions also have to be specified manually. This increases the amount of boilerplate code that has to be rewritten for every language. In contrast to Xtext’s Grammar Language, NaBL definitions are separated from syntax definitions in Spoofax. This separation allows us to specify more advanced name binding concepts without cluttering the grammar with these concepts. It also preserves language modularity. When syntax definitions are reused in different contexts, different name bindings can be defined for these contexts, without changing the grammar. From an infrastructure perspective, Spoofax and Xtext work similarly, using a global index to store summaries of files and URIs to identify program elements. EMFText [57] is another framework for developing textual software languages. Like Xtext, it is based on the Eclipse Modeling Framework [135] and relies on metamodels to capture the abstract syntax of a language. While in Xtext this metamodel is generated from a concrete syntax definition, EMFText takes the opposite approach and generates a default syntax definition based on the UML Human-Usable Textual Notation [65] from the metamodel. Lan- guage designers can then customize the syntax definition by adding their own grammar rules. In the default setup, reference resolution needs to be implemented in Java. Only simple cases are supported by default implementations [58]. JastEMF [20] allows to specify the semantics of EMF metamodels using JastAdd RAGs by integrating generated code from JastAdd and EMF.

Chapter 2. NaBL: Declarative Name Binding and Scope Rules 35 36 A Language Independent Task Engine for Incremental Name and Type Analysis Abstract 3 Interactive programming systems such as IDEs depend on incremental name and type analysis to provide responsive feedback for large programs. In this chapter, we present a language-independent approach to incremental name and type analysis. Analysis consists of two phases. The first phase analyzes lexical scopes and binding instances and creates deferred analysis tasks. A task captures a single name resolution or type analysis step. Tasks might depend on other tasks and are evaluated in the second phase. Incrementality is supported on file and task level. When a file changes, only this file is recollected and only those tasks are reevaluated, which are affected by the changes in the collected data. The analysis neither re-parses nor re-traverses unchanged files, even if they are affected by changes in other files. We implement the approach as part of the Spoofax Language Workbench and evaluate it for the WebDSL web programming language.

3.1 Introduction IDEs provide a wide variety of language-specific editor services such as syntax highlighting, error marking, code navigation, content completion, and outline views in real-time, while a program is edited. These services require syntactic and semantic analyses of the edited program. Thereby, timely availability of analysis results is essential for IDE responsiveness. Whole-program analyses do not scale because the size of the program determines the performance of such analyses. An incremental analysis reuses previous analysis results of unchanged program parts and reanalyses only parts affected by changes. The granularity of the incremental analysis directly impacts the performance of the analysis. A more fine-grained incremental analysis is able to reanalyze smaller units of change, but requires a more complex change and dependency analysis. At program level, any change requires reanalysis of the entire program, which might consider the results of the previous analysis. At file level, a file change requires reanalysis of the entire file and all dependent files. At program element level, changes to an element within a file require reanalysis of that element and dependent elements, but typically not of entire files. Incremental analyses are typically implemented manually. Thereby, change detection and dependency tracking are cross-cutting the implementation of the actual analysis. This raises complexity of the implementation and negatively affects maintenance, reusability, and modularity.

37 In this paper, we focus on incremental name and type analysis. We present a language-independent approach which consists of two phases. The first phase analyzes lexical scopes, collects information about binding instances, and creates deferred analysis tasks in a top-down traversal. An analysis task captures a single name resolution or type analysis step. Tasks might depend on other tasks and are evaluated in the second phase. Incrementality is supported on file level by the collection phase and on task level by the evaluation phase. When a file changes, only this file is recollected and only those tasks are reevaluated, which are affected by the changes in the collected data. As a consequence, the analysis does neither re-parse nor re-traverse unchanged files, even if they are affected by changes in other files. Only the affected analysis tasks are reevaluated. Our approach enables language engineers to abstract over incrementality. When applied directly, language engineers need to parametrize the collection phase, where they have full freedom to create and combine low-level analysis tasks. Thereby, they can focus solely on the name binding and typing rules of their language while the generic evaluation phase provides the incrementality. The approach can also form the basis for more high-level meta-languages for specifying the static semantics of programming languages. We use the task engine to implement incremental name analysis for name binding and scope rules expressed in NaBL, Spoofax’s declarative name binding language [89]. We have implemented the approach as part of the Spoofax language workbench [75] and evaluated it for WebDSL, a domain-specific language for the implementation of dynamic web applications [49], designed specifically to enable static analysis and cross-aspect consistency checking in mind [59]. We used real change-sets from the histories of two WebDSL applications to drive experiments for the evaluation of the correctness, performance and scalability of the obtained incremental static analysis. Experiment input data and the obtained results are publicly available. We proceed as follows. In section 3.2 we introduce the basics of name and type analysis and introduce the running example of the paper. In sections 3.3 and 3.4, we discuss the two analysis phases of our approach, collection and evaluation. In section 3.5, we discuss the implementation and its integration into the Spoofax language workbench. In section 3.6, we discuss the evaluation of our approach. Finally, we discuss related work in section 3.7 and conclude in section 3.8.

3.2 Name and Type Analysis In this section, we discuss name and type analysis in the context of the running example of the paper, a multi-file C# program shown in listing 3.1. 3.2.1 Name Analysis In textual programming languages, an identifier is a name given to program elements such as variables, methods, classes, and packages. The same identifier can have multiple instances in different places in a program. Name analysis establishes relations between a binding instance that defines a name and a

38 classA{ classB{ classC:A { B b; int m; int i; float f; floatm() { intm() { intn() { return1 + b.f; }} return0; }} return m(); }}

(a) Files before editing. The underlined expression causes a type error.

classA{ classB{ namespace N { B b; int m; int i; float f; classC: B{ int m(B b){ intm() { intn() { return1 + b. i;}} return 1;}} returnm(); }} }

(b) Files after editing. Changes w.r.t. listing 3.1a are highlighted.

Listing 3.1: C# class declarations in separate files with cross-file references. bound instance that uses that name [92]. Name analysis is typically defined programmatically through a name resolution algorithm that connects binding prospects to binding instances. When a prospect is successfully connected, it becomes a bound instance. Otherwise, it is a free instance. The C# class declarations in listing 3.1a contain several references, some of which cross file boundaries. The declared type of field b in class A refers to class B in a separate file. Also, the return expression of method m in class A accesses field f in class B. The parent of class C refers to class A in a separate file and the return expression of method n in class C is a call to method m in class A. Languages typically distinguish several namespaces, i.e. different kinds of names, such that an occurrence of a name in one namespace is not related to an occurrence of that same name in another. In the example, class A contains a field and a homonym method m, but C# distinguishes field and method names. Scopes restrict the visibility of binding instances. They can be nested and name analysis typically looks for binding instances from inner to outer scopes. In the example, b is resolved by first looking for a variable b in method A.m, before looking for a field b in class A.A named scope is the context for a binding instance, and scopes other binding instances. In the example, class A is a named scope. It is the context for a class name and a scope for method and field names. An alias introduces a new binding instance for an already existing one. An import introduces binding instances from one scope into another one. In the example, class C imports fields and methods from its parent class A. 3.2.2 Type Analysis In statically typed programming languages, a type classifies program elements such as expressions according to the kind of values they compute [117]. List- ing 3.1a declares method C.n of type int, meaning that this method is expected to compute signed 32-bit integer values. Type analysis assigns types to program elements. Types are typically calculated compositionally, with the type of a program element depending only on the types of its sub-elements [117]. Type checking compares expected with actual types of program elements. A

Chapter 3. A Task Engine for Incremental Name and Type Analysis 39 type error occurs if actual and expected type are incompatible. Type errors reveal at compile-time certain kinds of program misbehavior at run-time. In the example, the return expression in method C.n causes a type error. The expression is of type float, since the called method m returns values of this type. But the declaration of C.n states that it evaluates to values of type int. This will cause run-time errors, when a floating point value is returned by C.n, while an integer value is expected. Type analysis reveals this error early at compile-time. In some cases, type analysis depends on name analysis. In the example of the return expression in C.n, m needs to be resolved in order to calculate the type of the return expression. In other cases, name analysis depends on type analysis. For example, the type of b needs to be calculated in order to resolve f in the return expression of A.m. In general, name resolution cannot only depend on types, but on a variety of properties of binding and bound instances. 3.2.3 Incremental Analysis When a program changes, it needs to be reanalyzed. Different kinds of changes influence name and type analysis. First, adding a binding instance may introduce bindings for free instances, or rebind bound instances. Removing a binding instance influences all its bound instances, which are either rebound to other binding instances or become free instances. Changing a binding instance combines the effects of removing and adding. Second, adding a binding prospect requires resolution, while removing it makes a binding obsolete. Changing a binding prospect requires re-binding, resulting either in a new binding or a free instance. Third, addition, removal, or change of scopes or imports influence bound instances in the affected scopes, which might be rebound to different binding instances or become free instances. Similarly, they influence bound instances which are bound to binding instances in the affected scopes. Finally, addition of a typed element requires type analysis, while removing it makes a type calculation obsolete. Changing a typed element requires reanalysis. Furthermore, changes propagate along dependencies. When bound instances are rebound to different binding instances or become free instances, this influences bindings in the context of these bound instances, the type of these instances, the type of enclosing program elements, and bindings in the context of such types. Consider listing 3.1b for an example. It shows edited versions of the C# class declarations from listing 3.1a. We assume the following editing sequence:

1. The return type of method A.m is changed from float to int. This affects the type of the return expression of method C.n and solves the type error, but raises a new type error in the return expression of A.m. 2. The return expression of method A.m is changed to b.i. This requires resolution of i and affects the type of the expression, solving the type error.

40 3. Parameter B b is added to method A.m. This might affect the resolution and by this the type of b and i in the return expression, the type of the return expression, the resolution of m in method C.n, and the type of its return expression. Actually, only the resolution of b and m and the type of the return expression in C.n are affected. The latter resolution fails, causing a resolution error and leaving the return expression untyped. 4. The parent of class C is changed from A to B. This affects the resolution of m in method C.n and the type of its return expression. It ﬁxes the resolution error and the return expression becomes typed again. 5. Class C is enclosed in a new namespace N. This might affect the resolution of parent class B, the resolution of m in N.C.n, and the type of the return expression in N.C.n. Actually, it does not affect any of those. 6. The return expression of method m in class B is changed. This might affect the type of this expression, but actually it does not. We discuss incremental analysis in the next sections. We start with the collection phase in section 3.3, and continue with the evaluation phase in section 3.4.

3.3 Semantic Index We collect name binding information for all units in a project into a semantic index, a central data structure that is persisted across invocations of the analysis and across editing sessions. For the purpose of this paper, we model this data structure as binary relations over keys and values. As keys, we use URIs, which identify bindings uniquely across a project. As values, we use either URIs or terms. We use U and T to denote the set of all URIs and terms, respectively. 3.3.1 URIs We assign a URI to each binding instance, bound instance, and free instance. A bound instance shares the URI with its corresponding binding instance. A URI consists of a language name, a list of scope segments, the namespace of the instance, its name, and an optional unique qualifier. This qualifier helps to distinguish unique binding instances by numbering them consecutively. A segment for a named scope consists of the namespace, the name, and the qualifier of the scoping binding instance. Anonymous scopes are represented by a segment anon(u), where u is a unique string to distinguish different scopes. For example, C#://Class.A.1/Method.m.1 identifies method m in class A in the C# program in listing 3.1a. The qualifier 1 distinguishes the method. Possible homonym methods in the same class would get subsequent qualifiers. 3.3.2 Index Entries The index stores binding instances (B ⊆ U × U), aliases (A ⊆ U × U), transitive and non-transitive imports for each namespace ns (TIns ⊆ U × U and NIns ⊆ U × U), and types of binding instances (Ptype ⊆ U × T ). For a binding instance with URI u, B contains an entry (u0, u), where u0 is retrieved from u by omitting

Chapter 3. A Task Engine for Incremental Name and Type Analysis 41 Relation Key Value B C#:/Class.A C#:/Class.A.1 C#:/Class.A.1/Field.b C#:/Class.A.1/Field.b.1 C#:/Class.A.1/Field.m C#:/Class.A.1/Field.m.1 C#:/Class.A.1/Method.m C#:/Class.A.1/Method.m.1 C#:/Class.B C#:/Class.B.1 C#:/Class.B.1/Field.i C#:/Class.B.1/Field.i.1 C#:/Class.B.1/Field.f C#:/Class.B.1/Field.i.1 C#:/Class.B.1/Method.m C#:/Class.B.1/Method.m.1 C#:/Class.C C#:/Class.C.1 C#:/Class.C.1/Method.n C#:/Class.C.1/Method.n.1

NIField, TIField C#:/Class.C.1 Task:/31

NIMethod, TIMethod C#:/Class.C.1 Task:/31

Ptype C#:/Class.A.1/Field.b.1 Task:/6 C#:/Class.A.1/Field.m.1 int C#:/Class.A.1/Method.m.1 ([], float) C#:/Class.B.1/Field.i.1 int C#:/Class.B.1/Field.f.1 float C#:/Class.B.1/Method.m.1 ([], int) C#:/Class.C.1/Method.n.1 ([], int)

Table 3.1: Initial semantic index for the C# program in listing 3.1a. the unique qualifier. u0 is useful to resolve binding prospects, as we will show later. An alias consists of the new name, that is a binding instance, and the old name, that is a binding prospect. For each alias, A contains an entry (a, u), where a is the URI of the binding instance and u is the URI of the binding prospect. For a transitive wildcard import from a scope with URI u into a scope 0 0 with URI u , TIns contains an entry (u , u). Similarly, NIns contains entries for non-transitive imports. Finally, for a binding instance of URI u and of type t, Ptype contains an entry (u, t). P can also store other properties of binding instances, but we focus on types for this paper. Example. Table 3.1 shows the index for the running example. It contains entries in B for binding instances of classes A, B, and C, fields A.b, A.m, B.i, and B.f, and methods A.m, B.m, and C.n. Corresponding entries for Ptype contain the types of all fields and methods in the program. Since the running example does not define any aliases, A does not contain any entries. It also contains corresponding entries for NIField, TIField, NIMethod, and TIMethod. These entries model inheritance by a combination of a non-transitive and a transitive import. C first inherits the fields and methods from A (non-transitive import). Second, C inherits the fields and methods which are inherited by A (transitive import). 3.3.3 Initial Collection We collect index entries in a generic top-down traversal, which needs to be instantiated with language-specific name binding and scope rules. During the

42 traversal, a dictionary S is maintained to keep track of the current scope for each namespace. At each node, we perform the following actions: 1. If the node is the context of a binding instance of name n in namespace ns, we create a new unique qualifier q, construct URIs u0 = S(ns)/ns.n and u = u0.q, and add (u0, u) to B. If the instance is of type t, we add (u, t) to 0 Ptype. If the node is a scope for a namespace ns , we update S(ns) to u. 2. If the current node is an anonymous scope for a namespace ns, we extend S(ns) with an additional anonymous segment. 3. If the current node defines an alias, transitive, or non-transitive wildcard import, we add corresponding pairs of URIs to A, TIns, or NIns. Collection does not consider binding prospects which need to be resolved. Furthermore, entries in TIns, NIns, and Ptype might still require project-wide name resolution and type analysis. Instead of performing this analysis during the collection, we defer the remaining analysis tasks to a second phase of analysis and store unique placeholder URIs in the index. For example, the type of field A.b contains a class name B, which needs to be resolved. The index in table 3.1 does not contain an actual type, but a reference to a deferred resolution task (Task:/6 in this case). Also, the index entries for wildcard imports refers to a deferred task, since the name of the base class of class C needs to be resolved first. References to tasks are created by hash consing, and therefore stay the same when the task stays the same. Partitions. The semantic index is a project-wide data structure, but collection can be split over separate partitions. A partition is typically a file, but can also be a smaller unit. The only constraint we impose on partitions is that they need to be in global scope. This ensures that index collection is independent of other partitions. Collection for a partition p will provide us with a partial index consisting of Bp, Ap, TIp,ns, NIp,ns, and Pp,type. The overall index can be formed by combining all partial indices of a project. 3.3.4 Incremental Collection When a partition is edited, reanalysis is triggered. But only the partial index of the changed partition needs to be recollected, while partial indices of other partitions remain valid. Partial recollection will result in an updated relation 0 0 0 Bp. Given the original Bp, we define a change set ∆B = (Bp \ Bp) ∪ (Bp \ Bp) of entries added to or removed from B. In the same way, we can define ∆A, and

∆Ptype . For imports, the situation is slightly different, since we need to consider changes in transitive import chains. We keep a change set ∆Ins for a derived ∗ ∗ relation Ins = TIns ◦ NIns, where TI is the reﬂexive transitive closure of TI and I is the composition of this closure with NI. Example. Table 3.2 shows non-empty change sets for the running example, where superscripts indicate editing steps. In step 1, changing the return type of method A.m causes a change in Ptype. In step 3, adding a parameter to the same method causes changes to B and Ptype. In step 4, changing the parent of class C causes changes in IField and IMethod. In step 5, enclosing class C in a

Chapter 3. A Task Engine for Incremental Name and Type Analysis 43 Change Key Value ∆1 Ptype C#:/Class.A.1/Method.m.1 ([], float) C#:/Class.A.1/Method.m.1 ([], int) 3 ∆B C#:/Class.A.1/Method.m.1/Var.b C#:/Class.A.1/Method.m.1/Var.b.1 ∆3 Ptype C#:/Class.A.1/Method.m.1/Var.b.1 Task:/6 C#:/Class.A.1/Method.m.1 ([], int) C#:/Class.A.1/Method.m.1 ([Task:/6], int) ∆4 IField C#:/Class.C.1 Task:/31 C#:/Class.C.1 Task:/6 ∆4 IMethod C#:/Class.C.1 Task:/31 C#:/Class.C.1 Task:/6 5 ∆B C#:/Ns.N C#:/Ns.N.1 C#:/Class.C C#:/Class.C.1 C#:/Ns.N.1/Class.C C#:/Ns.N.1/Class.C.1 C#:/Class.C.1/Method.n C#:/Class.C.1/Method.n.1 C#:/Ns.N.1/Class.C.1/Method.n C#:/Ns.N.1/Class.C.1/Method.n.1 ∆5 IField C#:/Class.C.1 Task:/6 C#:/Ns.N.1/Class.C.1 Task:/54 ∆5 IMethod C#:/Class.C.1 Task:/6 C#:/Ns.N.1/Class.C.1 Task:/54 ∆5 Ptype C#:/Class.C.1/Method.n.1 ([], int) C#:/Ns.N.1/Class.C.1/Method.n.1 ([], int)

Table 3.2: Changes to the semantic index of table 3.1, based on the changes to the C# program from listing 3.1b. Highlighted parts are modiﬁed or new values/entries. Striked through parts are old values or removed entries. namespace affects all index entries for the class and its contained elements. The next section discusses how change-sets trigger reevaluation of deferred analysis tasks.

3.4 Deferred Analysis Tasks

In the previous section, we discussed the collection of index entries. This collection is efficient, since it requires only a single top-down traversal. When a partition changes, recollection is even more efficient, since it can be restricted to the changed partition, while the collected entries from other partitions remain valid. This is achieved by deferring name resolution and type analysis tasks, which might require information from other partitions or from other tasks. Tasks are collected together with index entries and evaluated afterwards in a second analysis phase. For evaluation, no traversal is needed. Instead, inter- task dependencies determine an evaluation order. When a partition changes, only the tasks for this partition are recollected in the first phase. Change sets determine which tasks need to re-evaluated, including affected tasks from

44 Instruction Semantics resolve uri B [uri] resolve alias uri A [uri] resolve import ns I [uri] into uri ns

lookup type of uri Ptype [uri] check type t in T {t} ∩ T cast type t to T C [t] ∩ T assign type t {t} s1 + s2 R [s1, s2] R [s1] , if 6= ∅ s1 <+ s2 R [s2] , otherwise

filter u ∈ R [s1,s2]| P ◦ C [u] ∩ T 6= ∅ s1 + s2 by type T type filter u ∈ R [s1]| (Ptype ◦ C) [u] ∩ T 6= ∅ , if 6= ∅ s1 <+ s2 by type T u ∈ R [s2]| Ptype ◦ C [u] ∩ T 6= ∅ , otherwise disambiguate 0 0 s1 + s2 { u ∈ R [s1,s2]| ∀u ∈ R [s1,s2] : δC(u , T) ≥ δC(u, T)} by type T disambiguate { u ∈ R [s1]| ∀u0 ∈ R [s1,s2] : δ (u0, T) ≥ δ (u, T)} , if 6= ∅ s1 <+ s2 C C { u ∈ R [s2]| ∀u0 ∈ R [s1,s2] : δ (u0, T) ≥ δ (u, T)} , ow. by type T C C

Table 3.3: Syntax and semantics of name and type analysis instructions. uri denotes a URI, ns a namespace, t a type, T a set of types, and s1, s2 subtask IDs.

other partitions.

3.4.1 Instructions

Each task consists of a special URI, which is used as a placeholder in the semantic index, its dependencies to other tasks, and an instruction. Table 3.3 lists the instructions which can be used in tasks. Their semantics is given with respect to the semantic index, a type cast relation C ⊆ T × T , where (t, t0) ∈ C 0 iff type t can be cast to type t , and a partial function δC : T × T → N for the distance between types. We write R [S] to denote the image of a set S under a relation R and omit set braces for ﬁnite sets, that is, we write R [e] instead of R [{e}]. We provide three name resolution instructions for looking up binding instances from B (resolve), named imports from A (resolve alias), and wildcard imports from the derived relation Ins (resolve import), and four type analysis instructions for type look-up from Ptype (lookup), for checks with respect to expected types (check), for casts to an expected type according to C (cast), and for assigning types to program elements (assign).

Chapter 3. A Task Engine for Incremental Name and Type Analysis 45 ID Instruction Results 1 resolve C#:/Class.B C#:/Class.B.1 2 resolve alias C#:/Class.B 3 resolve Task:/2 4 resolve import Class into C#:/ 5 resolve Task:/4/Class.B 6 Task:/1 + Task:/3 + Task:/5 C#:/Class.B.1 7 assign type int int 8 resolve C#:/Class.A.1/Method.m.1/Var.b 9 resolve C#:/Class.A.1/Field.b C#:/Class.A.1/Field.b.1 10 resolve import Field into C#:/Class.A.1 11 resolve Task:/10/Field.b 12 Task:/9 <+ Task:/11 C#:/Class.A.1/Field.b.1 13 Task:/8 <+ Task:/12 C#:/Class.A.1/Field.b.1 14 lookup type of Task:/13 C#:/Class.B.1 15 resolve Task:/14/Field.f C#:/Class.B.1/Field.f.1 16 resolve import Field into Task:/14 17 resolve Task:/16/Field.f 18 Task:/15 <+ Task:/17 C#:/Class.B.1/Field.f.1 19 lookup type of Task:/18 float 20 check type Task:/7 in int {int, long, float, double, String} 21 check type Task:/19 in float {int, long, float, double, String} 22 cast type Task:/21 to Task:/20 23 cast type Task:/20 to Task:/21 float 24 Task:/22 + Task:/23 float 25 cast type Task:/24 to float float 26 cast type Task:/20 to int int 27 resolve C#:/Class.A C#:/Class.A.1 28 resolve alias C#:/Class.A 29 resolve Task:/28 30 resolve Task:/4/Class.A 31 Task:/27 + Task:/29 + Task:/30 C#:/Class.A.1 32 resolve C#:/Class.C.1/Method.m 33 resolve import Method into C#:/Class.C.1 C#:/Class.A.1 34 resolve Task:/33/Method.m C#:/Class.A.1/Method.m.1 35 assign type [] [] 36 disambiguate Task:/32 <+ Task:/34 C#:/Class.A.1/Method.m.1 by type Task:/35 37 lookup type of Task:/36 ([], float) 38 cast type Task:/37 to int

Table 3.4: Tasks and their solutions for the C# program in listing 3.1a.

46 Example. Table 3.4 shows tasks and their solutions for the running example. Tasks 1 to 6 try to resolve class name B. Task 1 looks for B directly in the global scope. It finds an entry in B and succeeds. Task 2 looks for aliases, which task 3 tries to resolve next. Instead of a concrete URI, the task 3 has a reference to task 2. Since task 2 fails to find any named imports, task 3 also fails. Task 5 tries to resolve B inside imported scopes, which are yielded by task 4. Both tasks fail. Task 6 combines resolution results based on local classes, aliases, and imported classes. We will discuss such combinators in the next example. Tasks 7 to 25 are involved in type checking the return expression of A.m() in listing 3.1a. Task 7 assigns type int to the integer constant. Tasks 8 to 18 are an example for the interaction between name and type analysis. The first six tasks try to resolve b either as a local variable, a field in the current class, or an inherited field. Next, task 14 looks up the type of the resolved field A.b, before the remaining tasks resolve field f with respect to that type B. Task 19 looks up the type of the referred field. The remaining tasks analyze the binary expression: Tasks 20 and 21 check if the subexpressions are numeric or string types. Tasks 22 and 23 try to coerce the left to the right type and vice versa. Both tasks are combined by task 24. Finally, task 25 checks if the type of the return expression can be coerced to the declared return type of the method. 3.4.2 Combinators Table 3.3 also shows six instructions to combine the results of subtasks. The semantics of these combinators are expressed in terms of a relation R, where (t, r) ∈ R iff r is a result of task t. Notably, tasks can have multiple results. We will revisit R later, when we discuss task evaluation. The simplest combinators are a non-deterministic choice + and a deterministic pendant <+. The result of the non-deterministic choice is the union of the results of its subtasks. while the result of the deterministic choice is the result of its first non-failing subtask. Furthermore, we provide combinators filter and disambiguate. Both can be used in a non-deterministic or deterministic fashion to combine the result sets of resolution tasks with respect to expected types. filter keeps only compliant results. disambiguate keeps only results which fit best with respect to the expected types. The non-deterministic variant keeps all of them, while the deterministic variant chooses the first subtask which contributes to the best fitting results. Example. In table 3.4, task 6 combines resolution results based on local classes, aliased classes, and imported classes. The non-deterministic choice ensures that no result is preferred over another. Similarly, task 24 combines the results of alternative coercion tasks. In tasks 12 and 13, deterministic choices ensure that local fields win over inherited fields and variables win over fields, respectively. Method call resolution in the presence of overloaded methods is a well- known example for interaction between name and type analysis. Actual and formal argument types need to be considered by the resolution, since they need to comply. Furthermore, relations between these types indicate which declaration is more applicable. As an example, consider tasks 32 to 36 in table 3.4. They resolve method call m() in the return expression of C.n() from

Chapter 3. A Task Engine for Incremental Name and Type Analysis 47 listing 3.1a. Task 32 tries to resolve it locally, while tasks 33 and 34 consider inherited methods. Task 35 assigns an empty list as the type of the actual parameters of the call. Task 36 selects only these methods which ﬁts this type best, preferring local over inherited methods. Finally, the last two tasks check the return expression of C.n. Task 37 looks up the type of A.m. Task 38 tries to casts this to the declared return type, but fails.

3.4.3 Initial Evaluation

During the generic traversal in the collection phase, we do not only collect semantic index entries but also instructions of tasks (T ⊆ U × I) and inter-task dependencies (D ⊆ U × U). Language-speciﬁc collection rules are needed to control the collection of name resolution and type analysis tasks. D imposes an evaluation order for tasks. First, we can evaluate independent tasks. Next, we can evaluate tasks which only depend on already evaluated tasks. This will evaluate all tasks except those with cyclic dependencies, which we consider erroneous. As mentioned earlier, we capture task results in a relation R ⊆ U × (URIs ∪ T ).

Multiple Results. The instruction of each task is evaluated according to the semantics given in table 3.3. However, this only works, if we replace place- holders of dependent subtasks with their results. When a subtask has multiple results, we evaluate the dependent task for each of these results. Consider task 14 from table 3.4 as an example. It can only be evaluated after replacing the placeholder Task:/13 with a result of the corresponding task. Since this task has a single result C#:/Class.A.1/Field.b.1, we actually need to evaluate the instruction lookup type C#:/Class.A.1/Field.b.1, yielding C#://Class.B.1 as its only result.

3.4.4 Incremental Evaluation

When a partition is edited, the partial index and tasks for this partition will be 0 recollected, resulting in an updated relation Tp. We need to evaluate new tasks, which did not exist in another partition before. We collect the URIs of these 0 tasks in a change set: ∆Tp = dom(Tp \ Tp). Furthermore, a changed semantic index might affect the results of the tasks from all partitions, requiring the reevaluation of those tasks. The various change sets determine which tasks need to be reevaluated: 0 0 (u , u) ∈ ∆B: tasks which evaluated an instruction resolve u .

(a, u) ∈ ∆A: tasks which evaluated an instruction resolve alias a. 0 0 (u , u) ∈ ∆I: tasks which evaluated an instruction resolve import u .

(u, t) ∈ ∆Ptype : tasks which evaluated an instruction lookup type of u and filter or disambiguate tasks with a subtask s with u ∈ R [s].

We maintain the URIs of these tasks in another change set ∆T. The URIs of ∗ tasks which require evaluation is given by the set ∆Tp ∪ D [∆T].

48 ID Instruction Results 39 cast type Task:/24 to int 40 resolve Task:/14/Field.i C#:/Class.B.1/Field.i.1 41 resolve Task:/16/Field.i 42 Task:/40 <+ Task:/41 C#:/Class.B.1/Field.i.1 43 lookup type of Task:/42 int 44 check type Task:/43 in int {int, long, float, double, String} 45 cast type Task:/44 to Task:/20 int 46 cast type Task:/20 to Task:/44 int 47 Task:/45 + Task:/46 int 48 cast type Task:/47 to int int 49 resolve C#:/Ns.N.1/Class.B 50 resolve alias C#:/Ns.N.1/Class.B 51 resolve Task:/50 52 resolve import Class into C#:/Ns.N.1 53 resolve Task:/52/Class.B 54 Task:/49 + Task:/51 + Task:/53 55 Task:/31 + Task:/54 C#/Class.B.1 56 resolve C#:/Ns.N.1/Class.C.1/Method.m 57 resolve import Method C#:/Class.B.1 into C#:/Ns.N.1/Class.C.1 58 resolve Task:/57/Method.m C#:/Class.B.1/Method.m.1 59 disambiguate Task:/56 + Task:/58 C#:/Class.B.1/Method.m.1 by type Task:/35 60 lookup type of Task:/59 ([], int) 61 cast type Task:/60 to int int

Table 3.5: New tasks and their solutions for the C# program in listing 3.1b.

Example. In step 1 of the running example, task 25 becomes obsolete, since the return expression needs to be checked with respect to a new type, which is done by a new task 39, shown in table 3.5. Furthermore, the disambiguation in task depends on an element in ∆1 , which is to be reevaluated. Transitive 36 Ptype dependencies trigger also the reevaluation of tasks 37 and 38. Since task 38 succeeds now, it does no longer indicate a type error in C.n. But the new task 39 fails, indicating a new type error in A.m. In step 2, tasks 15, 17 to 19, 21 to 24, and 39 become obsolete, since another ﬁeld needs to be resolved. The semantic index was not changed, and only the corresponding new tasks 40 to 48 need to be evaluated. In step 3, the additional 3 variable parameter causes changes in the semantic index. ∆B requires the reevaluation of task 8 and its dependent tasks 14, 16, and 40 to 48. Furthermore, ∆3 requires the reevaluation of task and its dependent tasks and . Ptype 36 37 38 Similarly, ∆4 requires the reevaluation of task 33 and its dependent tasks 34 IField and 36 to 38.

Chapter 3. A Task Engine for Incremental Name and Type Analysis 49 Finally, the new enclosing namespace introduced in step 5 makes tasks 32 to 34 and 36 to 38 obsolete and introduces new tasks 49 to 61, which take the new namespace into account.

3.5 Implementation We have implemented the approach as four components of the Spoofax language workbench [75]. The first component is a Java implementation of the semantic index. It maintains a multimap storing relations B, A, I, and P, a set keeping partition names, and another multimap from partitions to their index entries. During collection, it calculates change sets on the fly, maintaining two multisets for newly added and removed elements. The second component is a task engine implemented in Java. It maintains a map from task IDs to their instructions and bidirectional multimaps between task IDs and their partitions, between task IDs and index entries they depend on, and for task dependencies. Just as the semantic index, the task engine exposes a collection API and calculates change sets on the fly, maintaining a set of added and a set of removed tasks. Additionally, it exposes an API for task evaluation. During evaluation, it maintains a queue of scheduled tasks and a bidirectional multimap of task dependencies which are discovered dynamically. Results and messages of tasks are kept in maps. Both components use hash-based data structures which can be persisted to file. They support Java representations of terms as values and expose their APIs to Stratego [18], Spoofax’s term rewriting language for analysis, transformation, and code generation. The third component implements index and task collection as a generic traversal in Stratego. At each tree node, the traversal applies language-specific rewrite rules for name and type analysis. These rules can either be manually written in Stratego, or generated from meta-languages such as NaBL. The fourth component is a compiler for NaBL, which generates language- specific rules for the index and task collection traversal. Additionally, we have created a new meta-DSL for specifying simple type systems, called TS, which also compiles to the index and task collection traversal. Therefore, language developers can specify their name and type analysis in NaBL and TS, from which a collection traversal is derived, which is then used to do incremental name and type analysis with the task engine. For example, listing 3.2 shows an extract of NaBL and TS rules for a C#-like language. If a certain name or type rule cannot be expressed in NaBL or TS, a collection rule can still be manually implemented by means of the API.

3.6 Evaluation We evaluate the correctness, performance, and scalability of our approach with an implementation for name and type analysis of WebDSL programs. Correctness is interesting since we only analyze affected program elements. We expect incremental analysis to yield the same result as a full analysis. Performance and scalability are crucial since they are the main purpose of incremental

50 binding rules Class(NonPartial(),c,_,_): defines Classc scopes Field, Method Field(_,f): defines Fieldf Method(_,m,_,_): defines Methodm scopes Var

Base(c): imports Field, imported Field, Method, imported Method from Classc

ClassType(c): refers to Classc FieldAcc(e,f): refers to Fieldf in Classc wheree has typec VarRef(x): refers to Varx otherwise refers to Fieldx ThisCall(m,p*): refers to best Methodm of typet* wherep* has typet*

(a) Declarative name binding and scope rules in NaBL.

type rules Add(x,y): ty wherex: x-ty andy: y-ty and( ( x-ty ty or y-ty ty ) or ( x-ty (x-ty, y-ty) => ty ) )

(b) Declarative type rule for string concatenation or numerical addition in TS.

Listing 3.2: Declarative name and type rules for a C#-like language. analysis. We want to assess whether performance is acceptable for practical use in IDEs and how the approach scales for large projects. Speciﬁcally, we evaluate the following research questions: RQ1) Does incremental name and type analysis of WebDSL applications yield the same results as full analysis? RQ2) What is the performance gain of incremental name and type analysis of WebDSL applications compared to full analysis? RQ3) How does the size of a WebDSL application inﬂuence the performance of incremental name and type analysis? RQ4) Is incremental name and type analysis suitable for a WebDSL IDE?

3.6.1 Research method In a controlled setting, we quantitatively compare the results and performance of incremental and full analysis of different versions of WebDSL applications. We have reimplemented name and type analysis for WebDSL, using NaBL to specify name binding and scope rules and Stratego to specify type analysis. We apply the same algorithm to perform full and incremental analyses to the

Chapter 3. A Task Engine for Incremental Name and Type Analysis 51 source code histories of two WebDSL applications. We run a full analysis on all ﬁles in a revision, and and incremental analysis only on changed ﬁles with respect to the result of a full analysis of the previous revision.

Subjects. WebDSL is a domain-speciﬁc language for the implementation of dynamic web applications [49]. It was designed from the ground up with static analysis and cross-aspect consistency checking in mind [59]. This focus makes it is an ideal candidate to evaluate its static analysis. WebDSL provides many language constructs on which constraints have to be checked. It also embodies a complex expression language that is representative of expressions in general purpose languages such as Java and C#. It has been used for several applications in production, including the issue tracker YellowGrass [149], which is a subject of this evaluation, the digital library Researchr, and the online education platform WebLab. When developing such larger applications, the usability of the WebDSL IDE sometimes suffered from the lack of incremental analyses.

We focus on two open source WebDSL applications: Blog [154], a web application for wikis and blogs, and YellowGrass [148], a tag-based issue tracker. In their latest revisions, their code bases consist of approximately 7 and 9 KLOC.

Data collection. We perform measurements by repeating the following for every revision of each application. We run an incremental and a full analysis. During each of the analyses we record execution timings. After each analysis we preserve the data from the semantic index and the task engine which we analyze afterwards.

Each analysis is sequentially executed on command line in a separate invocation of the Java Virtual Machine (JVM) and garbage collection is invoked before each analysis. After starting the virtual machine, we run three analyses and discard results allowing for the warmup period of the JVM’s JIT compiler. All executions are carried out on the same machine with 2.7 Ghz Intel Core i-7, 16 GB of memory, and Oracle Java Hotspot VM version 1.6.0 45 in server JIT mode. We ﬁx the JVM’s heap size at 4 GB to decrease the noise caused by garbage collection. We set the maximum stack size at 16 MB.

Analysis procedure. For RQ1, we evaluate the structural equality of data from the semantic index and the task engine produced by full and incremental analysis. For RQ2, we determine absolute execution times of full and incremental analysis and the relative speed up. We calculate the relative performance gain between analyses separately for each revision. We report geometric mean and distribution of absolute and relative performance of all revisions. For RQ3, we determine the number of lines and the number of changed lines of a revision. We relate the incremental analysis time to these numbers. For RQ4, we ﬁlter revisions which changed only a single ﬁle. On these revisions, we determine the execution time of incremental analysis.

52 SEQNUM PARSE [FULL] COLLECT [FULL] EVALUATE [FULL] PARSE [INCR] COLLECT [INCR] EVALUATE [INCR] DELTAFILES PARSE/DELTAFILES COLLECT/ EVALUATE/ TOTAL [FULL] TOTAL [INCR] RATIO INDEX-SIZE INDEX-DIFF cov(TOTAL[INCR], cov(TOTAL[INCR],IND DELTALOC TOTALLOC TOTAL[INCR]/ full P full C full E incr P incr C incr E [INCR] DELTAFILES [INCR] DELTAFILES [INCR] INDEX-SIZE) EX-DIFF) DELTALOC 0 1030 2640 1070 1050 2470 1060 4 262.5 617.5 265 4,740 4,580 0.966244725738397 7,047 7,047 -0.167824104408355 0.689878474422275 3139 3139 1.45906339598598 1.03 2.64 1.07 1.05 2.47 1.06 15 1 1130 2560 1140 380 420 100 11 34.5454545454545 38.1818181818182 9.09090909090909 4,830 900 18.63% 7,469 424 224 3336 4.01785714285714 1.13 2.56 1.14 0.38 0.42 0.1 Parse Collect Evaluate 2 1130 2770 1170 380 370 100 9 42.2222222222222 41.1111111111111 11.1111111111111 5,070 850 16.77% 7,756 341 196 3487 4.33673469387755 1.13 2.77 1.17 0.38 0.37 0.1 12 3 1100 2770 1190 390 230 30 4 97.5 57.5 7.5 5,060 650 12.85% 7,819 135 219 3554 2.96803652968037 1.1 2.77 1.19 0.39 0.23 0.03 4 1100 2570 1170 340 30 0 1 340 30 0 4,840 370 7.64% 7,819 0 4 3554 92.5 1.1 2.57 1.17 0.34 0.03 0 9 5 1160 2790 1200 390 330 70 6 65 55 11.6666666666667 5,150 790 15.34% 7,954 163 115 3629 6.8695652173913 1.16 2.79 1.2 0.39 0.33 0.07 6 1140 2730 1340 430 590 190 10 43 59 19 5,210 1,210 23.22% 8,235 439 237 3754 5.10548523206751 1.14 2.73 1.34 0.43 0.59 0.19 6 1140 2730 1340 430 590 190 10 43 59 19 5,210 1,210 23.22% 8,235 439 237 3754 5.10548523206751 1.14 2.73 1.34 0.43 0.59 0.19 6 Time (s) 7 1150 2880 1270 400 400 70 4 100 100 17.5 5,300 870 16.42% 8,349 148 56 3799 15.5357142857143 1.15 2.88 1.27 0.4 0.4 0.07 Time (s) 8 1160 3050 1320 370 280 40 3 123.333333333333 93.3333333333333 13.3333333333333 5,530 690 12.48% 8,426 153 56 3823 12.3214285714286 1.16 3.05 1.32 0.37 0.28 0.04 8 1160 3050 1320 370 280 40 3 123.333333333333 93.3333333333333 13.3333333333333 5,530 690 12.48% 8,426 153 56 3823 12.3214285714286 1.16 3.05 1.32 0.37 0.28 0.04 3 9 1210 3190 1470 400 280 20 2 200 140 10 5,870 700 11.93% 8,479 53 28 3843 25 1.21 3.19 1.47 0.4 0.28 0.02 10 1200 3010 1410 350 100 0 1 350 100 0 5,620 450 8.01% 8,481 2 10 3849 45 1.2 3.01 1.41 0.35 0.1 0 10 1200 3010 1410 350 100 0 1 350 100 0 5,620 450 8.01% 8,481 2 10 3849 45 1.2 3.01 1.41 0.35 0.1 0 0 11 1350 2970 1380 400 290 20 4 100 72.5 5 5,700 710 12.46% 8,515 120 35 3864 20.2857142857143 1.35 2.97 1.38 0.4 0.29 0.02 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 12 1300 3180 1360 430 390 20 5 86 78 4 5,840 840 14.38% 8,564 139 37 3877 22.7027027027027 1.3 3.18 1.36 0.43 0.39 0.02 13 1220 3020 1440 390 280 40 3 130 93.3333333333333 13.3333333333333 5,680 710 12.50% 8,595 127 46 3884 15.4347826086957 1.22 3.02 1.44 0.39 0.28 0.04 Revision 14 1230 2920 1330 420 380 10 4 105 95 2.5 5,480 810 14.78% 8,645 50 49 3919 16.530612244898 1.23 2.92 1.33 0.42 0.38 0.01 15 1230 3090 1410 370 170 0 1 370 170 0 5,730 540 9.42% 8,645 0 2 3919 270 1.23 3.09 1.41 0.37 0.17 0 (a) Run time for full analysis, for each revision in the source code repository. 16 1200 3030 1410 390 250 0 4 97.5 62.5 0 5,640 640 11.35% 8,641 4 8 3919 80 1.2 3.03 1.41 0.39 0.25 0 16 1200 3030 1410 390 250 0 4 97.5 62.5 0 5,640 640 11.35% 8,641 4 8 3919 80 1.2 3.03 1.41 0.39 0.25 0 5 17 1250 3010 1330 390 260 20 2 195 130 10 5,590 670 11.99% 8,669 64 28 3927 23.9285714285714 1.25 3.01 1.33 0.39 0.26 0.02 5 17 1250 3010 1330 390 260 20 2 195 130 10 5,590 670 11.99% 8,669 64 28 3927 23.9285714285714 1.25 3.01 1.33 0.39 0.26 0.02 Parse Collect Evaluate 18 1190 2940 1400 390 260 0 2 195 130 0 5,530 650 11.75% 8,678 9 12 3931 54.1666666666667 1.19 2.94 1.4 0.39 0.26 0 Parse Collect Evaluate 4 19 1190 3050 1420 430 340 0 5 86 68 0 5,660 770 13.60% 8,702 24 19 3944 40.5263157894737 1.19 3.05 1.42 0.43 0.34 0 20 1240 3160 1440 420 370 20 3 140 123.333333333333 6.66666666666667 5,840 810 13.87% 8,709 15 10 3946 81 1.24 3.16 1.44 0.42 0.37 0.02 3 21 1200 2930 1390 400 240 10 2 200 120 5 5,520 650 11.78% 8,773 66 59 3995 11.0169491525424 1.2 2.93 1.39 0.4 0.24 0.01 22 1220 3150 1450 390 260 140 3 130 86.6666666666667 46.6666666666667 5,820 790 13.57% 8,773 70 140 3967 5.64285714285714 1.22 3.15 1.45 0.39 0.26 0.14 2

23 1220 2950 1410 510 950 90 9 56.6666666666667 105.555555555556 10 5,580 1,550 27.78% 8,995 304 79 4036 19.620253164557 1.22 2.95 1.41 0.51 0.95 0.09 Time (s)

23 1220 2950 1410 510 950 90 9 56.6666666666667 105.555555555556 10 5,580 1,550 27.78% 8,995 304 79 4036 19.620253164557 1.22 2.95 1.41 0.51 0.95 0.09 Time (s) 24 1220 3070 1460 450 540 30 4 112.5 135 7.5 5,750 1,020 17.74% 9,046 97 30 4046 34 1.22 3.07 1.46 0.45 0.54 0.03 1 25 1260 3170 1460 400 250 160 3 133.333333333333 83.3333333333333 53.3333333333333 5,890 810 13.75% 9,068 106 22 4064 36.8181818181818 1.26 3.17 1.46 0.4 0.25 0.16 1 26 1270 3140 1520 440 DELTALOC660 PARSE150 [FULL] COLLECT6 [FULL]73.3333333333333EVALUATE [FULL] PARSE [INCR]110 COLLECT [INCR]25 EVALUATE5,930 [INCR] DELTAFILES1,250 PARSE/DELTAFILES21.08% COLLECT/9,332 EVALUATE/684 TOTAL [FULL] TOTAL [INCR] RATIO 206 INDEX-SIZE4158 6.06796116504854INDEX-DIFF cov(TOTAL[INCR],1.27 cov(TOTAL[INCR],IND3.14 DELTALOC 1.52 TOTALLOC 0.44 TOTAL[INCR]/ 0.66 TOTAL [INCR] [S]0.15 p c e 0 27 1250 3220 1500 360 220 20 3 120 73.3333333333333 6.66666666666667 5,970 600 [INCR]10.05% DELTAFILES 9,339[INCR] DELTAFILES [INCR]75 13 4161 46.1538461538462 INDEX-SIZE) 1.25 EX-DIFF) 3.22 1.5 0.36 DELTALOC 0.22 0.02 0 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 28 1320 3340 1510 410 0 380 190 1980 3 5680136.666666666667 3330126.666666666667 0 63.3333333333333 0 6,170 0 980 4 15.88% 0 9,407 0 74 0 10,990 0 37 15,7574190 26.48648648648650 1.32 3.34 1.510 72380.41 0.38 0.190 0 0 20 40 60 0 80 100 1200 140 160 180 200 220 240 260 280 300 320 340 29 1320 3330 1560 430 1 450 100 1640 3 4290143.333333333333 2500 150 380 33.3333333333333 200 6,210 0 980 11 34.545454545454515.78% 18.18181818181829,465 140 0 8,430 580 6.88%58 13,4904205 16.89655172413796 1.32 3.33 1.561 59990.43 5800.45 0.580.1 0.38 0.2 0 Revision 30 1350 3310 1730 460 1 730 250 1770 6 481076.6666666666667 2570121.666666666667 330 41.6666666666667 40 6,390 10 1,440 9 36.666666666666722.54% 4.444444444444449,720 1.11111111111111485 9,150 380 4.15%189 13,6124336 7.619047619047623 1.35 3.31 1.731 61500.46 3800.73 0.380.25 0.33 0.04 0.01 31 1330 TOTALLOC3390 PARSE [FULL]1610 COLLECT [FULL]370 EVALUATE [FULL]1 250 PARSE [INCR] 10 COLLECT1840 [INCR] 3 EVALUATE4900123.333333333333 [INCR] 2670DELTAFILES83.3333333333333 PARSE/DELTAFILES320 3.33333333333333 60COLLECT/ 6,330 10EVALUATE/ 630 TOTAL4 [FULL] 9.95% TOTAL80 [INCR] 9,750 15 RATIO 362.5INDEX-SIZE 9,410INDEX-DIFF cov(TOTAL[INCR],390 cov(TOTAL[INCR],IND4.14%33 14,1714343DELTALOC19.0909090909091 TOTALLOC7 1.33TOTAL[INCR]/ TOTAL3.39 [INCR] [S] 1.611 p 65380.37 c 3900.25 e 0.390.01 (b)0.32 Run time for incremental0.06 analysis,0.01 for each revision in the source code repository 32 1350 3130 1570 410 1 340 0 1850 2 5730 205 3340 170 380 [INCR] 0DELTAFILES380 [INCR]6,050DELTAFILES0 [INCR] 750 1 12.40%380 9,750380 4 0 10,920 760INDEX-SIZE) 6.96%18 EX-DIFF) 15,5334347 41.66666666666677 1.35DELTALOC 3.13 1.571 71490.41 7600.34 0.76 0 0.38 0.38 0 33 1290 3139 3580 15901030 3702640 11070280 105060 1980 2470 2 5670 1060185 3360 4140 400 262.5 30 400 617.56,460 0 265 710 6 66.66666666666674,74010.99% 66.66666666666674,5809,8600.966244725738397 234 0 7,047 11,010 7,047 -0.167824104408355800 7.27%720.689878474422276 15,5784381 9.861111111111113139 6 3139 1.291.45906339598598 3.58 4.58 1.591 1.05 71710.37 2.47 8000.28 1.06 0.80.06 5 0.4 0.4 0 5 34 1350 3336 3440 16501130 4202560 11140380 38020 1970 420 3 5680 100140 3380126.66666666666711 43034.54545454545456.66666666666667 51038.18181818181826,440 9.090909090909090 820 10 4,83012.73% 43 9009,870 51 18.63% 50 0 7,469 11,030 424 940 8.52%42 15,6594379 19.5238095238095224 3 3336 1.354.01785714285714 3.44 0.9 1.651 0.38 72010.42 0.42 9400.38 0.1 0.940.02 0.43 Parse Collect0.51 Evaluate 0 Parse Collect Evaluate 35 1310 3487 3510 16801130 3502770 1117030 380 0 1990 370 1 6040 100350 3550 9 30 34042.2222222222222 0 41.111111111111160 6,500 11.11111111111110 380 4 5,0705.85% 85 8509,866 15 16.77% 4 0 7,756 11,580 341 400 3.45%21 16,2294378 18.0952380952381196 1 3487 1.314.33673469387755 3.51 0.85 1.681 0.38 79670.35 0.37 4000.03 0.1 0.4 0 0.34 0.06 0 35 1310 3510 1680 350 30 0 1 350 30 0 6,500 380 5.85% 9,866 4 21 4378 18.0952380952381 1.31 3.51 1.68 0.35 0.03 0 4 4 36 1310 3554 3290 15701100 3902770 11190110 39010 2300 230 2 6800 30195 3740 4 55 350 97.5 5 150 57.56,170 0 7.5 510 3 116.6666666666675,0608.27% 6509,796 50 12.85% 112 0 7,819 12,840 135 500 3.89%15 17,6734377 219 341 3554 1.312.96803652968037 3.29 0.65 1.571 0.39 87660.39 0.23 5000.11 0.03 0.50.01 0.35 0.15 0 37 1370 3554 3490 17401100 5102570 11170560 34080 2280 30 4 6860 127.50 3830 1140 350 340 20 90 306,600 10 1,1500 2 4,84017.42%175 37010,039 45 7.64% 259 5 7,819 12,970 0 450 3.47%96 18,1964460 11.97916666666674 5 3554 1.37 92.5 3.49 0.37 1.741 0.34 89710.51 0.03 4500.56 0 0.450.08 0.35 0.09 0.01 38 1340 3629 3410 16901160 5102790 21200990 390160 1230 330 6 3090 7085 1410 6165 370 26.666666666666765 170 556,440 11.66666666666670 1,660 1 5,15025.78%370 79010,315170 15.34% 586 0 7,954 5,730 163 540 9.42%193 45498,645 8.60103626943005115 0 3629 1.346.8695652173913 3.41 0.79 1.692 0.39 39190.51 0.33 2700.99 0.07 0.540.16 3 0.37 0.17 0 3 39 1380 3754 3520 17701140 5102730 21340860 43070 1410 590 7 349072.8571428571429190 1930122.85714285714310 390 43 10 320 596,670 0 191,440 4 5,21021.59%97.5 1,21010,314 80 23.22% 233 0 8,235 6,830 439 710 10.40%62 10,9644543 23.2258064516129237 0 3754 1.385.10548523206751 3.52 1.21 1.772 0.43 48380.51 0.59 3550.86 0.19 0.710.07 0.39 0.32 0 3799 1150 2880 21270 400 1500 400 3670 70 2080 4 350 100 90 100 0 17.5 5 5,300 70 870 18 16.42% 0 8,349 7,250 148 440 6.07% 11,428 56 3 3799 15.5357142857143 0.87 2 0.4 5041 0.4 220 0.07 0.44 0.35 0.09 0 40 1350 3510 1710 490 740 230 5 98 148 46 6,570 1,460 22.22% 10,385 163 45 4574 32.4444444444444 1.35 3.51 1.71 0.49 0.74 0.23 2 2 Time (s) 41 1370 3823 3590 17101160 5203050 21320990 370120 1470 280 5 3970 40104 2020 3198 390123.333333333333 24 93.333333333333350 6,670 13.33333333333330 1,630 3 5,53024.44%130 16.666666666666769010,461 12.48% 394 0 8,426 7,460 153 440 5.90%63 11,4514615 25.873015873015956 1 3823 1.3712.3214285714286 3.59 0.69 1.712 0.37 50410.52 0.28 2200.99 0.04 0.440.12 Time (s) 0.39 0.05 0 42 1380 3843 3570 17501210 5303190 210701470 40020 1570 280 7 431075.714285714285720 2320152.8571428571432 380 2.85714285714286200 260 1406,700 0 101,620 4 5,87024.18% 95 70010,481 65 11.93% 44 0 8,479 8,200 53 640 7.80%56 12,3844615 28.928571428571428 2 3843 1.38 25 3.57 0.7 1.752 0.4 55160.53 0.28 3201.07 0.02 0.640.02 0.38 0.26 0 43 1350 3849 3620 17401200 4103010 21410350 35010 1650 100 2 4460 0205 2540 1175 400 350 5 310 1006,710 0 0 770 1 5,62011.48%400 45010,485310 8.01% 4 0 8,481 8,650 2 710 8.21%6 12,9664615 128.33333333333310 0 3849 1.35 45 3.62 0.45 1.742 0.35 58430.41 0.1 3550.35 0 0.710.01 1 0.4 0.31 0 1 2 1680 4440 2510 330 50 0 4 82.5 12.5 0 8,630 380 4.40% 13,492 2 2 5999 190 0.38 0.33 0.05 0 44 1350 3864 3440 17401350 5102970 1380670 40030 290 4 127.520 167.54 1007.5 72.56,530 1,2105 5,70018.53% 71010,491 12.46% 12 8,515 120 22 4615 35 55 3864 1.3520.2857142857143 3.44 0.71 1.74 0.4 0.51 0.29 0.67 0.02 0.03 2 1760 4790 2570 330 110 0 2 165 55 0 9,120 440 4.82% 13,702 4 2 6212 220 0.44 0.33 0.11 0 45 1390 3877 3440 19301300 6103180 15601360 430460 39010 2061 5156 86 46 786,760 2,6304 5,84038.91% 84010,708 14.38% 955 8,564 139 251 4714 10.478087649402437 3877 1.3922.7027027027027 3.44 0.84 1.93 0.43 0.61 0.39 1.56 0.02 0.46 0 0 2 1740 5150 2630 360 100 10 2 180 50 5 9,520 470 4.94% 13,706 3 2 6217 235 0.47 0.36 0.1 0.01 46 1390 3884 3560 18801220 4703020 1440520 390210 280 5 4094 3104 130 42 93.33333333333336,830 13.33333333333331,200 5,68017.57% 71010,735 12.50% 33 8,595 127 75 4742 46 16 3884 1.3915.4347826086957 3.56 0.71 1.88 0.39 0.47 0.28 0.52 0.04 0.21 3139 5516 6538 7836 8835 0 2 5 10 14 23 33 46 67 103 189 2 1800 4680 2650 330 50 0 5 66 10 0 9,130 380 4.16% 14,172 0 2 6504 190 0.38 0.33 0.05 0 47 1410 3919 3700 18401230 3902920 1330300 420 0 380 3 10130 4100 105 0 956,950 2.5 690 5,4809.93% 81010,736 14.78% 1 8,645 50 6 4746 49 115 3919 1.4116.530612244898 3.7 0.81 1.84 0.42 0.39 0.38 0.3 0.01 0 3919 1230 3090 21410 370 1840 170 5200 0 2870 1 360 370 140 170 10 0 3 5,730 120 46.6666666666667540 3.333333333333339.42% 8,645 9,910 0 510 5.15% 14,496 2 4 3919 270 0.54 2 0.37 6676 0.17 255 0 0.51 0.36 0.14 LOC 0.01 ∆LOC 48 1400 3690 1820 530 810 240 9 58.8888888888889 90 26.6666666666667 6,910 1,580 22.87% 10,839 125 133 4791 11.8796992481203 1.4 3.69 1.82 0.53 0.81 0.24 3919 1200 3030 21410 390 1890 250 5050 0 2950 4 340 97.5 80 62.5 10 0 2 5,640 170 640 40 11.35% 5 8,641 9,890 4 430 4.35% 14,686 8 4 3919 80 0.64 2 0.39 6766 0.25 215 0 0.43 0.34 0.08 0.01 49 1400 3760 1880 360 80 10 1 360 80 10 7,040 450 6.39% 10,841 6 14 4791 32.1428571428571 1.4 3.76 1.88 0.36 0.08 0.01 3927 1250 3010 21330 390 1850 260 5390 20 2990 2 460 195 400 130 50 10 3 153.3333333333335,590 133.333333333333670 16.666666666666711.99% 8,669 10,230 64 910 8.90% 14,686 28 4 3927 23.9285714285714 0.67 2 0.39 6766 0.26 455 0.02 0.91 0.46 0.4 0.05 50 1420 3730 1860 430 590 70 3 143.333333333333 196.666666666667 23.3333333333333 7,010 1,090 15.55% 10,964 175 53 4838 20.5660377358491 1.42 3.73 1.86 0.43 0.59 0.07 (c) Incremental analysis time ordered by total (d) Incremental analysis time ordered by the 3931 1190 2940 21400 390 1950 260 5670 0 3160 2 360 195 80 130 10 0 9 5,530 40 8.88888888888889650 1.1111111111111111.75% 8,678 10,780 9 450 4.17% 15,083 12 3 3931 54.1666666666667 0.65 2 0.39 6948 0.26 225 0 0.45 0.36 0.08 0.01 51 1410 3490 1930 390 320 0 1 390 320 0 6,830 710 10.40% 10,964 0 2 4838 355 1.41 3.49 1.93 0.39 0.32 0 3944 1190 3050 21420 430 1900 340 5630 0 3150 5 390 86 160 68 0 0 4 5,660 97.5 770 40 13.60% 0 8,702 10,680 24 550 5.15% 15,088 19 5 3944 40.5263157894737 0.77 2 0.43 6948 0.34 275 0 0.55 LinesSEQNUM of0.39 CodeDELTAFILES (LOC). 0.16PARSE COLLECT 0 EVALUATE TOTALsize [INCR] of theDELTALOC change: TOTALLOC∆LOC. TOTAL[INCR] p c e 52 1410 3460 1920 520 870 30 5 104 174 6 6,790 1,420 20.91% 10,968 190 107 4845 13.2710280373832 1.41 3.46 1.92 0.52 0.87 0.03 [INCR] [INCR] [INCR] /DELTALOC 1.5 3946 1240 3160 21440 420 1920 370 5690 20 3370 3 380 140 360123.333333333333 6.666666666666670 3 126.6666666666675,840 810 120 13.87% 0 8,709 10,980 15 740 6.74% 15,526 10 5 3946 81 0.81 2 0.42 7148 0.37 370 0.02 0.74 0.38 0.36[INCR] [INCR] 0 [INCR] /DELTALOC 1.5 53 1380 3430 1910 450 470 10 3 150 156.666666666667 3.33333333333333 6,720 930 13.84% 10,978 34 20 4843 46.5 1.38 3.43 1.91 0.45 0.47 0.01 4 1 340 30 0 370 4 3554 92.5 0.34 0.03 0 3967 1220 3150 21450 390 1870 260 5600 140 3370 2 460 195 520 130 0 70 6 76.66666666666675,820 86.6666666666667790 13.57% 0 8,773 10,840 70 980 9.04% 15,770 140 4 3967 5.64285714285714 0.79 2 0.39 7241 0.26 490 0.14 0.98 4 0.46 1 0.52 340 300 0 370 4 3554 92.5 0.34 0.03 0 Parse Collect Evaluate 54 1410 3610 1950 690 1600 460 10 69 160 46 6,970 2,750 39.45% 11,309 667 183 4983 15.0273224043716 1.41 3.61 1.95 0.69 1.6 0.46 10 1 350 100 0 450 10 3849 45 0.35 0.1 0 1.2 3995 1200 2930 21390 400 2190 240 6420 10 3540 3 350133.333333333333 160 80 3.333333333333330 3 116.6666666666675,520 53.3333333333333650 11.78% 0 8,773 12,150 66 510 4.20% 15,969 59 4 3995 11.0169491525424 0.65 2 0.4 7760 0.24 255 0.01 0.51 10 0.35 1 0.16 350 1000 0 450 10 3849 45 0.35 0.1 0 1.2 55 1520 3720 1940 570 1090 20 6 95 181.666666666667 3.33333333333333 7,180 1,680 23.40% 11,329 24 29 4994 57.9310344827586 1.52 3.72 1.94 0.57 1.09 0.02 Figure 3.1: Benchmarking results 4036 1220 2950 21410 510 2040 950 6330 90 3430 9 34056.6666666666667 105.55555555555650 0 10 3 113.3333333333335,580 16.66666666666671,550 27.78% 0 8,995 11,800 304 390 3.31% 16,149 79 2 4036 19.620253164557 1.55 2 0.51 7822 0.95 195 0.09 0.39 15 0.34 1 0.05 370 1700 0 540 2 3919 270 0.37 0.17 0 56 1740 4080 2030 390 270 10 2 195 135 5 7,850 670 8.54% 11,328 1 9 4997 74.4444444444444 1.74 4.08 2.03 0.39 0.27 0.01 4046 1220 3070 21460 450 2160 540 6390 30 3510 4 370 112.5 200 135 0 7.5 3 123.3333333333335,750 66.66666666666671,020 17.74% 0 9,046 12,060 97 570 4.73% 16,061 30 0 4046 34 1.02 2 0.45 7836 0.54 285 0.03 0.57 35 0.37 1 0.2 350 300 0 380 21 4378 18.095238095 0.35 0.03 0 0.9 57 1700 4370 2050 630 1370 120 11 57.2727272727273 124.545454545455 10.9090909090909 8,120 2,120 26.11% 11,327 575 237 4996 8.94514767932489 1.7 4.37 2.05 0.63 1.37 0.12 4064 1260 3170 21460 400 2130 250 6380 160 3530 3 400133.333333333333 39083.3333333333333 53.33333333333330 6 66.66666666666675,890 810 65 13.75% 0 9,068 12,040 106 790 6.56% 16,055 22 6 4064 36.8181818181818 0.81 2 0.4 7836 0.25 395 0.16 0.79 49 0.4 1 0.39 360 800 10 450 14 4791 32.142857143 0.36 0.08 0.01 58 1430 3890 2100 550 1140 90 6 91.6666666666667 190 15 7,420 1,780 23.99% 11,401 438 64 5032 27.8125 1.43 3.89 2.1 0.55 1.14 0.09 4158 1270 3140 21520 440 2180 660 6300 150 3520 6 40073.3333333333333 240 110 40 25 3 133.3333333333335,930 1,250 80 13.333333333333321.08% 9,332 12,000 684 680 5.67% 16,336 206 2 4158 6.06796116504854 1.25 2 0.44 8123 0.66 340 0.15 0.68 51 0.4 1 0.24 390 0.04320 0 710 2 4838 355 0.39 0.32 0 0.6 Time (s) 59 1430 3670 2000 440 490 0 2 220 245 0 7,100 930 13.10% 11,401 0 6 5032 155 1.43 3.67 2 0.44 0.49 0 Time (s) 4161 1250 3220 21500 360 2210 220 6480 20 3620 3 340 120 12073.3333333333333 6.666666666666670 2 5,970 170 600 60 10.05% 0 9,339 12,310 75 460 3.74% 16,299 13 0 4161 46.1538461538462 0.6 2 0.36 8130 0.22 230 0.02 0.46 61 0.34 1 0.12 350 900 0 440 2 5041 220 0.35 0.09 0 60 1460 3740 2130 570 1120 20 6 95 186.666666666667 3.33333333333333 7,330 1,710 23.33% 11,431 50 37 5041 46.2162162162162 1.46 3.74 2.13 0.57 1.12 0.02 3.6.2 Results and interpretation 4190 1320 3340 21510 410 2280 380 6740 190 3770 3 350136.666666666667 140126.666666666667 63.33333333333330 2 6,170 175 980 70 15.88% 0 9,407 12,790 74 490 3.83% 17,473 37 1 4190 26.4864864864865 0.98 2 0.41 8633 0.38 245 0.19 0.49 64 0.35 1 0.14 390 500 0 440 2 5041 220 0.39 0.05 0 0.3 61 1500 3670 2080 350 90 0 1 350 90 0 7,250 440 6.07% 11,428 3 2 5041 220 1.5 3.67 2.08 0.35 0.09 0 4205 1320 3330 21560 430 2320 450 6790 100 3820 3 350143.333333333333 200 150 33.33333333333330 3 116.6666666666676,210 66.6666666666667980 15.78% 0 9,465 12,930 140 550 4.25% 17,791 58 2 4205 16.8965517241379 0.98 2 0.43 8817 0.45 275 0.1 0.55 69 0.35 1 0.2 340 400 0 380 16 5078 23.75 0.34 0.04 0 62 1460 3930 2080 550 970 0 5 110 194 0 7,470 1,520 20.35% 11,446 18 36 5041 42.2222222222222 1.46 3.93 2.08 0.55 0.97 0 4336 1350 3310 21730 460 2340 730 6990 250 3860 6 37076.6666666666667 140121.666666666667 41.66666666666670 1 6,390 370 1,440 140 22.54% 0 9,720 13,190 485 510 3.87% 18,030 189 2 4336 7.61904761904762 1.44 2 0.46 8835 0.73 255 0.25 0.51 70 0.37 1 0.14 350 1000 10 460 9 5083 51.111111111 0.35 0.1 0.01 0 63 1480 3860 1990 520 840 0 3 173.333333333333 280 0 7,330 1,360 18.55% 11,450 4 8 5041 170 1.48 3.86 1.99 0.52 0.84 0 We published the collected data and all analysis results in a public reposi- 4343 1330 3390 21610 370 2310 250 6890 10 3880 3 400123.333333333333 18083.3333333333333 3.3333333333333310 2 6,330 200 630 90 9.95% 5 9,750 13,080 36 590 4.51% 18,148 33 8 4343 19.0909090909091 0.63 2 0.37 8959 0.25 295 0.01 0.59 71 0.4 1 0.18 410 0.01350 0 760 33 5088 23.03030303 0.41 0.35 0 0 20 40 60 80 100 120 64 1470 3970 2020 390 50 0 1 390 50 0 7,460 440 5.90% 11,451 1 2 5041 220 1.47 3.97 2.02 0.39 0.05 0 4347 1350 3130 21570 410 2000 340 6570 0 3810 2 360 205 140 170 10 0 4 6,050 90 750 35 12.40% 2.5 9,750 12,380 4 510 4.12% 18,150 18 1 4347 41.6666666666667 0.75 2 0.41 8960 0.34 255 0 0.51 tory73 [0.36144], including1 0.14 instructions420 0.01280 on reproducing0 700 our29 experiments.5110 24.137931034 Since both0.42 0.28 0 65 1460 3730 2040 440 410 10 2 220 205 5 7,230 860 11.89% 11,453 10 19 5049 45.2631578947368 1.46 3.73 2.04 0.44 0.41 0.01 Revision 65 4377 1310 3290 31570 390 1660 110 4600 10 2510 2 340 195 100 55 0 5 6 56.66666666666676,170 16.6666666666667510 8.27% 0 9,796 8,770 112 440 5.02% 13,530 15 0 4377 34 0.51 3 0.39 6014 146.6666666666670.11 0.01 0.44 83 0.34 1 0.1 380 2600 0 640 2 5516 320 0.38 0.26 0 Revision 66 1480 3990 2020 430 430 10 3 143.333333333333 143.333333333333 3.33333333333333 7,490 870 11.62% 11,484 67 44 5066 19.7727272727273 1.48 3.99 2.02 0.43 0.43 0.01 66 1480 4378 3990 20201310 4303510 31680430 35010 1830 30 3 4880143.3333333333330 2650143.3333333333333 360116.6666666666673.33333333333333 70 107,490 0 0 870 7 51.42857142857146,50011.62% 38011,484 10 5.85% 67 0 9,866 9,360 4 430 4.59%44 14,1865066 19.772727272727321 1 4378 1.4818.0952380952381 3.99 0.38 2.023 0.35 65390.43 143.3333333333330.03 0.43 0 0.430.01 applications93 0.36 yield1 0.07 similar380 results,2500 we discuss10 only640 Yellowgrass15 5720 data42.666666667 here. Data0.38 0.25 0.01 67 1490 3800 2010 390 280 30 3 130 93.3333333333333 10 7,300 700 9.59% 11,476 188 56 5070 12.5 1.49 3.8 2.01 0.39 0.28 0.03 67 1490 4379 3800 20101350 3903440 31650280 42030 1920 380 3 5670 20130 340093.33333333333331 350 420 10 50 3807,300 0 20 700 5 6,4409.59% 70 82011,476 10 12.73% 188 0 9,870 10,990 50 400 3.64%56 15,7565070 42 12.53 4379 1.4919.5238095238095 3.8 0.82 2.013 0.42 72370.39 133.3333333333330.38 0.28 0.02 0.40.03 99 0.35 1 0.05 400 3100 0 710 2 5843 355 0.4 0.31 0 68 1460 3970 2040 370 180 0 2 185 90 0 7,470 550 7.36% 11,483 7 10 5074 55 1.46 3.97 2.04 0.37 0.18 0 for Blog can be found in the repository. For the future, we plan to collect data 68 1460 4381 3970 20401290 3703580 31590180 370 0 2300 280 2 6930 60185 3850 2 90 370 185 0 180 1407,470 10 30 550 5 6,4607.36% 74 71011,483 36 10.99% 7 2 9,860 13,080 234 560 4.28%10 18,0305074 72 554 4381 1.469.86111111111111 3.97 0.71 2.043 0.37 88390.37 186.6666666666670.28 0.18 0.06 0.56 0 100 0.37 1 0.18 400 0.01320 10 730 23 5842 31.739130435 0.4 0.32 0.01 69 1440 3940 2030 340 40 0 1 340 40 0 7,410 380 5.13% 11,485 2 16 5078 23.75 1.44 3.94 2.03 0.34 0.04 0 69 1440 4460 3940 20301370 3403490 4174040 510 0 1100 560 1 2570 80340 1170 4 40 340 127.5 0 30 1407,410 0 20 380 7 48.57142857142866,6005.13% 4.285714285714291,15011,485 17.42% 2 0 10,039 4,840 259 370 7.64%16 50787,819 96 23.750 4460 1.4411.9791666666667 3.94 1.15 2.034 0.51 35540.34 0.56 92.50.04 0.08 0.37 0 104 0.34 1 0.03 350 1000 0 450 4 5938 112.5 0.35 0.1 0 70 1490 3790 2060 350 100 10 1 350 100 10 7,340 460 6.27% 11,493 8 9 5083 51.1111111111111 1.49 3.79 2.06 0.35 0.1 0.01 on more WebDSL applications and on more programming languages. Our 70 1490 4543 3790 20601380 3503520 41770100 51010 1670 860 1 4250 70350 2250 6100 470 85 10 640143.3333333333337,340 11.66666666666670 460 2 6,6706.27%235 1,44011,493320 21.59% 8 0 10,314 8,170 233 1,110 13.59%9 12,1225083 51.111111111111162 2 4543 1.4923.2258064516129 3.79 1.44 2.064 0.51 53780.35 0.86 277.50.1 0.07 1.110.01 109 0.47 1 0.64 350 500 0 400 6 5998 66.666666667 0.35 0.05 0 71 1470 3990 2030 410 350 0 1 410 350 0 7,490 760 10.15% 11,492 5 33 5088 23.030303030303 1.47 3.99 2.03 0.41 0.35 0 71 1470 4549 3990 20301340 4103410 41690350 510 0 1690 990 1 4690 160410 2440 7350 35072.8571428571429 0 100141.4285714285717,490 22.85714285714290 760 4 6,44010.15%87.5 1,66011,492 25 25.78% 5 0 10,315 8,820 586 450 5.10%33 13,2985088 23.030303030303193 2 4549 1.478.60103626943005 3.99 1.66 2.034 0.51 59380.41 0.99 112.50.35 0.16 0.45 0 implementation110 0.35 1 and0.1 the380 subjects2000 are also0 open580 source.1 5999 580 0.38 0.2 0 72 1450 3950 2000 360 190 20 2 180 95 10 7,400 570 7.70% 11,492 72 33 5089 17.2727272727273 1.45 3.95 2 0.36 0.19 0.02 72 1450 4574 3950 20001350 3603510 41710190 49020 1750 740 2 4770 230180 2560 5 95 360 98 10 200 1487,400 0 46 570 10 6,5707.70% 36 1,46011,492 20 22.22% 72 0 10,385 9,080 163 560 6.17%33 13,6255089 17.272727272727345 0 4574 1.4532.4444444444444 3.95 1.46 42 0.49 61490.36 0.74 1400.19 0.23 0.560.02 111 0.36 1 0.2 330 500 0 380 2 5999 190 0.33 0.05 0 73 1500 3950 2030 420 280 0 1 420 280 0 7,480 700 9.36% 11,501 29 29 5110 24.1379310344828 1.5 3.95 2.03 0.42 0.28 0 73 1500 4615 3950 20301370 4203590 41710280 520 0 1730 990 1 4840 120420 2510 5280 330 104 0 70 1987,480 10 24 700 5 6,6709.36% 66 1,63011,501 14 24.44% 29 2 10,461 9,080 394 410 4.52%29 13,6895110 24.137931034482863 8 4615 1.525.8730158730159 3.95 1.63 2.034 0.52 62070.42 0.99 102.50.28 0.12 0.41 0 114 RQ0.331) For all1 revisions0.07 340 of both0.01100 applications,0 440 incremental3 6014 and146.66666667 full analysis0.34 0.1 0 74 1540 3850 2030 400 310 170 2 200 155 85 7,420 880 11.86% 11,483 24 54 5120 16.2962962962963 1.54 3.85 2.03 0.4 0.31 0.17 74 1540 4615 3850 20301380 4003570 41750310 530170 1770 1070 2 4350 20200 2550 7155 36075.7142857142857 85 200152.8571428571437,420 2.8571428571428610 880 3 6,70011.86%120 66.66666666666671,62011,483 3.3333333333333324.18% 24 10,481 8,670 44 570 6.57%54 13,9175120 16.296296296296356 10 4615 1.5428.9285714285714 3.85 1.62 2.034 0.53 63420.4 1.07 142.50.31 0.02 0.570.17 120 0.36 1 0.2 410 0.01290 10 710 8 6058 88.75 0.41 0.29 0.01 75 1500 4130 2140 620 1550 370 9 68.8888888888889 172.222222222222 41.1111111111111 7,770 2,540 32.69% 11,814 869 189 5240 13.4391534391534 1.5 4.13 2.14 0.62 1.55 0.37 produce structurally equal data in semantic index and task engine. This is the 75 1500 4615 4130 21401350 6203620 415501740 410370 1850 350 9 500068.888888888888910 2690172.2222222222222 340 41.1111111111111205 60 1757,770 0 2,5405 9 37.77777777777786,71032.69% 6.6666666666666777011,814 11.48% 869 0 10,485 9,540 4 400 4.19%189 14,1715240 13.43915343915346 0 4615 1.5128.333333333333 4.13 0.77 2.144 0.41 65380.62 0.35 1001.55 0.01 0.40.37 123 0.34 1 0.06 420 1100 0 530 6 6143 88.333333333 0.42 0.11 0 76 1560 4270 2290 620 1430 160 7 88.5714285714286 204.285714285714 22.8571428571429 8,120 2,210 27.22% 11,952 770 114 5298 19.3859649122807 1.56 4.27 2.29 0.62 1.43 0.16 76 1560 4615 4270 22901350 6203440 414301740 510160 1930 670 7 562088.571428571428630 3220204.2857142857144 340 22.8571428571429127.5 50 167.58,120 10 7.52,210 1 6,53027.22%340 1,21011,952 50 18.53% 77010 10,491 10,770 12 400 3.71%114 15,5725298 19.385964912280722 24 4615 1.56 55 4.27 1.21 2.294 0.51 71670.62 0.67 1001.43 0.03 0.40.16 128 0.34 1 0.05 330 0.0140 10 380 1 6150 380 0.33 0.04 0.01 77 1640 4370 2170 440 760 90 5 88 152 18 8,180 1,290 15.77% 12,107 403 130 5372 9.92307692307692 1.64 4.37 2.17 0.44 0.76 0.09 expected outcome and supports the equivalence of both analyses. 77 1640 4714 4370 21701390 4403440 41930760 61090 1970 1560 5 6090 46088 3410 10152 410 61 18 470 1568,180 0 461,290 3 136.6666666666676,76015.77% 156.6666666666672,63012,107 38.91% 403 0 10,708 11,470 955 880 7.67%130 15,7465372 9.92307692307692251 10 4714 1.6410.4780876494024 4.37 2.63 2.174 0.61 72370.44 1.56 2200.76 0.46 0.880.09 129 0.41 1 0.47 340 1100 0 450 16 6156 28.125 0.34 0.11 0 78 1540 3940 2280 450 670 30 5 90 134 6 7,760 1,150 14.82% 12,122 135 26 5378 44.2307692307692 1.54 3.94 2.28 0.45 0.67 0.03 78 1540 4742 3940 22801390 4503560 41880670 47030 1930 520 5 5880 21090 3380 5134 430 94 6 630 1047,760 60 421,150 1 6,83014.82%430 1,20012,122630 17.57% 13560 10,735 11,190 33 1,120 10.01%26 15,7665378 44.230769230769275 224 4742 1.54 16 3.94 1.2 2.284 0.47 72410.45 0.52 2800.67 0.21 1.120.03 130 RQ0.432) Figures1 30.63.1a and340 3.1b0.06130 show the20 absolute490 execution48 6182 times10.208333333 of full and0.34 0.13 0.02 79 1670 4250 2250 470 640 0 2 235 320 0 8,170 1,110 13.59% 12,122 2 4 5378 277.5 1.67 4.25 2.25 0.47 0.64 0 79 1670 4746 4250 22501410 4703700 41840640 390 0 2110 300 2 6180 0235 3430 3320 340 130 0 50 1008,170 0 1,1100 5 6,95013.59% 68 69012,122 10 9.93% 2 0 10,736 11,720 1 390 3.33%4 15,7865378 6 277.52 4746 1.67 115 4.25 0.69 2.254 0.39 76200.47 0.3 97.50.64 0 0.39 0 135 0.34 1 0.05 360 1000 10 470 2 6217 235 0.36 0.1 0.01 80 1590 4310 2290 600 1230 60 9 66.6666666666667 136.666666666667 6.66666666666667 8,190 1,890 23.08% 12,324 230 201 5491 9.40298507462687 1.59 4.31 2.29 0.6 1.23 0.06 80 1590 4791 4310 22901400 6003690 412301820 53060 2020 810 9 607066.6666666666667240 3360136.6666666666679 33058.88888888888896.66666666666667 50 908,190 26.66666666666670 1,890 3 6,91023.08%110 16.66666666666671,58012,324 22.87% 230 0 10,839 11,450 125 380 3.32%201 16,0925491 9.40298507462687133 4 4791 1.5911.8796992481203 4.31 1.58 2.294 0.53 78500.6 0.81 1.2395 0.24 0.380.06 incremental138 0.33 analyses1 0.05 330 of all revisions.700 10 Full analysis410 takes4 between6207 102.54.74 and0.33 0.07 0.01 81 1560 4050 2250 640 1620 180 7 91.4285714285714 231.428571428571 25.7142857142857 7,860 2,440 31.04% 12,402 826 113 5516 21.5929203539823 1.56 4.05 2.25 0.64 1.62 0.18 81 1560 4791 4050 22501400 6403760 416201880 360180 2160 80 7 630091.428571428571410 3500231.4285714285711 340 25.7142857142857360 70 807,860 10 102,440 10 7,04031.04% 34 45012,402 7 6.39% 826 1 10,841 11,960 6 420 3.51%113 16,2945516 21.592920353982314 18 4791 1.5632.1428571428571 4.05 0.45 2.254 0.36 80570.64 0.08 1051.62 0.01 0.420.18 140 0.34 1 0.07 330 0.01110 0 440 2 6212 220 0.33 0.11 0 82 1560 4310 2230 410 430 10 2 205 215 5 8,100 850 10.49% 12,382 96 26 5514 32.6923076923077 1.56 4.31 2.23 0.41 0.43 0.01 82 1560 4838 4310 22301420 4103730 41860430 43010 2180 590 2 6360 70205 3600 3215 340143.333333333333 5 130196.6666666666678,100 23.33333333333330 850 6 56.66666666666677,01010.49% 21.66666666666671,09012,382 15.55% 96 0 10,964 12,140 175 470 3.87%26 16,3215514 32.692307692307753 0 4838 1.5620.5660377358491 4.31 1.09 2.234 0.43 81090.41 0.59 117.50.43 0.07 0.470.01 143 0.34 1 0.13 440 3300 0 770 16 6242 48.125 0.44 0.33 0 83 1570 4310 2320 380 260 0 1 380 260 0 8,200 640 7.80% 12,384 2 2 5516 320 1.57 4.31 2.32 0.38 0.26 0 83 1570 4838 4310 23201410 3803490 41930260 390 0 1860 320 1 6590 0380 3840 1260 460 390 0 320 3208,200 0 0 640 2 6,8307.80%230 71012,384160 10.40% 2 0 10,964 12,290 0 780 6.35%2 17,4785516 2 3205 4838 1.57 355 4.31 0.71 2.324 0.39 86350.38 0.32 1950.26 0 0.78 0 144 0.46 1 0.32 370 2700 40 680 27 6245 25.185185185 0.37 0.27 0.04 84 1560 4230 2280 570 41090 100 1880 3 6710 190 3810363.333333333333 330 33.3333333333333 70 8,070 10 1,760 11 21.81% 30 6.3636363636363612,379 0.909090909090909379 12,400 410 3.31%58 17,7605532 30.34482758620691 1.56 4.23 2.284 87650.57 102.51.09 0.410.1 Chapter0.33 3. A Task0.07 Engine for Incremental0.01 Name and Type Analysis 53 84 1560 4843 4230 22801380 5703430 10901910 450100 470 3 10190 363.3333333333335 33.333333333333390 948,070 1,7602 6,72021.81% 93012,379 13.84% 379 10,978 34 58 5532 30.344827586206920 4843 1.56 46.5 4.23 0.93 2.28 0.45 0.57 0.47 1.09 0.01 0.1 146 1 360 110 0 470 8 6248 58.75 0.36 0.11 0 85 1570 4290 2300 760 42100 580 2290 15 683050.6666666666667 3800 140 380 38.6666666666667 380 8,160 0 3,440 6 63.333333333333342.16% 63.333333333333312,428 2,555 0 12,920 760 5.88%1102 17,7585561 3.121597096188752 1.57 4.29 2.34 87650.76 1902.1 0.760.58 0.38 0.38 0 85 1570 4845 4290 23001410 7603460 21001920 520580 87015 50.666666666666730 3140 173.33333333333338.6666666666667 2908,160 103,440 6,79042.16% 1,42012,428 20.91%2,555 10,968 190 1102 5561 3.12159709618875107 4845 1.5713.2710280373832 4.29 1.42 2.3 0.52 0.76 0.87 2.1 0.03 0.58 149 1 360 200 10 570 10 6249 57 0.36 0.2 0.01 86 1660 4240 2280 360 4 240 40 2290 3 6860 120 3820 80 370 13.3333333333333 90 8,180 0 640 2 7.82%185 12,578 45 202 0 12,970 460 3.55%93 17,7585615 6.881720430107534 1.66 4.24 2.284 87650.36 1150.24 0.460.04 0.37 0.09 0 86 1660 4983 4240 22801410 3603610 1950240 69040 1600 3 460120 10 80 13.333333333333369 1608,180 46 640 6,9707.82% 2,75012,578 39.45% 202 11,309 667 93 5615 6.88172043010753183 4983 1.6615.0273224043716 4.24 2.75 2.28 0.69 0.36 1.6 0.24 0.46 0.04 150 1 370 270 50 690 24 6259 28.75 0.37 0.27 0.05 87 1590 4140 2390 550 51130 80 2140 8 6340 68.75 3540 141.25 320 10 60 8,120 10 1,760 6 53.333333333333321.67% 12,701 10 1.66666666666667401 12,020 390 3.24%133 16,0965640 13.23308270676695 1.59 4.14 2.395 78500.55 1.1378 0.390.08 0.32 0.06 0.01 87 1590 4994 4140 23901520 5503720 11301940 57080 1090 8 68.7520 141.256 95 10 181.6666666666678,120 3.333333333333331,760 7,18021.67% 1,68012,701 23.40% 401 11,329 24 133 5640 13.233082706766929 4994 1.5957.9310344827586 4.14 1.68 2.39 0.57 0.55 1.09 1.13 0.02 0.08 155 1 350 220 10 580 9 6338 64.444444444 0.35 0.22 0.01 88 1560 4996 4390 24101700 3704370 52050100 63010 2400 1370 2 6630 120185 3630 2 50 400 315 5 270 6858,360 10 60 480 1 8,1205.74%400 2,12012,729270 26.11% 3410 11,327 12,660 575 680 5.37%39 16,2035655 12.3076923076923237 11 4996 1.568.94514767932489 4.39 2.12 2.415 0.63 79600.37 1.37 1360.1 0.12 0.680.01 0.4 0.27 0.01 88 1560 4390 2410 370 100 10 2 185 50 5 8,360 480 5.74% 12,729 34 39 5655 12.3076923076923 1.56 4.39 2.41 0.37 0.1 0.01 156 1 360 200 10 570 4 6342 142.5 0.36 0.2 0.01 89 1560 4997 4460 23601740 5104080 52030710 390 0 1970 270 5 6520 10102 3740 11142 39035.4545454545455 0 40024.54545454545458,380 0.9090909090909090 1,220 5 7,85014.56% 78 67012,732 80 8.54% 3 0 11,328 12,230 1 790 6.46%37 17,0445658 32.9729729729739 1 4997 1.5674.4444444444444 4.46 0.67 2.365 0.39 84600.51 0.27 1580.71 0.01 0.79 0 0.39 0.4 0 89 1560 4460 2360 510 710 0 5 102 142 0 8,380 1,220 14.56% 12,732 3 37 5658 32.972972972973 1.56 4.46 2.36 0.51 0.71 0 157 1 360 200 10 570 14 6346 40.714285714 0.36 0.2 0.01 90 1610 5032 4370 23701430 4303890 52100380 55040 1860 1140 2 6780 90215 3830 6190 39091.6666666666667 20 250 1908,350 0 15 850 3 7,42010.18%130 83.33333333333331,78012,761 23.99% 173 0 11,401 12,470 438 640 5.13%58 18,0325708 14.655172413793164 2 5032 1.61 27.8125 4.37 1.78 2.375 0.55 88380.43 1.14 1280.38 0.09 0.640.04 0.39 0.25 0 90 1610 4370 2370 430 380 40 2 215 190 20 8,350 850 10.18% 12,761 173 58 5708 14.6551724137931 1.61 4.37 2.37 0.43 0.38 0.04 160 1 350 70 180 600 14 6466 42.857142857 0.35 0.07 0.18 91 1610 5032 4140 23301430 4803670 52000440 440 0 1990 490 2 6710 0240 3820 2220 370 220 0 150 2458,080 0 0 920 1 7,10011.39%370 93012,764150 13.10% 9 0 11,401 12,520 0 520 4.15%6 18,1915712 153.3333333333336 3 5032 1.61 155 4.14 0.93 2.335 0.44 89700.48 0.49 1040.44 0 0.52 0 0.37 0.15 0 91 1610 4140 2330 480 440 0 2 240 220 0 8,080 920 11.39% 12,764 9 6 5712 153.333333333333 1.61 4.14 2.33 0.48 0.44 0 161 1 350 70 170 590 17 6483 34.705882353 0.35 0.07 0.17 92 1660 5041 4840 23601460 5003740 62130790 57060 1350 1120 5 3620 20100 1740 6158 410 95 12 350186.6666666666678,860 3.3333333333333310 1,350 2 7,33015.24%205 1,71012,827175 23.33% 187 5 11,431 6,710 50 770 11.48%39 10,4855715 34.615384615384637 4 5041 1.6646.2162162162162 4.84 1.71 2.366 0.57 46150.5 128.3333333333331.12 0.79 0.02 0.770.06 0.41 0.35 0.01 92 1660 4840 2360 500 790 60 5 100 158 12 8,860 1,350 15.24% 12,827 187 39 5715 34.6153846153846 1.66 4.84 2.36 0.5 0.79 0.06 163 1 360 70 0 430 3 6539 143.33333333 0.36 0.07 0 93 1610 5041 4290 23301500 3803670 62080250 35010 1410 90 1 3700 0380 1840 1250 390 350 10 300 908,230 0 0 640 3 7,2507.78%130 44012,832100 6.07% 15 0 11,428 6,950 3 690 9.93%15 10,7365720 42.66666666666672 1 5041 1.61 220 4.29 0.44 2.336 0.35 47460.38 0.09 1150.25 0 0.690.01 0.39 0.3 0 93 1610 4290 2330 380 250 10 1 380 250 10 8,230 640 7.78% 12,832 15 15 5720 42.6666666666667 1.61 4.29 2.33 0.38 0.25 0.01 165 1 340 60 0 400 4 6538 100 0.34 0.06 0 94 1580 5041 4360 23801460 5603930 62080630 55080 1430 970 4 3670 0140 2000 157.55 440 110 20 490 1948,320 0 1,2700 3 146.6666666666677,47015.26% 163.3333333333331,52012,923 20.35% 257 0 11,446 7,100 18 930 13.10%67 11,4015752 18.95522388059736 0 5041 1.5842.2222222222222 4.36 1.52 2.386 0.55 50320.56 0.97 1550.63 0 0.930.08 0.44 0.49 0 94 1580 4360 2380 560 630 80 4 140 157.5 20 8,320 1,270 15.26% 12,923 257 67 5752 18.955223880597 1.58 4.36 2.38 0.56 0.63 0.08 167 1 320 60 10 390 1 6538 390 0.32 0.06 0.01 95 1750 5041 4480 23301480 5103860 61990650 520240 1610 840 5 4140 0102 2330 3130 480173.333333333333 48 440 2808,560 0 1,4000 2 7,33016.36%240 1,36012,990220 18.55% 309 0 11,450 8,080 4 920 11.39%109 12,7645781 12.84403669724778 9 5041 1.75 170 4.48 1.36 2.336 0.52 57120.51 153.3333333333330.84 0.65 0 0.920.24 0.48 0.44 0 95 1750 4480 2330 510 650 240 5 102 130 48 8,560 1,400 16.36% 12,990 309 109 5781 12.8440366972477 1.75 4.48 2.33 0.51 0.65 0.24 168 1 350 90 0 440 36 6504 12.222222222 0.35 0.09 0 96 1620 5041 4230 23101470 6703970 615302020 39010 1680 5014 459047.85714285714290 2600109.2857142857141 3500.714285714285714390 50 508,160 0 2,2100 1 7,46027.08%350 44012,927 50 5.90% 169 0 11,451 8,870 1 400 4.51%78 13,4845797 28.33333333333332 2 5041 1.62 220 4.23 0.44 2.316 0.39 59980.67 66.66666666666670.05 1.53 0 0.40.01 0.35 0.05 0 96 1620 4230 2310 670 1530 10 14 47.8571428571429 109.285714285714 0.714285714285714 8,160 2,210 27.08% 12,927 169 78 5797 28.3333333333333 1.62 4.23 2.31 0.67 1.53 0.01 169 1 330 50 0 380 2 6504 190 0.33 0.05 0 97 1740 5049 4390 23701460 6603730 616002040 440170 1720 41012 4770 1055 2590133.3333333333332 420 14.1666666666667220 110 2058,500 0 2,4305 1 7,23028.59%420 86012,952110 11.89% 533 0 11,453 9,080 10 530 5.84%100 13,6225837 19 24.30 5049 1.7445.2631578947368 4.39 0.86 2.376 0.44 61430.66 88.33333333333330.41 1.6 0.01 0.530.17 0.42 0.11 0 97 1740 4390 2370 660 1600 170 12 55 133.333333333333 14.1666666666667 8,500 2,430 28.59% 12,952 533 100 5837 24.3 1.74 4.39 2.37 0.66 1.6 0.17 176 1 380 280 10 670 7 6676 95.714285714 0.38 0.28 0.01 98 1650 5066 4320 23301480 4603990 62020570 430 0 1970 430 4 6040 10115 3420 142.53 350143.333333333333 0 143.33333333333360 8,300 3.333333333333330 1,030 1 7,49012.41%350 87012,966 60 11.62% 16 0 11,484 11,430 67 410 3.59%14 16,1115843 73.571428571428644 1 5066 1.6519.7727272727273 4.32 0.87 2.336 0.43 78030.46 68.33333333333330.43 0.57 0.01 0.41 0 0.35 0.06 0 98 1650 4320 2330 460 570 0 4 115 142.5 0 8,300 1,030 12.41% 12,966 16 14 5843 73.5714285714286 1.65 4.32 2.33 0.46 0.57 0 177 1 360 140 10 510 2 6676 255 0.36 0.14 0.01 99 1650 5070 4460 25401490 4003800 62010310 390 0 2170 280 1 6220 30400 3440 3310 340 130 0 93.333333333333380 8,650 0 10 710 2 7,3008.21%170 70012,966 40 9.59% 0 0 11,476 11,830 188 420 3.55%2 16,1235843 56 3551 5070 1.65 12.5 4.46 0.7 2.546 0.39 79200.4 0.28 0.3170 0.03 0.42 0 0.34 0.08 0 99 1650 4460 2540 400 310 0 1 400 310 0 8,650 710 8.21% 12,966 0 2 5843 355 1.65 4.46 2.54 0.4 0.31 0 180 1 340 80 10 430 2 6766 215 0.34 0.08 0.01 100 1640 5074 4240 23701460 4003970 62040320 37010 2230 180 1 6680 0400 3540 2320 390 185 10 400 908,250 0 0 730 1 7,4708.85%390 55012,973400 7.36% 7 0 11,483 12,450 7 790 6.35%23 16,2995842 31.739130434782610 1 5074 1.64 55 4.24 0.55 2.376 0.37 81300.4 131.6666666666670.18 0.32 0 0.790.01 0.39 0.4 0 100 1640 4240 2370 400 320 10 1 400 320 10 8,250 730 8.85% 12,973 7 23 5842 31.7391304347826 1.64 4.24 2.37 0.4 0.32 0.01 181 1 460 400 50 910 2 6766 455 0.46 0.4 0.05 101 1650 5078 4520 23501440 5203940 611902030 340180 2050 40 7 667074.28571428571430 3800 1170 390 25.7142857142857340 230 408,520 0 1,8900 2 7,41022.18%195 38013,134115 5.13% 699 0 11,485 12,520 2 620 4.95%94 18,0305904 20.106382978723416 1 5078 1.65 23.75 4.52 0.38 2.356 0.34 88350.52 103.3333333333330.04 1.19 0 0.620.18 0.39 0.23 0 101 1650 4520 2350 520 1190 180 7 74.2857142857143 170 25.7142857142857 8,520 1,890 22.18% 13,134 699 94 5904 20.1063829787234 1.65 4.52 2.35 0.52 1.19 0.18 185 1 360 80 10 450 2 6948 225 0.36 0.08 0.01 102 1650 5083 4450 24601490 5403790 710302060 35060 1850 10011 519049.090909090909110 287093.63636363636361 380 5.45454545454545350 280 1008,560 10 101,630 9 42.22222222222227,34019.04% 31.111111111111146013,280 1.111111111111116.27% 246 11,493 9,910 8 670 6.76%103 14,4965928 15.82524271844669 13 5083 1.6551.1111111111111 4.45 0.46 2.467 0.35 66760.54 95.71428571428570.1 1.03 0.01 0.670.06 0.38 0.28 0.01 102 1650 4450 2460 540 1030 60 11 49.0909090909091 93.6363636363636 5.45454545454545 8,560 1,630 19.04% 13,280 246 103 5928 15.8252427184466 1.65 4.45 2.46 0.54 1.03 0.06 186 1 390 160 0 550 2 6948 275 0.39 0.16 0 103 1660 5088 4520 24401470 3803990 72030300 410 0 1950 350 4 5740 095 3240 1 75 390 410 0 470 3508,620 60 0 680 7 55.71428571428577,4907.89% 67.142857142857176013,298 8.5714285714285710.15% 32 11,492 10,930 5 920 8.42%24 15,4295938 28.333333333333333 152 5088 1.6623.030303030303 4.52 0.76 2.447 0.41 71160.38 131.4285714285710.35 0.3 0 0.92 0 0.39 0.47 0.06 103 1660 4520 2440 380 300 0 4 95 75 0 8,620 680 7.89% 13,298 32 24 5938 28.3333333333333 1.66 4.52 2.44 0.38 0.3 0 189 1 370 200 20 590 34 6998 17.352941176 0.37 0.2 0.02 104 1690 5089 4690 24401450 3503950 72000100 360 0 2150 190 1 6330 20350 3470 2100 330 180 0 90 958,820 0 10 450 5 7,4005.10% 66 57013,298 18 7.70% 2 0 11,492 11,950 72 420 3.51%4 16,2285938 33 112.53 5089 1.6917.2727272727273 4.69 0.57 2.447 0.36 79660.35 0.19 600.1 0.02 0.42 0 0.33 0.09 0 104 1690 4690 2440 350 100 0 1 350 100 0 8,820 450 5.10% 13,298 2 4 5938 112.5 1.69 4.69 2.44 0.35 0.1 0 191 1 370 410 70 850 22 7030 38.636363636 0.37 0.41 0.07 105 1740 5110 4580 25401500 4303950 72030560 420240 2290 280 3 6920143.3333333333330 3850186.6666666666671 420 420 80 180 2808,860 20 1,2300 5 7,48013.88% 84 70013,369 36 9.36% 253 4 11,501 13,060 29 620 4.75%27 18,1485965 45.555555555555629 10 5110 1.7424.1379310344828 4.58 0.7 2.547 0.42 89590.43 88.57142857142860.28 0.56 0 0.620.24 0.42 0.18 0.02 105 1740 4580 2540 430 560 240 3 143.333333333333 186.666666666667 80 8,860 1,230 13.88% 13,369 253 27 5965 45.5555555555556 1.74 4.58 2.54 0.43 0.56 0.24 195 1 390 470 60 920 7 7116 131.42857143 0.39 0.47 0.06 5120 1540 3850 72030 400 2300 310 6900 170 3800 2 360 200 150 155 10 85 2 7,420 180 880 75 11.86% 5 11,483 13,000 24 520 4.00% 18,149 54 27 5120 16.2962962962963 0.88 7 0.4 8960 74.28571428571430.31 0.17 0.52 0.36 0.15 0.01 106 1680 4660 2560 360 310 40 4 90 77.5 10 8,900 710 7.98% 13,448 139 33 5984 21.5151515151515 1.68 4.66 2.56 0.36 0.31 0.04 196 1 390 410 10 810 45 7119 18 0.39 0.41 0.01 5240 1500 4130 82140 620 1200 1550 3030 370 1410 9 39068.8888888888889 250172.222222222222 41.11111111111110 9 43.33333333333337,770 27.77777777777782,540 32.69% 0 11,814 5,640 869 640 11.35% 8,641 189 4 5240 13.4391534391534 2.54 8 0.62 3919 1.55 80 0.37 0.64 196 0.39 1 0.25 390 4100 10 810 45 7119 18 0.39 0.41 0.01 107 1710 4800 2460 420 480 0 3 140 160 0 8,970 900 10.03% 13,481 33 19 5997 47.3684210526316 1.71 4.8 2.46 0.42 0.48 0 197 1 410 450 30 890 24 7141 37.083333333 0.41 0.45 0.03 5298 1560 4270 82290 620 1480 1430 3860 160 1990 7 52088.5714285714286 840204.285714285714 22.85714285714290 7 74.28571428571438,120 2,210 120 27.22% 0 11,952 7,330 770 1,360 18.55% 11,450 114 4 5298 19.3859649122807 2.21 8 0.62 5041 1.43 170 0.16 1.36 197 0.52 1 0.84 410 4500 30 890 24 7141 37.083333333 0.41 0.45 0.03 108 1680 4630 2460 480 640 0 3 160 213.333333333333 0 8,770 1,120 12.77% 13,482 17 67 5998 16.7164179104478 1.68 4.63 2.46 0.48 0.64 0 199 1 380 360 0 740 2 7148 370 0.38 0.36 0 5372 1640 4370 82170 440 1870 760 5020 90 2530 5 410 88 290 152 10 18 2 8,180 205 1,290 145 15.77% 5 12,107 9,420 403 710 7.54% 13,519 130 2 5372 9.92307692307692 1.29 8 0.44 6058 0.76 88.75 0.09 0.71 199 0.41 1 0.29 380 0.01360 0 740 2 7148 370 0.38 0.36 0 109 1680 4590 2600 350 50 0 1 350 50 0 8,870 400 4.51% 13,484 2 6 5998 66.6666666666667 1.68 4.59 2.6 0.35 0.05 0 200 1 380 380 0 760 1 7149 760 0.38 0.38 0 5378 1540 3940 82280 450 1790 670 4850 30 2660 5 360 90 110 134 0 6 1 7,760 360 1,150 110 14.82% 0 12,122 9,300 135 470 5.05% 13,712 26 6 5378 44.2307692307692 1.15 8 0.45 6248 0.67 58.75 0.03 0.47 200 0.36 1 0.11 380 3800 0 760 1 7149 760 0.38 0.38 0 110 1640 4290 2500 380 200 0 1 380 200 0 8,430 580 6.88% 13,490 6 1 5999 580 1.64 4.29 2.5 0.38 0.2 0 201 1 410 520 60 990 14 7163 70.714285714 0.41 0.52 0.06 5378 1670 4250 82250 470 1890 640 5410 0 2960 2 360 235 300 320 0 0 3 8,170 120 1,110 100 13.59% 0 12,122 10,260 2 660 6.43% 14,876 4 4 5378 277.5 1.11 8 0.47 6874 0.64 82.5 0 0.66 201 0.36 1 0.3 410 5200 60 990 14 7163 70.714285714 0.41 0.52 0.06 111 1680 4440 2510 330 50 0 1 330 50 0 8,630 380 4.40% 13,492 2 2 5999 190 1.68 4.44 2.51 0.33 0.05 0 202 1 340 50 10 400 4 7167 100 0.34 0.05 0.01 5491 1590 4310 82290 600 1970 1230 6070 60 3420 9 33066.6666666666667 136.66666666666760 6.666666666666670 15 8,190 22 1,890 4 23.08% 0 12,324 11,460 230 390 3.40% 15,757 201 9 5491 9.40298507462687 1.89 8 0.6 7238 1.23 48.75 0.06 0.39 202 0.33 1 0.06 340 500 10 400 4 7167 100 0.34 0.05 0.01 112 1670 4360 2530 410 410 30 2 205 205 15 8,560 850 9.93% 13,512 50 13 6010 65.3846153846154 1.67 4.36 2.53 0.41 0.41 0.03 204 1 400 400 0 800 1 7171 800 0.4 0.4 0 5514 1560 4310 82230 410 1990 430 6050 10 3370 7 36058.5714285714286 18061.4285714285714 1.428571428571430 3 8,100 120 850 60 10.49% 0 12,382 11,410 96 540 4.73% 16,075 26 4 5514 32.6923076923077 0.85 8 0.41 7845 0.43 67.5 0.01 0.54 204 0.36 1 0.18 400 4000 0 800 1 7171 800 0.4 0.4 0 113 1690 4700 2420 440 490 10 3 146.666666666667 163.333333333333 3.33333333333333 8,810 940 10.67% 13,530 26 9 6017 104.444444444444 1.69 4.7 2.42 0.44 0.49 0.01 206 1 400 390 0 790 139 7198 5.6834532374 0.4 0.39 0 5516 1560 4050 82250 640 2030 1620 6390 180 3700 2 400 320 210 810 0 90 8 7,860 50 2,440 26.25 31.04% 0 12,402 12,120 826 610 5.03% 17,050 113 6 5516 21.5929203539823 2.44 8 0.64 8460 1.62 76.25 0.18 0.61 206 0.4 1 0.21 400 3900 0 790 139 7198 5.6834532374 0.4 0.39 0 114 1660 4600 2510 340 100 0 1 340 100 0 8,770 440 5.02% 13,530 0 3 6014 146.666666666667 1.66 4.6 2.51 0.34 0.1 0 208 1 430 510 0 940 1 7201 940 0.43 0.51 0 5516 1570 4310 82320 380 2070 260 6570 0 3760 1 360 380 140 260 10 0 2 8,200 180 640 70 7.80% 5 12,384 12,400 2 510 4.11% 17,464 2 12 5516 320 0.64 8 0.38 8632 0.26 63.75 0 0.51 208 0.36 1 0.14 430 0.01510 0 940 1 7201 940 0.43 0.51 0 115 1680 4370 2490 520 690 30 5 104 138 6 8,540 1,240 14.52% 13,529 63 30 6022 41.3333333333333 1.68 4.37 2.49 0.52 0.69 0.03 209 1 360 230 40 630 32 7221 19.6875 0.36 0.23 0.04 5532 1560 4230 82280 570 2270 1090 6730 100 3750 3 380 190 200363.333333333333 33.33333333333330 5 8,070 76 1,760 40 21.81% 0 12,379 12,750 379 580 4.55% 17,479 58 7 5532 30.3448275862069 1.76 8 0.57 8634 1.09 72.5 0.1 0.58 209 0.38 1 0.2 360 2300 40 630 32 7221 19.6875 0.36 0.23 0.04 116 1730 4320 2610 410 460 60 3 136.666666666667 153.333333333333 20 8,660 930 10.74% 13,567 72 31 6042 30 1.73 4.32 2.61 0.41 0.46 0.06 5561 1570 4290 82300 760 2010 2100 6690 580 3710 15 32050.6666666666667 60 140 38.66666666666670 2 8,160 160 3,440 30 42.16% 0 12,428 12,410 2,555 380 3.06% 17,737 1102 8 5561 3.12159709618875 3.44 8 0.76 8738 2.1 47.5 0.58 0.38 211 0.32 1 0.06 430 6300 60 1,120 4 7241 280 0.43 0.63 0.06 117 1690 4510 2570 350 110 0 2 175 55 0 8,770 460 5.25% 13,565 8 10 6042 46 1.69 4.51 2.57 0.35 0.11 0 5615 1660 4240 92280 360 1740 240 4080 40 2030 3 390 120 270 80 13.333333333333310 2 8,180 195 640 135 7.82% 5 12,578 7,850 202 670 8.54% 11,328 93 1 5615 6.88172043010753 0.64 9 0.36 4997 74.44444444444440.24 0.04 0.67 212 0.39 1 0.27 460 0.01520 0 980 2 7241 490 0.46 0.52 0 118 1720 4770 2540 410 630 110 2 205 315 55 9,030 1,150 12.74% 13,587 380 72 6048 15.9722222222222 1.72 4.77 2.54 0.41 0.63 0.11 5640 1590 4140 92390 550 1490 1130 3790 80 2060 8 350 68.75 100 141.25 10 10 5 8,120 70 1,760 20 21.67% 2 12,701 7,340 401 460 6.27% 11,493 133 8 5640 13.2330827067669 1.76 9 0.55 5083 51.11111111111111.13 0.08 0.46 213 0.35 1 0.1 340 0.0170 0 410 19 7244 21.578947368 0.34 0.07 0 119 1730 4760 2630 430 450 70 2 215 225 35 9,120 950 10.42% 13,519 328 46 6062 20.6521739130435 1.73 4.76 2.63 0.43 0.45 0.07 5655 1560 4390 92410 370 1690 100 4700 10 2420 2 440 185 490 50 10 5 1 8,360 440 480 490 5.74% 10 12,729 8,810 34 940 10.67% 13,530 39 26 5655 12.3076923076923 0.48 9 0.37 6017 104.4444444444440.1 0.01 0.94 214 0.44 1 0.49 330 0.0160 0 390 8 7238 48.75 0.33 0.06 0 120 1870 5020 2530 410 290 10 1 410 290 10 9,420 710 7.54% 13,519 2 8 6058 88.75 1.87 5.02 2.53 0.41 0.29 0.01 5658 1560 4460 92360 510 1790 710 4800 0 2540 5 350 102 220 142 10 0 4 8,380 87.5 1,220 55 14.56% 2.5 12,732 9,130 3 580 6.35% 13,907 37 48 5658 32.972972972973 1.22 9 0.51 6338 64.44444444444440.71 0 0.58 216 0.35 1 0.22 350 0.0150 0 400 3 7237 133.33333333 0.35 0.05 0 121 1750 4740 2610 620 1380 100 7 88.5714285714286 197.142857142857 14.2857142857143 9,100 2,100 23.08% 13,553 430 161 6061 13.0434782608696 1.75 4.74 2.61 0.62 1.38 0.1 5708 1610 4370 92370 430 2120 380 7000 40 3690 2 350 215 110 190 0 20 5 8,350 70 850 22 10.18% 0 12,761 12,810 173 460 3.59% 16,233 58 2 5708 14.6551724137931 0.85 9 0.43 8003 51.11111111111110.38 0.04 0.46 221 0.35 1 0.11 340 900 10 440 58 7377 7.5862068966 0.34 0.09 0.01 122 1860 4670 2590 380 430 70 5 76 86 14 9,120 880 9.65% 13,622 293 134 6141 6.56716417910448 1.86 4.67 2.59 0.38 0.43 0.07 5712 1610 4140 102330 480 1200 440 3010 0 1410 2 350 240 100 220 0 0 14 8,080 25 7.14285714285714920 11.39% 0 12,764 5,620 9 450 8.01% 8,481 6 2 5712 153.333333333333 0.92 10 0.48 3849 0.44 45 0 0.45 224 0.35 1 0.1 340 600 10 410 70 6883 5.8571428571 0.34 0.06 0.01 123 1720 4770 2590 420 110 0 1 420 110 0 9,080 530 5.84% 13,622 0 6 6143 88.3333333333333 1.72 4.77 2.59 0.42 0.11 0 5715 1660 4840 102360 500 1240 790 3160 60 1440 5 420 100 370 158 20 12 12 8,860 35 30.83333333333331,350 1.6666666666666715.24% 12,827 5,840 187 810 13.87% 8,709 39 15 5715 34.6153846153846 1.35 10 0.5 3946 0.79 81 0.06 0.81 225 0.42 1 0.37 340 0.0250 0 390 24 6885 16.25 0.34 0.05 0 124 1750 4670 2520 360 170 0 2 180 85 0 8,940 530 5.93% 13,625 3 81 6149 6.54320987654321 1.75 4.67 2.52 0.36 0.17 0 5720 1610 4290 102330 380 1460 250 3970 10 2040 1 370 380 180 250 0 10 4 8,230 92.5 640 45 7.78% 0 12,832 7,470 15 550 7.36% 11,483 15 7 5720 42.6666666666667 0.64 10 0.38 5074 0.25 55 0.01 0.55 226 0.37 1 0.18 370 1300 10 510 34 6885 15 0.37 0.13 0.01 125 1750 4770 2560 360 200 0 2 180 100 0 9,080 560 6.17% 13,625 0 4 6149 140 1.75 4.77 2.56 0.36 0.2 0 5752 1580 4360 102380 560 1690 630 4510 80 2570 4 350 140 110 157.5 0 20 1 8,320 350 1,270 110 15.26% 0 12,923 8,770 257 460 5.25% 13,565 67 8 5752 18.955223880597 1.27 10 0.56 6042 0.63 46 0.08 0.46 231 0.35 1 0.11 340 500 0 390 4 7620 97.5 0.34 0.05 0 126 1760 4710 2550 390 260 20 3 130 86.6666666666667 6.66666666666667 9,020 670 7.43% 13,639 72 32 6157 20.9375 1.76 4.71 2.55 0.39 0.26 0.02 126 5781 1750 4480 102330 510 1780 650 4750 240 2560 5 420 102 440 130 0 48 1 8,560 420 1,400 440 16.36% 0 12,990 9,090 309 860 9.46% 13,609 109 34 5781 12.8440366972477 1.4 10 0.51 6149 0.65 86 0.24 0.86 232 0.42 1 0.44 360 1700 0 530 77 7599 6.8831168831 0.36 0.17 0 127 1780 4750 2560 420 440 0 2 210 220 0 9,090 860 9.46% 13,609 34 10 6149 86 1.78 4.75 2.56 0.42 0.44 0 127 1780 5797 4750 25601620 4204230 102310440 670 0 1760 1530 2 4710 10210 2570 14220 36047.8571428571429 0 200109.2857142857149,090 0.71428571428571410 860 7 51.42857142857148,1609.46% 28.57142857142862,21013,609 1.4285714285714327.08% 34 12,927 9,040 169 570 6.31%10 13,7126149 78 8675 5797 1.7828.3333333333333 4.75 2.21 2.5610 0.67 62490.42 1.53 0.4457 0.01 0.57 0 234 0.36 1 0.2 340 0.0190 0 430 144 7642 2.9861111111 0.34 0.09 0 128 1770 4810 2570 330 40 10 1 330 40 10 9,150 380 4.15% 13,612 3 1 6150 380 1.77 4.81 2.57 0.33 0.04 0.01 128 1770 5837 4810 25701740 3304390 10237040 66010 1990 1600 1 5470 170330 3070 12 40 460 55 10 680133.3333333333339,150 14.16666666666670 380 11 41.81818181818188,5004.15% 61.81818181818182,43013,612 28.59% 3 0 12,952 10,530 533 1,140 10.83%1 15,0966150 100 3808 5837 1.77 24.3 4.81 2.43 2.5710 0.66 69480.33 1.6 1140.04 0.17 1.140.01 238 0.46 1 0.68 350 1600 0 510 2 7760 255 0.35 0.16 0 129 1760 4540 2550 340 110 0 1 340 110 0 8,850 450 5.08% 13,627 17 16 6156 28.125 1.76 4.54 2.55 0.34 0.11 0 129 1760 5842 4540 25501640 3404240 102370110 400 0 2160 320 1 6290 10340 3410 4110 350 100 0 80 808,850 0 2.5 450 4 8,2505.08%87.5 73013,627 20 8.85% 17 0 12,973 11,860 7 430 3.63%16 16,3176156 23 28.12526 5842 1.7631.7391304347826 4.54 0.73 2.5510 0.4 81220.34 0.32 0.1143 0.01 0.43 0 239 0.35 1 0.08 380 1900 0 570 16 7760 35.625 0.38 0.19 0 130 1820 4590 2580 340 130 20 1 340 130 20 8,990 490 5.45% 13,680 107 48 6182 10.2083333333333 1.82 4.59 2.58 0.34 0.13 0.02 130 1820 5843 4590 25801650 3404320 112330130 46020 1750 570 1 4720 0340 2540 1130 390 460 20 270 5708,990 20 0 490 1 8,3005.45%390 1,03013,680270 12.41% 10720 12,966 9,010 16 680 7.55%48 13,7276182 10.208333333333314 10 5843 1.8273.5714285714286 4.59 1.03 2.5811 0.46 62580.34 61.81818181818180.57 0.13 0 0.680.02 246 0.39 1 0.27 350 0.0260 0 410 6 7803 68.333333333 0.35 0.06 0 131 1730 4760 2670 400 380 40 2 200 190 20 9,160 820 8.95% 13,708 244 15 6197 54.6666666666667 1.73 4.76 2.67 0.4 0.38 0.04 131 1730 5843 4760 26701650 4004460 112540380 40040 1850 310 2 4620 0200 2670 1190 420 400 20 430 3109,160 40 0 820 3 8,6508.95%140 143.33333333333371013,708 13.33333333333338.21% 244 12,966 9,140 0 890 9.74%15 14,1726197 54.66666666666672 11 5843 1.73 355 4.76 0.71 2.6711 0.4 65390.4 80.90909090909090.31 0.38 0 0.890.04 248 0.42 1 0.43 340 0.0450 0 390 2 7822 195 0.34 0.05 0 132 1760 4850 2660 400 460 250 3 133.333333333333 153.333333333333 83.3333333333333 9,270 1,110 11.97% 13,848 140 60 6254 18.5 1.76 4.85 2.66 0.4 0.46 0.25 132 1760 5904 4850 26601650 4004520 112350460 520250 1830 1190 3 5180133.333333333333180 2820153.3333333333337 61074.285714285714383.33333333333331220 1709,270 25.714285714285720 1,110 4 8,52011.97%152.5 1,89013,848305 22.18% 140 5 13,134 9,830 699 1,850 18.82%60 14,3296254 94 18.521 5904 1.7620.1063829787234 4.85 1.89 2.6611 0.52 66090.4 168.1818181818181.19 0.46 0.18 1.850.25 250 0.61 1 1.22 370 0.02200 0 570 2 7836 285 0.37 0.2 0 133 1750 4880 2670 380 390 50 4 95 97.5 12.5 9,300 820 8.82% 13,846 260 42 6268 19.5238095238095 1.75 4.88 2.67 0.38 0.39 0.05 133 1750 5928 4880 26701650 3804450 112460390 54050 2260 1030 4 6820 6095 3760 1197.5 35049.090909090909112.5 93.636363636363690 9,300 5.4545454545454510 820 3 116.6666666666678,5608.82% 1,63013,846 30 3.3333333333333319.04% 260 13,280 12,840 246 450 3.50%42 17,4726268 19.5238095238095103 12 5928 1.7515.8252427184466 4.88 1.63 2.6711 0.54 86330.38 40.90909090909091.03 0.39 0.06 0.450.05 251 0.35 1 0.09 400 0.01390 0 790 2 7836 395 0.4 0.39 0 134 1790 4420 2550 420 500 0 4 105 125 0 8,760 920 10.50% 13,705 141 53 6217 17.3584905660377 1.79 4.42 2.55 0.42 0.5 0 134 1790 5938 4420 25501660 4204520 122440500 380 0 1190 300 4 2940 0105 1400 4125 390 95 0 260 758,760 0 0 920 3 8,62010.50%130 86.666666666666768013,705 7.89% 141 0 13,298 5,530 32 650 11.75%53 62178,678 17.358490566037724 9 5938 1.7928.3333333333333 4.42 0.68 2.5512 0.38 39310.42 54.16666666666670.3 0.5 0 0.65 0 252 0.39 1 0.26 350 1100 0 460 13 7845 35.384615385 0.35 0.11 0 135 1740 5150 2630 360 100 10 1 360 100 10 9,520 470 4.94% 13,706 3 2 6217 235 1.74 5.15 2.63 0.36 0.1 0.01 135 1740 5938 5150 26301690 3604690 122440100 35010 1920 100 1 4420 0360 2480 1100 350 350 10 200 1009,520 10 0 470 1 8,8204.94%350 45013,706200 5.10% 310 13,298 8,820 2 560 6.35%2 13,7176217 4 23524 5938 1.74 112.5 5.15 0.45 2.6312 0.35 62450.36 46.66666666666670.1 0.1 0 0.560.01 253 0.35 1 0.2 340 0.0170 0 410 12 7845 34.166666667 0.34 0.07 0 136 1760 4590 2580 690 1440 350 10 69 144 35 8,930 2,480 27.77% 13,668 356 87 6184 28.5057471264368 1.76 4.59 2.58 0.69 1.44 0.35 136 1760 5965 4590 25801740 6904580 1214402540 430350 1950 56010 5750 24069 3160 3144 400143.333333333333 35 390186.6666666666678,930 0 802,480 1 8,86027.77%400 1,23013,668390 13.88% 356 0 13,369 10,860 253 790 7.27%87 15,3506184 28.505747126436827 14 5965 1.7645.5555555555556 4.59 1.23 2.5812 0.43 70710.69 65.83333333333330.56 1.44 0.24 0.790.35 254 0.4 1 0.39 360 1800 0 540 8 7845 67.5 0.36 0.18 0 137 1760 4860 2540 370 130 180 3 123.333333333333 43.3333333333333 60 9,160 680 7.42% 13,681 37 23 6205 29.5652173913043 1.76 4.86 2.54 0.37 0.13 0.18 137 1760 5984 4860 25401680 3704660 122560130 360180 2340 310 3 6660123.33333333333340 351043.33333333333334 340 90 60 70 77.59,160 0 10 680 1 8,9007.42%340 71013,681 70 7.98% 37 0 13,448 12,510 139 410 3.28%23 16,0756205 29.565217391304333 8 5984 1.7621.5151515151515 4.86 0.71 2.5412 0.36 78450.37 34.16666666666670.31 0.13 0.04 0.410.18 255 0.34 1 0.07 370 1100 10 490 31 7848 15.806451613 0.37 0.11 0.01 138 1730 4840 2510 330 70 10 1 330 70 10 9,080 410 4.52% 13,689 8 4 6207 102.5 1.73 4.84 2.51 0.33 0.07 0.01 138 1730 5997 4840 25101710 3304800 12246070 42010 1980 480 1 6270 0330 3740 3 70 380 140 10 380 1609,080 0 0 410 2 8,9704.52%190 90013,689190 10.03% 8 0 13,481 11,990 33 760 6.34%4 16,2276207 19 102.50 5997 1.7347.3684210526316 4.84 0.9 2.5112 0.42 80180.33 63.33333333333330.48 0.07 0 0.760.01 256 0.38 1 0.38 330 500 0 380 4 7850 95 0.33 0.05 0 139 1750 4730 2570 420 470 40 4 105 117.5 10 9,050 930 10.28% 13,702 167 19 6212 48.9473684210526 1.75 4.73 2.57 0.42 0.47 0.04 139 1750 5998 4730 25701680 4204630 132460470 48040 1250 640 4 3220 0105 1500 117.53 360 160 10 220213.3333333333339,050 20 0 930 3 8,77010.28%120 73.33333333333331,12013,702 6.6666666666666712.77% 167 13,482 5,970 17 600 10.05%19 62129,339 48.947368421052667 75 5998 1.7516.7164179104478 4.73 1.12 2.5713 0.48 41610.42 46.15384615384620.64 0.47 0 0.60.04 257 0.36 1 0.22 340 0.0240 0 380 13 7849 29.230769231 0.34 0.04 0 140 1760 4790 2570 330 110 0 1 330 110 0 9,120 440 4.82% 13,702 4 2 6212 220 1.76 4.79 2.57 0.33 0.11 0 140 1760 5998 4790 25701680 3304590 132600110 350 0 1670 50 1 4360 0330 2530 1110 410 350 0 410 509,120 30 0 440 1 8,8704.82%410 40013,702410 4.51% 430 13,484 8,560 2 850 9.93%2 13,5126212 6 22050 5998 1.7666.6666666666667 4.79 0.4 2.5713 0.35 60100.33 65.38461538461540.05 0.11 0 0.85 0 258 0.41 1 0.41 320 0.0360 10 390 5 7850 78 0.32 0.06 0.01 141 1750 4830 2620 560 1130 130 5 112 226 26 9,200 1,820 19.78% 13,707 507 36 6216 50.5555555555556 1.75 4.83 2.62 0.56 1.13 0.13 141 1750 5999 4830 26201640 5604290 1311302500 380130 1770 200 5 4510 0112 2510 1226 380 380 26 230 2009,200 0 1,8200 5 8,43019.78% 76 58013,707 46 6.88% 507 0 13,490 8,790 6 610 6.94%36 13,8916216 50.55555555555561 19 5999 1.75 580 4.83 0.58 2.6213 0.38 63370.56 46.92307692307690.2 1.13 0 0.610.13 259 0.38 1 0.23 360 1300 10 500 26 7872 19.230769231 0.36 0.13 0.01 142 1860 4710 2580 350 180 10 2 175 90 5 9,150 540 5.90% 13,725 76 18 6230 30 1.86 4.71 2.58 0.35 0.18 0.01 142 1860 5999 4710 25801680 3504440 132510180 33010 2100 50 2 6400 0175 3660 1 90 350 330 5 110 509,150 0 0 540 3 116.6666666666678,6305.90% 36.666666666666738013,725 4.40% 76 0 13,492 12,160 2 460 3.78%18 16,0716230 2 3016 5999 1.86 190 4.71 0.38 2.5813 0.33 78450.35 35.38461538461540.05 0.18 0 0.460.01 262 0.35 1 0.11 340 800 0 420 6 7920 70 0.34 0.08 0 143 1810 4520 2600 440 330 0 1 440 330 0 8,930 770 8.62% 13,729 4 16 6242 48.125 1.81 4.52 2.6 0.44 0.33 0 143 1810 6010 4520 26001670 4404360 132530330 410 0 2000 410 1 6130 30440 3420 2330 340 205 0 40 2058,930 0 15 770 2 8,5608.62%170 85013,729 20 9.93% 4 0 13,512 11,550 50 380 3.29%16 16,0916242 13 48.12513 6010 1.8165.3846153846154 4.52 0.85 2.613 0.41 78490.44 29.23076923076920.41 0.33 0.03 0.38 0 263 0.34 1 0.04 380 4100 20 810 13 7929 62.307692308 0.38 0.41 0.02 144 1760 4820 2570 370 270 40 1 370 270 40 9,150 680 7.43% 13,715 200 27 6245 25.1851851851852 1.76 4.82 2.57 0.37 0.27 0.04 144 1760 6014 4820 25701660 3704600 132510270 34040 2120 100 1 6370 0370 3950 3270 380113.333333333333 40 41033.33333333333339,150 20 0 680 2 8,7707.43%190 44013,715205 5.02% 20010 13,530 12,440 0 810 6.51%27 16,1576245 25.18518518518523 34 6014 1.76146.666666666667 4.82 0.44 2.5713 0.34 79290.37 62.30769230769230.1 0.27 0 0.810.04 265 0.38 1 0.41 400 0.02270 10 680 5 7960 136 0.4 0.27 0.01 145 1940 4910 2580 400 290 0 2040 2 6540 200 3750 145 350 0 190 9,430 10 690 2 7.32%175 13,718 95 5 5 12,330 550 4.46%17 17,4736252 40.588235294117610 1.94 4.91 2.5813 86330.4 42.30769230769230.29 0.55 0 0.35 0.19 0.01 145 1940 6017 4910 25801690 4004700 132420290 440 0 490 2 10200 1145 440 0 4909,430 10 690 8,8107.32% 94013,718 10.67% 5 13,530 26 17 6252 40.58823529411769 6017 1.94104.444444444444 4.91 0.94 2.58 0.44 0.4 0.49 0.29 0.01 0 266 1 330 50 0 380 23 7971 16.52173913 0.33 0.05 0 146 1790 4850 2660 360 13110 0 2020 1 6590 360 3740 110 350 0 180 9,300 0 470 1 5.05%350 13,712180 6 0 12,350 530 4.29%8 17,4726248 58.7516 1.79 4.85 2.6613 86340.36 40.76923076923080.11 0.53 0 0.35 0.18 0 146 1790 6022 4850 26601680 3604370 2490110 520 0 690 1 30360 5110 104 0 1389,300 6 470 8,5405.05% 1,24013,712 14.52% 6 13,529 63 8 6248 30 58.75 6022 1.7941.3333333333333 4.85 1.24 2.66 0.52 0.36 0.69 0.11 0.03 0 267 1 350 100 0 450 23 7954 19.565217391 0.35 0.1 0 147 1770 4480 2530 380 13460 70 2280 3 6690126.666666666667 3760153.333333333333 350 23.3333333333333 200 8,780 0 910 7 10.36% 50 28.571428571428613,717 423 0 12,730 550 4.32%55 17,4736245 16.545454545454510 1.77 4.48 2.5313 86340.38 42.30769230769230.46 0.550.07 0.35 0.2 0 147 1770 6042 4480 25301730 3804320 2610460 41070 460 3 126.66666666666760 153.3333333333333 136.66666666666723.3333333333333 153.3333333333338,780 20 910 8,66010.36% 93013,717 10.74% 423 13,567 72 55 6245 16.545454545454531 6042 1.77 30 4.48 0.93 2.53 0.41 0.38 0.46 0.46 0.06 0.07 268 1 360 230 30 620 33 7961 18.787878788 0.36 0.23 0.03 148 1920 4420 2480 350 13200 10 2350 3 6930116.666666666667 389066.6666666666667 410 3.33333333333333 270 8,820 0 560 5 6.35% 82 13,717 54 24 0 13,170 680 5.16%12 18,2046245 46.66666666666678 1.92 4.42 2.4813 89910.35 52.30769230769230.2 0.680.01 0.41 0.27 0 148 1920 6042 4420 24801690 3504510 2570200 35010 110 3 116.6666666666670 66.66666666666672 3.33333333333333175 558,820 0 560 8,7706.35% 46013,717 5.25% 24 13,565 8 12 6245 46.666666666666710 6042 1.92 46 4.42 0.46 2.48 0.35 0.35 0.11 0.2 0 0.01 269 1 330 90 0 420 7 7966 60 0.33 0.09 0 149 1760 6048 4710 25701720 3604770 142540200 41010 1400 630 1 3760 110360 1880 2200 360 205 10 80 3159,040 10 55 570 1 9,0306.31%360 1,15013,712 80 12.74% 7510 13,587 7,040 380 450 6.39%10 10,8416249 72 576 6048 1.7615.9722222222222 4.71 1.15 2.5714 0.41 47910.36 32.14285714285710.63 0.2 0.11 0.450.01 0.36 0.08 0.01 149 1760 4710 2570 360 200 10 1 360 200 10 9,040 570 6.31% 13,712 75 10 6249 57 1.76 4.71 2.57 0.36 0.2 0.01 270 1 340 60 0 400 1 7967 400 0.34 0.06 0 150 1760 6058 4740 25701870 3705020 142530270 41050 1650 290 1 4320 10370 2330 2270 460 205 50 570 1459,070 0 5 690 2 9,4207.61%230 71013,729285 7.54% 181 0 13,519 8,300 2 1,030 12.41%24 12,9666259 8 28.7516 6058 1.76 88.75 4.74 0.71 2.5714 0.41 58430.37 73.57142857142860.29 0.27 0.01 1.030.05 0.46 0.57 0 150 1760 4740 2570 370 270 50 1 370 270 50 9,070 690 7.61% 13,729 181 24 6259 28.75 1.76 4.74 2.57 0.37 0.27 0.05 272 1 350 110 0 460 9 8003 51.111111111 0.35 0.11 0 151 1750 6061 4720 25401750 3904740 142610270 62020 1740 1380 3 4700 100130 2560 1 90 360 6.66666666666667620 200 1,3809,010 10 100 680 2 9,1007.55%180 2,10013,727100 23.08% 10 5 13,553 9,000 430 570 6.33%11 13,8966258 61.8181818181818161 59 6061 1.7513.0434782608696 4.72 2.1 2.5414 0.62 63460.39 40.71428571428571.38 0.27 0.1 0.570.02 0.36 0.2 0.01 151 1750 4720 2540 390 270 20 3 130 90 6.66666666666667 9,010 680 7.55% 13,727 10 11 6258 61.8181818181818 1.75 4.72 2.54 0.39 0.27 0.02 273 1 370 210 10 590 28 8013 21.071428571 0.37 0.21 0.01 152 1760 6062 4530 25301730 4604760 142630680 430220 1840 450 4 4440 70115 2520 7170 35061.4285714285714 55 64.285714285714370 8,820 180 101,360 3 116.6666666666679,12015.42% 23.333333333333395013,899 10.42% 29860 13,519 8,800 328 600 6.82%138 14,0276356 9.8550724637681246 6 6062 1.7620.6521739130435 4.53 0.95 2.5314 0.43 64660.46 42.85714285714290.45 0.68 0.07 0.60.22 0.35 0.07 0.18 152 1760 4530 2530 460 680 220 4 115 170 55 8,820 1,360 15.42% 13,899 298 138 6356 9.85507246376812 1.76 4.53 2.53 0.46 0.68 0.22 276 1 380 180 0 560 39 8018 14.358974359 0.38 0.18 0 153 1830 6141 4730 25201860 5504670 142590790 380 0 2000 430 4 5790 137.570 3250 197.55 410 76 0 520 869,080 60 141,340 2 9,12014.76%205 88013,902260 9.65% 1130 13,622 11,040 293 990 8.97%38 15,5666342 35.2631578947368134 143 6141 1.836.56716417910448 4.73 0.88 2.5214 0.38 71630.55 70.71428571428570.43 0.79 0.07 0.99 0 0.41 0.52 0.06 153 1830 4730 2520 550 790 0 4 137.5 197.5 0 9,080 1,340 14.76% 13,902 11 38 6342 35.2631578947368 1.83 4.73 2.52 0.55 0.79 0 277 1 380 380 0 760 12 8018 63.333333333 0.38 0.38 0 154 1770 6143 4510 25101720 3804770 142590230 420 0 1970 110 2 6040 0190 3640 1115 370 420 0 190 1108,790 0 0 610 1 9,0806.94%370 53013,891190 5.84% 19 0 13,622 11,650 0 560 4.81%13 16,2996337 46.92307692307696 0 6143 1.7788.3333333333333 4.51 0.53 2.5114 0.42 81180.38 0.11 0.2340 0 0.56 0 0.37 0.19 0 154 1770 4510 2510 380 230 0 2 190 115 0 8,790 610 6.94% 13,891 19 13 6337 46.9230769230769 1.77 4.51 2.51 0.38 0.23 0 278 1 370 240 30 640 103 8027 6.213592233 0.37 0.24 0.03 155 1790 6149 4800 25401750 3504670 152520220 36010 1310 170 1 3290 0350 1570 2220 390 180 10 110 859,130 10 0 580 1 8,9406.35%390 53013,907110 5.93% 4810 13,625 6,170 3 510 8.27%9 63389,796 64.444444444444481 112 6149 1.796.54320987654321 4.8 0.53 2.5415 0.36 43770.35 0.17 0.2234 0 0.510.01 0.39 0.11 0.01 155 1790 4800 2540 350 220 10 1 350 220 10 9,130 580 6.35% 13,907 48 9 6338 64.4444444444444 1.79 4.8 2.54 0.35 0.22 0.01 280 1 340 70 10 420 4 8057 105 0.34 0.07 0.01 156 1770 6149 4350 25501750 3604770 152560200 36010 1610 200 1 4290 0360 2330 2200 380 180 10 250 1008,670 10 0 570 1 9,0806.57%380 56013,917250 6.17% 1010 13,625 8,230 0 640 7.78%4 12,8326342 4 142.515 6149 1.77 140 4.35 0.56 2.5515 0.36 57200.36 42.66666666666670.2 0.2 0 0.640.01 0.38 0.25 0.01 156 1770 4350 2550 360 200 10 1 360 200 10 8,670 570 6.57% 13,917 10 4 6342 142.5 1.77 4.35 2.55 0.36 0.2 0.01 290 1 400 240 40 680 2 8123 340 0.4 0.24 0.04 157 1740 6149 4700 25601780 3604750 152560200 42010 1730 440 1 4760 0360 2670 3200 400 140 10 380146.6666666666679,000 40 0 570 2 9,0906.33%200 86013,896190 9.46% 5920 13,609 9,160 34 820 8.95%14 13,7086346 40.714285714285710 244 6149 1.74 86 4.7 0.86 2.5615 0.42 61970.36 54.66666666666670.44 0.2 0 0.820.01 0.4 0.38 0.04 157 1740 4700 2560 360 200 10 1 360 200 10 9,000 570 6.33% 13,896 59 14 6346 40.7142857142857 1.74 4.7 2.56 0.36 0.2 0.01 291 1 350 240 20 610 241 8130 2.531120332 0.35 0.24 0.02 158 1760 6150 4690 25801770 4904810 152570560 330180 2170 40 5 6650 1098 3580 2112 390 165 36 280 209,030 10 1,2305 3 9,15013.62%130 93.333333333333338013,931 3.333333333333334.15% 35 13,612 12,400 3 680 5.48%89 16,2756432 13.82022471910111 4 6150 1.76 380 4.69 0.38 2.5815 0.33 80670.49 45.33333333333330.04 0.56 0.01 0.680.18 0.39 0.28 0.01 158 1760 4690 2580 490 560 180 5 98 112 36 9,030 1,230 13.62% 13,931 35 89 6432 13.8202247191011 1.76 4.69 2.58 0.49 0.56 0.18 292 1 390 400 0 790 6 8130 131.66666667 0.39 0.4 0 159 1760 6156 4470 25101760 6604540 1513702550 34040 2280 11011 6870 060 3910124.5454545454551 430 3.63636363636364340 270 1108,740 0 2,0700 4 8,85023.68%107.5 45014,02167.5 5.08% 182 0 13,627 13,060 17 700 5.36%74 18,1966454 27.97297297297316 0 6156 1.76 28.125 4.47 0.45 2.5115 0.34 89840.66 46.66666666666670.11 1.37 0 0.70.04 0.43 0.27 0 159 1760 4470 2510 660 1370 40 11 60 124.545454545455 3.63636363636364 8,740 2,070 23.68% 14,021 182 74 6454 27.972972972973 1.76 4.47 2.51 0.66 1.37 0.04 293 1 340 120 0 460 2 8130 230 0.34 0.12 0 160 1840 6157 4440 25201760 3504710 16255070 390180 1440 260 1 3940 20350 2030 1 70 340 390180 40 2608,800 0 20 600 4 9,0206.82% 85 67014,027 10 7.43% 6 0 13,639 7,410 72 380 5.13%14 11,4856466 42.857142857142932 2 6157 1.84 20.9375 4.44 0.67 2.5216 0.39 50780.35 0.26 23.750.07 0.02 0.380.18 0.34 0.04 0 160 1840 4440 2520 350 70 180 1 350 70 180 8,800 600 6.82% 14,027 6 14 6466 42.8571428571429 1.84 4.44 2.52 0.35 0.07 0.18 294 1 370 190 0 560 14 8118 40 0.37 0.19 0 161 1770 6182 4500 25101820 3504590 16258070 340170 1760 130 1 4540 20350 2550 1 70 340 340170 110 1308,780 0 20 590 1 8,9906.72%340 49014,033110 5.45% 6 0 13,680 8,850 107 450 5.08%17 13,6276483 34.705882352941248 17 6182 1.7710.2083333333333 4.5 0.49 2.5116 0.34 61560.35 0.13 28.1250.07 0.02 0.450.17 0.34 0.11 0 161 1770 4500 2510 350 70 170 1 350 70 170 8,780 590 6.72% 14,033 6 17 6483 34.7058823529412 1.77 4.5 2.51 0.35 0.07 0.17 295 1 350 80 0 430 10 8122 43 0.35 0.08 0 162 1780 6184 4820 25201760 4504590 162580780 69080 1810 1440 6 4520 35075 2600 2130 440 13.3333333333333345 330 7209,120 0 1751,310 10 8,93014.36% 44 2,48014,185 33 27.77% 314 0 13,668 8,930 356 770 8.62%128 13,7296538 10.23437587 4 6184 1.7828.5057471264368 4.82 2.48 2.5216 0.69 62420.45 1.44 48.1250.78 0.35 0.770.08 0.44 0.33 0 162 1780 4820 2520 450 780 80 6 75 130 13.3333333333333 9,120 1,310 14.36% 14,185 314 128 6538 10.234375 1.78 4.82 2.52 0.45 0.78 0.08 296 1 350 180 30 560 63 8143 8.8888888889 0.35 0.18 0.03 163 1830 6197 4880 26501730 3604760 16267070 400 0 2090 380 1 6250 40360 3470 3 70 380133.333333333333 0 190126.6666666666679,360 13.33333333333330 430 3 126.6666666666679,1604.59% 63.333333333333382014,186 8.95% 1 0 13,708 11,810 244 570 4.83%3 15,9696539 143.33333333333315 8 6197 1.8354.6666666666667 4.88 0.82 2.6516 0.4 77600.36 0.38 35.6250.07 0.04 0.57 0 0.38 0.19 0 163 1830 4880 2650 360 70 0 1 360 70 0 9,360 430 4.59% 14,186 1 3 6539 143.333333333333 1.83 4.88 2.65 0.36 0.07 0 305 1 390 400 0 790 5 8460 158 0.39 0.4 0 164 1880 6205 4560 26701760 4704860 162540650 370120 1790 130 4 6750 180117.5 3820 162.54 410 92.5 30 310 32.59,110 10 451,240 1 9,16013.61%410 68014,171310 7.42% 6710 13,681 12,360 37 730 5.91%33 17,8506538 37.575757575757623 3 6205 1.8829.5652173913043 4.56 0.68 2.6716 0.37 88210.47 0.13 45.6250.65 0.18 0.730.12 0.41 0.31 0.01 164 1880 4560 2670 470 650 120 4 117.5 162.5 30 9,110 1,240 13.61% 14,171 67 33 6538 37.5757575757576 1.88 4.56 2.67 0.47 0.65 0.12 308 1 360 140 10 510 8 8632 63.75 0.36 0.14 0.01 165 1850 6207 5000 26901730 3404840 17251060 330 0 1940 70 1 4910 10340 2580 4 60 400 82.5 0 290 17.59,540 0 2.5 400 4 9,0804.19%100 41014,17172.5 4.52% 0 0 13,689 9,430 8 690 7.32%4 13,7186538 4 1005 6207 1.85 102.5 5 0.41 2.6917 0.33 62520.34 40.58823529411760.07 0.06 0.01 0.69 0 0.4 0.29 0 165 1850 5000 2690 340 60 0 1 340 60 0 9,540 400 4.19% 14,171 0 4 6538 100 1.85 5 2.69 0.34 0.06 0 309 1 350 90 10 450 11 8633 40.909090909 0.35 0.09 0.01 166 1850 6212 4620 26701750 4204730 172570430 42040 1770 470 3 4500 40140 2510143.3333333333331 350 13.3333333333333420 70 4709,140 170 40 890 1 9,0509.74%350 93014,172 70 10.28% 11170 13,702 8,780 167 590 6.72%11 14,0336539 80.909090909090919 6 6212 1.8548.9473684210526 4.62 0.93 2.6717 0.42 64830.42 34.70588235294120.47 0.43 0.04 0.590.04 0.35 0.07 0.17 166 1850 4620 2670 420 430 40 3 140 143.333333333333 13.3333333333333 9,140 890 9.74% 14,172 11 11 6539 80.9090909090909 1.85 4.62 2.67 0.42 0.43 0.04 310 1 350 140 0 490 2 8633 245 0.35 0.14 0 6212 1760 4790 172570 330 1970 110 6180 0 3480 10 380 33 220 11 0 0 5 9,120 76 440 44 4.82% 0 13,702 11,630 4 600 5.16% 16,122 2 8 6212 220 0.44 17 0.33 7920 35.29411764705880.11 0 0.6 0.38 0.22 0 167 1840 4900 2670 320 60 10 1 320 60 10 9,410 390 4.14% 14,171 7 1 6538 390 1.84 4.9 2.67 0.32 0.06 0.01 312 1 350 180 0 530 13 8634 40.769230769 0.35 0.18 0 6216 1750 4830 172620 560 2280 1130 6780 130 3740 3 370186.666666666667 270376.666666666667 43.33333333333330 2 9,200 185 1,820 135 19.78% 0 13,707 12,800 507 640 5.00% 17,479 36 12 6216 50.5555555555556 1.82 17 0.56 8635 37.64705882352941.13 0.13 0.64 312 0.37 1 0.27 350 1800 0 530 13 8634 40.769230769 0.35 0.18 0 168 1800 4930 2570 350 90 0 1 350 90 0 9,300 440 4.73% 14,172 1 36 6504 12.2222222222222 1.8 4.93 2.57 0.35 0.09 0 313 1 380 200 0 580 8 8634 72.5 0.38 0.2 0 6217 1790 4420 182550 420 1350 500 3130 0 1570 1 410 420 340 500 0 0 1 8,760 410 920 340 10.50% 0 13,705 6,050 141 750 12.40% 9,750 53 4 6217 17.3584905660377 0.92 18 0.42 4347 41.66666666666670.5 0 0.75 313 0.41 1 0.34 380 2000 0 580 8 8634 72.5 0.38 0.2 0 169 1800 4680 2650 330 50 0 1 330 50 0 9,130 380 4.16% 14,172 0 2 6504 190 1.8 4.68 2.65 0.33 0.05 0 314 1 350 190 10 550 13 8633 42.307692308 0.35 0.19 0.01 6217 1740 5150 182630 360 1860 100 4710 10 2580 4 350 90 180 25 10 2.5 1 9,520 350 470 180 4.94% 10 13,706 9,150 3 540 5.90% 13,725 2 76 6217 235 0.47 18 0.36 6230 0.1 30 0.01 0.54 314 0.35 1 0.18 350 0.01190 10 550 13 8633 42.307692308 0.35 0.19 0.01 170 1800 4580 2670 540 1020 90 8 67.5 127.5 11.25 9,050 1,650 18.23% 13,990 752 131 6426 12.5954198473282 1.8 4.58 2.67 0.54 1.02 0.09 316 1 350 200 0 550 13 8634 42.307692308 0.35 0.2 0 6230 1860 4710 182580 350 1890 180 6090 10 3610 1 330 350 160 180 10 10 2 9,150 165 540 80 5.90% 5 13,725 11,590 76 500 4.31% 16,321 18 20 6230 30 0.54 18 0.35 8109 27.77777777777780.18 0.01 0.5 316 0.33 1 0.16 350 0.01200 0 550 13 8634 42.307692308 0.35 0.2 0 171 1800 4720 2750 440 670 110 5 88 134 22 9,270 1,220 13.16% 14,148 254 74 6499 16.4864864864865 1.8 4.72 2.75 0.44 0.67 0.11 318 1 320 60 0 380 8 8738 47.5 0.32 0.06 0 6242 1810 4520 182600 440 2030 330 6380 0 3650 5 400 88 330 66 40 0 1 8,930 400 770 330 8.62% 40 13,729 12,060 4 770 6.38% 16,988 16 34 6242 48.125 0.77 18 0.44 8435 42.77777777777780.33 0 0.77 318 0.4 1 0.33 320 0.0460 0 380 8 8738 47.5 0.32 0.06 0 172 1820 5120 2680 680 1490 100 8 85 186.25 12.5 9,620 2,270 23.60% 14,224 350 80 6540 28.375 1.82 5.12 2.68 0.68 1.49 0.1 320 1 330 70 10 410 4 8765 102.5 0.33 0.07 0.01 6245 1760 4820 192570 370 1190 270 3050 40 1420 2 430 185 340 135 0 20 3 143.3333333333339,150 113.333333333333680 7.43% 0 13,715 5,660 200 770 13.60% 8,702 27 24 6245 25.1851851851852 0.68 19 0.37 3944 40.52631578947370.27 0.04 0.77 320 0.43 1 0.34 330 700 10 410 4 8765 102.5 0.33 0.07 0.01 173 1820 5090 2830 690 1570 80 8 86.25 196.25 10 9,740 2,340 24.02% 14,310 254 68 6600 34.4117647058824 1.82 5.09 2.83 0.69 1.57 0.08 322 1 370 90 0 460 4 8765 115 0.37 0.09 0 6245 1770 4480 192530 380 1460 460 3730 70 2040 1 440 380 410 460 10 70 3 146.6666666666678,780 136.666666666667910 3.3333333333333310.36% 13,717 7,230 423 860 11.89% 11,453 55 10 6245 16.5454545454545 0.91 19 0.38 5049 45.26315789473680.46 0.07 0.86 322 0.44 1 0.41 370 0.0190 0 460 4 8765 115 0.37 0.09 0 174 1830 5180 2820 610 1220 20 5 122 244 4 9,830 1,850 18.82% 14,329 21 11 6609 168.181818181818 1.83 5.18 2.82 0.61 1.22 0.02 324 1 350 150 0 500 1 8766 500 0.35 0.15 0 6245 1920 4420 192480 350 1710 200 4800 10 2460 1 420 350 480 200 0 10 1 8,820 420 560 480 6.35% 0 13,717 8,970 24 900 10.03% 13,481 12 33 6245 46.6666666666667 0.56 19 0.35 5997 47.36842105263160.2 0.01 0.9 324 0.42 1 0.48 350 1500 0 500 1 8766 500 0.35 0.15 0 175 1870 4890 2780 530 960 80 4 132.5 240 20 9,540 1,570 16.46% 14,483 254 61 6669 25.7377049180328 1.87 4.89 2.78 0.53 0.96 0.08 326 1 350 130 10 490 26 8815 18.846153846 0.35 0.13 0.01 6248 1790 4850 192660 360 1750 110 4730 0 2570 2 420 180 470 55 40 0 1 9,300 420 470 470 5.05% 40 13,712 9,050 6 930 10.28% 13,702 8 167 6248 58.75 0.47 19 0.36 6212 48.94736842105260.11 0 0.93 326 0.42 1 0.47 350 0.04130 10 490 26 8815 18.846153846 0.35 0.13 0.01 176 1850 5190 2870 380 280 10 1 380 280 10 9,910 670 6.76% 14,496 13 7 6676 95.7142857142857 1.85 5.19 2.87 0.38 0.28 0.01 327 1 350 200 0 550 2 8817 275 0.35 0.2 0 6249 1760 4710 192570 360 1940 200 6130 10 3400 1 340 360 70 200 0 10 3 113.3333333333339,040 23.3333333333333570 6.31% 0 13,712 11,470 75 410 3.57% 15,766 10 10 6249 57 0.57 19 0.36 7244 21.57894736842110.2 0.01 0.41 327 0.34 1 0.07 350 2000 0 550 2 8817 275 0.35 0.2 0 177 1840 5200 2870 360 140 10 1 360 140 10 9,910 510 5.15% 14,496 4 2 6676 255 1.84 5.2 2.87 0.36 0.14 0.01 6252 1940 4910 202580 400 1380 290 3430 0 1910 3 450133.333333333333 47096.6666666666667 10 0 4 9,430 112.5 690 117.5 7.32% 2.5 13,718 6,720 5 930 13.84% 10,978 17 34 6252 40.5882352941176 0.69 20 0.4 4843 0.29 46.5 0 0.93 331 0.45 1 0.47 360 0.0180 0 440 104 8835 4.2307692308 0.36 0.08 0 178 1850 5230 2860 390 190 190 3 130 63.3333333333333 63.3333333333333 9,940 770 7.75% 14,537 63 34 6703 22.6470588235294 1.85 5.23 2.86 0.39 0.19 0.19 6254 1760 4850 202660 400 1910 460 5630 250 3400 3 500133.333333333333 970153.333333333333 83.333333333333360 4 9,270 125 1,110 242.5 11.97% 15 13,848 10,940 140 1,530 13.99% 15,752 60 196 6254 18.5 1.11 20 0.4 7237 0.46 76.5 0.25 1.53 332 0.5 1 0.97 390 0.06230 0 620 6 8835 103.33333333 0.39 0.23 0 179 1860 5330 2980 470 1020 360 6 78.3333333333333 170 60 10,170 1,850 18.19% 14,682 459 117 6766 15.8119658119658 1.86 5.33 2.98 0.47 1.02 0.36 6258 1750 4720 202540 390 2250 270 6640 20 3880 1 410 390 210 270 10 20 2 9,010 205 680 105 7.55% 5 13,727 12,770 10 630 4.93% 16,968 11 31 6258 61.8181818181818 0.68 20 0.39 8425 0.27 31.5 0.02 0.63 333 0.41 1 0.21 370 0.01140 0 510 2 8835 255 0.37 0.14 0 180 1890 5050 2950 340 80 10 1 340 80 10 9,890 430 4.35% 14,686 4 2 6766 215 1.89 5.05 2.95 0.34 0.08 0.01 6259 1760 4740 212570 370 1310 270 3510 50 1680 1 350 370 30 270 0 50 1 9,070 350 690 30 7.61% 0 13,729 6,500 181 380 5.85% 9,866 24 4 6259 28.75 0.69 21 0.37 4378 18.09523809523810.27 0.05 0.38 334 0.35 1 0.03 390 2500 0 640 5 8838 128 0.39 0.25 0 181 1850 5390 2990 460 400 50 1 460 400 50 10,230 910 8.90% 14,686 4 2 6766 455 1.85 5.39 2.99 0.46 0.4 0.05 6268 1750 4880 222670 380 1260 390 3170 50 1460 3 400126.666666666667 250 130 16016.6666666666667 1 9,300 400 820 250 8.82% 160 13,846 5,890 260 810 13.75% 9,068 42 106 6268 19.5238095238095 0.82 22 0.38 4064 36.81818181818180.39 0.05 0.81 335 0.4 1 0.25 370 0.16180 10 560 3 8839 186.66666667 0.37 0.18 0.01 182 1920 5160 3030 650 1570 150 7 92.8571428571429 224.285714285714 21.4285714285714 10,110 2,370 23.44% 14,876 270 136 6872 17.4264705882353 1.92 5.16 3.03 0.65 1.57 0.15 6337 1770 4510 222510 380 1350 230 3440 0 1740 4 510 95 670 57.5 30 0 1 8,790 510 610 670 6.94% 30 13,891 6,530 19 1,210 18.53% 10,491 13 12 6337 46.9230769230769 0.61 22 0.38 4615 0.23 55 0 1.21 337 0.51 1 0.67 420 0.03180 20 620 7 8959 88.571428571 0.42 0.18 0.02 183 1890 5410 2960 360 300 0 2 180 150 0 10,260 660 6.43% 14,876 4 8 6874 82.5 1.89 5.41 2.96 0.36 0.3 0 6338 1790 4800 222540 350 1930 220 5380 10 3110 4 370 87.5 410 55 70 2.5 5 9,130 74 580 82 6.35% 14 13,907 10,420 48 850 8.16% 15,274 9 192 6338 64.4444444444444 0.58 22 0.35 7030 38.63636363636360.22 0.01 0.85 338 0.37 1 0.41 400 0.07180 10 590 2 8959 295 0.4 0.18 0.01 184 1890 5280 3130 650 1740 230 5 130 348 46 10,300 2,620 25.44% 15,082 536 125 6948 20.96 1.89 5.28 3.13 0.65 1.74 0.23 6342 1830 4730 232520 550 1640 790 4240 0 2370 2 400 275 320 395 10 0 11 36.36363636363649,080 29.09090909090911,340 0.90909090909090914.76% 13,902 8,250 11 730 8.85% 12,973 38 7 6342 35.2631578947368 1.34 23 0.55 5842 31.73913043478260.79 0 0.73 339 0.4 1 0.32 360 0.01150 10 520 7 8960 74.285714286 0.36 0.15 0.01 185 1950 5670 3160 360 80 10 1 360 80 10 10,780 450 4.17% 15,083 3 2 6948 225 1.95 5.67 3.16 0.36 0.08 0.01 6342 1770 4350 232550 360 1760 200 4860 10 2540 1 370 360 130 200 180 10 1 8,670 370 570 130 6.57% 180 13,917 9,160 10 680 7.42% 13,681 4 37 6342 142.5 0.57 23 0.36 6205 29.56521739130430.2 0.01 0.68 340 0.37 1 0.13 360 0.18140 10 510 2 8960 255 0.36 0.14 0.01 186 1900 5630 3150 390 160 0 1 390 160 0 10,680 550 5.15% 15,088 5 2 6948 275 1.9 5.63 3.15 0.39 0.16 0 6346 1740 4700 232560 360 2130 200 6450 10 3550 1 350 360 100 200 0 10 1 9,000 350 570 100 6.33% 0 13,896 12,130 59 450 3.71% 16,211 14 11 6346 40.7142857142857 0.57 23 0.36 7954 19.56521739130430.2 0.01 0.45 341 0.35 1 0.1 360 2000 40 600 27 8967 22.222222222 0.36 0.2 0.04 187 1990 5470 3070 460 680 0 3 153.333333333333 226.666666666667 0 10,530 1,140 10.83% 15,096 8 10 6948 114 1.99 5.47 3.07 0.46 0.68 0 6356 1760 4530 232530 460 2220 680 6320 220 3520 1 330 460 50 680 0 220 6 8,820 55 8.333333333333331,360 15.42% 0 13,899 12,060 298 380 3.15% 16,208 138 21 6356 9.85507246376812 1.36 23 0.46 7971 16.52173913043480.68 0.22 0.38 342 0.33 1 0.05 370 1500 0 520 5 8970 104 0.37 0.15 0 188 1900 5310 3090 410 570 110 2 205 285 55 10,300 1,090 10.58% 15,126 250 24 6964 45.4166666666667 1.9 5.31 3.09 0.41 0.57 0.11 188 1900 6426 5310 30901800 4104580 242670570 540110 1660 1020 2 4520 90205 2440 5285 380 108 55 300 20410,300 0 181,090 1 9,05010.58%380 1,65015,126300 18.23% 250 0 13,990 8,620 752 680 7.89%24 13,2986964 45.4166666666667131 32 6426 1.912.5954198473282 5.31 1.65 3.0924 0.54 59380.41 28.33333333333331.02 0.57 0.09 0.680.11 343 0.38 1 0.3 350 900 10 450 1 8971 450 0.35 0.09 0.01 189 1930 5660 3160 370 200 20 1 370 200 20 10,750 590 5.49% 15,211 85 34 6998 17.3529411764706 1.93 5.66 3.16 0.37 0.2 0.02 189 1930 6432 5660 31601760 3704690 242580200 49020 1760 560 1 4740 180370 2570 11200 37044.5454545454545 20 27050.909090909090910,750 16.363636363636450 590 4 9,0305.49%92.5 1,23015,21167.5 13.62% 8512.5 13,931 9,070 35 690 7.61%34 13,7296998 17.352941176470689 181 6432 1.9313.8202247191011 5.66 1.23 3.1624 0.49 62590.37 0.56 28.750.2 0.18 0.690.02 344 0.37 1 0.27 430 0.05270 0 700 15 8984 46.666666667 0.43 0.27 0 190 1920 5400 3130 360 510 100 2 180 255 50 10,450 970 9.28% 15,270 199 26 7024 37.3076923076923 1.92 5.4 3.13 0.36 0.51 0.1 190 1920 6454 5400 31301760 3604470 242510510 660100 1880 1370 2 5010 40180 2840 1255 340 660 50 50 1,37010,450 0 40 970 1 8,7409.28%340 2,07015,270 50 23.68% 199 0 14,021 9,730 182 390 4.01%26 14,4687024 37.307692307692374 3 6454 1.9227.972972972973 5.4 2.07 3.1324 0.66 68850.36 1.37 16.250.51 0.04 0.390.1 0.34 0.05 0 191 1930 5380 3110 370 410 70 1 370 410 70 10,420 850 8.16% 15,274 192 22 7030 38.6363636363636 1.93 5.38 3.11 0.37 0.41 0.07 191 1930 6466 5380 31101840 3704440 242520410 35070 1900 70 1 5310 180370 3090 1410 410 350 70 570 10,42070 110 180 850 3 136.6666666666678,8008.16% 60015,274190 36.66666666666676.82% 192 14,027 10,300 6 1,090 10.58%22 15,1267030 38.636363636363614 250 6466 1.9342.8571428571429 5.38 0.6 3.1124 0.35 69640.37 45.41666666666670.07 0.41 0.18 1.090.07 0.41 0.57 0.11 192 1930 5810 3270 380 450 50 2 190 225 25 11,010 880 7.99% 15,336 106 69 7063 12.7536231884058 1.93 5.81 3.27 0.38 0.45 0.05 192 1930 6483 5810 32701770 3804500 242510450 35050 1920 70 2 5630 170190 3280 6225 41058.3333333333333 25 45011.666666666666711,010 28.333333333333330 880 1 8,7807.99%410 59015,336450 6.72% 10630 14,033 10,830 6 890 8.22%69 15,5137063 12.753623188405817 63 6483 1.9334.7058823529412 5.81 0.59 3.2724 0.35 71410.38 37.08333333333330.07 0.45 0.17 0.890.05 0.41 0.45 0.03 193 1950 5750 3160 400 390 0 2 200 195 0 10,860 790 7.27% 15,350 14 12 7071 65.8333333333333 1.95 5.75 3.16 0.4 0.39 0 193 1950 6499 5750 31601800 4004720 252750390 440 0 2000 670 2 6210 110200 3570 1195 370 440 0 230 67010,860 20 110 790 1 9,2707.27%370 1,22015,350230 13.16% 1420 14,148 11,780 254 620 5.26%12 16,3267071 65.833333333333374 60 6499 1.9516.4864864864865 5.75 1.22 3.1625 0.44 80800.4 0.67 24.80.39 0.11 0.62 0 0.37 0.23 0.02 194 1910 5440 3260 400 330 220 3 133.333333333333 110 73.3333333333333 10,610 950 8.95% 15,419 73 52 7115 18.2692307692308 1.91 5.44 3.26 0.4 0.33 0.22 194 1910 6504 5440 32601800 4004930 252570330 350220 2270 90 3 6630133.3333333333330 3620 4110 360 73.333333333333387.5 250 22.510,610 10 0 950 1 9,3008.95%360 44015,419250 4.73% 7310 14,172 12,520 1 620 4.95%52 16,3077115 18.269230769230836 60 6504 1.9112.2222222222222 5.44 0.44 3.2625 0.35 80900.4 0.09 24.80.33 0 0.620.22 0.36 0.25 0.01 195 1950 5740 3240 390 470 60 1 390 470 60 10,930 920 8.42% 15,429 152 7 7116 131.428571428571 1.95 5.74 3.24 0.39 0.47 0.06 195 1950 6504 5740 32401800 3904680 262650470 33060 1540 50 1 3940 0390 2280 1470 450 330 60 670 10,93050 30 0 920 8 9,1308.42%56.25 38015,42983.75 4.16% 1523.75 14,172 7,760 0 1,150 14.82%7 12,1227116 131.4285714285712 135 6504 1.95 190 5.74 0.38 3.2426 0.33 53780.39 44.23076923076920.05 0.47 0 1.150.06 0.45 0.67 0.03 196 1980 5980 3290 390 410 10 1 390 410 10 11,250 810 7.20% 15,450 21 45 7119 18 1.98 5.98 3.29 0.39 0.41 0.01 196 1980 6538 5980 32901780 3904820 262520410 45010 1560 780 1 4310 80390 2230 3410 410 150 10 430 26011,250 26.666666666666710 810 5 9,1207.20% 82 1,31015,450 86 14.36% 21 2 14,185 8,100 314 850 10.49%45 12,3827119 128 1896 6538 1.98 10.234375 5.98 1.31 3.2926 0.45 55140.39 32.69230769230770.78 0.41 0.08 0.850.01 0.41 0.43 0.01 197 1920 5630 3280 410 450 30 1 410 450 30 10,830 890 8.22% 15,513 63 24 7141 37.0833333333333 1.92 5.63 3.28 0.41 0.45 0.03 197 1920 6538 5630 32801880 4104560 262670450 47030 1920 650 1 5400 120410 3130 1450 360 470 30 510 65010,830 100 120 890 8 9,1108.22% 45 1,24015,51363.75 13.61% 6312.5 14,171 10,450 67 970 9.28%24 15,2707141 37.083333333333333 199 6538 1.9237.5757575757576 5.63 1.24 3.2826 0.47 70240.41 37.30769230769230.65 0.45 0.12 0.970.03 0.36 0.51 0.1 198 1850 6538 5800 32901850 4705000 262690940 34070 1990 60 2 6240 0235 3520 1470 360 340 35 170 10,94060 0 1,4800 8 9,54013.53% 45 40015,52921.25 4.19% 192 0 14,171 11,750 0 530 4.51%101 16,0497148 14.65346534653474 6 6538 1.85 100 5.8 0.4 3.2926 0.34 77920.47 20.38461538461540.06 0.94 0 0.530.07 0.36 0.17 0 199 1920 6538 5690 33701840 3804900 262670360 320 0 2000 60 1 6160 10380 3340 1360 360 320 0 130 10,98060 10 10 740 5 9,4106.74% 72 39015,526 26 4.14% 5 2 14,171 11,500 7 500 4.35%2 16,0967148 1 3706 6538 1.92 390 5.69 0.39 3.3726 0.32 78720.38 19.23076923076920.06 0.36 0.01 0.5 0 0.36 0.13 0.01 200 1850 6539 5730 33401830 3804880 262650380 360 0 1930 70 1 6720 0380 3810 8380 350 45 0 130 8.7510,920 10 0 760 4 9,3606.96%87.5 43015,53332.5 4.59% 72.5 14,186 12,460 1 490 3.93%1 17,7897149 3 76011 6539 1.85143.333333333333 5.73 0.43 3.3426 0.36 88150.38 18.84615384615380.07 0.38 0 0.49 0 0.35 0.13 0.01 201 2000 6539 5790 32501850 4104620 272670520 42060 1740 430 1 4580 40410 2540 5520 430 84 60 560 11,04086 240 8 990 1 9,1408.97%430 89015,566560 9.74% 143240 14,172 8,860 11 1,230 13.88%14 13,3697163 70.714285714285711 253 6539 80.90909090909092 5.79 0.89 3.2527 0.42 59650.41 45.55555555555560.43 0.52 0.04 1.230.06 0.43 0.56 0.24 202 1930 6540 5620 32201820 3405120 27268050 68010 1760 1490 1 4820 100340 2570 8 50 370 85 10 270 186.2510,770 40 12.5 400 1 9,6203.71%370 2,27015,572270 23.60% 2440 14,224 9,150 350 680 7.43%4 13,7157167 80 100200 6540 1.93 28.375 5.62 2.27 3.2227 0.68 62450.34 25.18518518518521.49 0.05 0.1 0.680.01 0.37 0.27 0.04 203 1940 6600 5640 33901820 4405090 272830540 690 0 2280 1570 2 6760 80220 3800 8270 360 86.25 0 200 196.2510,970 40 10 980 3 9,7408.93%120 66.66666666666672,34015,572 13.333333333333324.02% 0 14,310 12,840 254 600 4.67%39 18,1887170 25.128205128205168 150 6600 1.9434.4117647058824 5.64 2.34 3.3927 0.69 89670.44 22.22222222222221.57 0.54 0.08 0.6 0 0.36 0.2 0.04 204 1980 6609 5670 33601830 4005180 282820400 610 0 1210 1220 1 3190 20400 1470 5400 400 122 0 280 24411,010 20 4 800 6 66.66666666666679,8307.27% 46.66666666666671,85015,578 3.3333333333333318.82% 6 14,329 5,870 21 700 11.93%1 71718,479 11 80053 6609 1.98168.181818181818 5.67 1.85 3.3628 0.61 38430.4 1.22 250.4 0.02 0.7 0 0.4 0.28 0.02 205 2030 6669 6050 32901870 4804890 282780990 530100 1250 960 2 3010 80240 1330 4495 390 132.5 50 260 24011,370 20 201,570 1 9,54013.81%390 1,57015,647260 16.46% 21720 14,483 5,590 254 670 11.99%69 72008,669 22.753623188405861 64 6669 2.0325.7377049180328 6.05 1.57 3.2928 0.53 39270.48 23.92857142857140.96 0.99 0.08 0.670.1 0.39 0.26 0.02 206 1960 6676 5940 33101850 4005190 282870390 380 0 2000 280 1 6220 10400 3610 1390 370 380 0 210 28011,210 10 10 790 1 9,9107.05%370 67015,647210 6.76% 010 14,496 11,830 13 590 4.99%139 16,2557198 5.683453237410077 76 6676 1.9695.7142857142857 5.94 0.67 3.3128 0.38 80130.4 21.07142857142860.28 0.39 0.01 0.59 0 0.37 0.21 0.01 207 1950 6676 6100 32901840 4805200 292870750 360 0 1520 140 3 3720 10160 1940 1250 570 360 01090 14011,340 20 101,230 7 81.42857142857149,91010.85% 155.71428571428651015,656 2.857142857142865.15% 9 14,496 7,180 4 1,680 23.40%48 11,3297200 2 25.62524 6676 1.95 255 6.1 0.51 3.2929 0.36 49940.48 57.93103448275860.14 0.75 0.01 1.68 0 0.57 1.09 0.02 208 1970 6703 5680 33801850 4305230 292860510 390 0 1500 190 1 3950 190430 2030 3510 420 130 0 28063.333333333333311,030 63.33333333333330 940 2 9,9408.52%210 77015,659140 7.75% 3 0 14,537 7,480 63 700 9.36%1 11,5017201 34 94029 6703 1.9722.6470588235294 5.68 0.77 3.3829 0.39 51100.43 24.13793103448280.19 0.51 0.19 0.7 0 0.42 0.28 0 209 1960 6766 5950 35401860 3605330 302980230 47040 1220 1020 1 3070 360360 1460 6230 45078.3333333333333 40 540 17011,450 30 60 630 5 10,1705.50% 90 1,85015,708108 18.19% 65 6 14,682 5,750 459 1,020 17.74%32 72219,046 11719.687597 6766 1.9615.8119658119658 5.95 1.85 3.5430 0.47 40460.36 1.02 0.2334 0.36 1.020.04 0.45 0.54 0.03 210 1910 6766 5630 34001890 5005050 302950970 34060 1680 80 2 4370 10250 2490 1485 520 340 30 690 10,94080 30 101,530 1 9,89013.99%520 43015,752690 4.35% 19630 14,686 8,540 4 1,240 14.52%20 13,5297237 2 76.563 6766 1.91 215 5.63 0.43 3.430 0.34 60220.5 41.33333333333330.08 0.97 0.01 1.240.06 0.52 0.69 0.03 211 1930 6766 5880 33801850 4305390 312990630 46060 1730 400 1 4320 50430 2610 1630 410 460 60 460 40011,190 60 501,120 1 10,23010.01%410 91015,766460 8.90% 22460 14,686 8,660 4 930 10.74%4 13,5677241 2 28072 6766 1.93 455 5.88 0.91 3.3831 0.46 60420.43 0.4 0.6330 0.05 0.930.06 0.41 0.46 0.06 212 1870 6872 5600 33701920 4605160 313030520 650 0 2040 1570 1 6340 150460 3490 7520 44092.8571428571429 0 440224.28571428571410,840 21.42857142857140 980 3 146.66666666666710,1109.04% 146.6666666666672,37015,770 23.44% 4 0 14,876 11,870 270 880 7.41%2 15,8037241 136 49014 6872 1.8717.4264705882353 5.6 2.37 3.3731 0.65 76310.46 28.38709677419351.57 0.52 0.15 0.88 0 0.44 0.44 0 213 1940 6874 6130 34001890 3405410 31296070 360 0 2110 300 1 6230 0340 3480 2 70 370 180 0 110 15011,470 10 0 410 2 10,2603.57%185 66015,766 55 6.43% 10 5 14,876 11,820 4 490 4.15%19 16,0887244 21.57894736842118 35 6874 1.94 82.5 6.13 0.66 3.431 0.36 78480.34 15.80645161290320.3 0.07 0 0.49 0 0.37 0.11 0.01 214 1970 6883 6070 34201900 3305290 32284060 340 0 1760 60 1 4710 10330 2550 5 60 390 68 0 260 11,46012 20 2 390 1 10,0303.40%390 41015,757260 4.09% 920 14,467 9,020 34 670 7.43%8 13,6397238 70 48.7572 6883 1.975.85714285714286 6.07 0.41 3.4232 0.34 61570.33 0.0620.93750.06 0.01 0.67 0 0.39 0.26 0.02 215 1980 6885 5680 33301880 50100 3228400 340 0 1960 50 0 5950 0 3540 1 360 340 230 10,99050 40 0 0 2 9,730 180 39015,757115 4.01% 020 14,468 11,450 3 630 5.50%0 15,7087238 24 65 6885 1.98 16.25 5.68 0.39 3.3332 0.34 72210 0.0519.68750 0 0.63 0 0.36 0.23 0.04 216 1920 6885 5670 34001900 3504990 32283050 370 0 2570 130 1 7180 10350 3560 1 50 370 370 0 240 13010,990 20 10 400 1 9,7203.64%370 51015,756240 5.25% 320 14,452 13,310 94 630 4.73%3 16,1927237 133.33333333333334 55 6885 1.92 15 5.67 0.51 3.432 0.37 79550.35 0.1319.68750.05 0.01 0.63 0 0.37 0.24 0.02 217 1970 6887 6090 34101910 4105000 332820470 390 0 1330 270 2 3390 90205 1610 3235 370 130 0 250 11,47090 10 30 880 2 9,7307.67%185 75015,746125 7.71% 10 5 14,477 6,330 663 630 9.95%4 72379,750 236 22036 6887 1.973.17796610169492 6.09 0.75 3.4133 0.39 43430.41 19.09090909090910.27 0.47 0.09 0.63 0 0.37 0.25 0.01 218 1910 6948 5710 34401890 3905280 333130240 65030 1470 1740 2 3990 230195 2030 2120 410 325 15 350 87011,060 0 115 660 2 10,3005.97%205 2,62015,822175 25.44% 110 0 15,082 7,490 536 760 10.15%45 11,4927272 14.6666666666667125 5 6948 1.91 20.96 5.71 2.62 3.4433 0.65 50880.39 23.0303030303031.74 0.24 0.23 0.760.03 0.41 0.35 0 219 1990 6948 5960 34101950 10705670 3336003160 360300 1450 8031 395034.516129032258110 2000116.1290322580651 360 9.67741935483871360 190 11,36080 20 104,970 3 10,78043.75%120 63.333333333333345015,600 6.666666666666674.17%2,394 15,083 7,400 3 570 7.70%3110 11,4927328 1.598070739549842 72 6948 1.99 225 5.96 0.45 3.4133 0.36 50891.07 17.27272727272730.08 3.6 0.01 0.570.3 0.36 0.19 0.02 220 2050 6948 6250 33901900 4305630 333150560 390170 1680 160 6 466071.66666666666670 256093.33333333333332 360 28.3333333333333195 310 11,69080 40 1,1600 1 10,6809.92%360 55015,643310 5.15% 43140 15,088 8,900 5 710 7.98%177 13,4487379 6.553672316384182 139 6948 2.05 275 6.25 0.55 3.3933 0.39 59840.43 21.51515151515150.16 0.56 0 0.710.17 0.36 0.31 0.04 221 2000 6948 6020 33801990 3405470 33307090 46010 1880 680 1 4560 0340 2670 1 90 470 460 10 650 68011,400 120 0 440 1 10,5303.86%470 1,14015,631650 10.83% 40120 15,096 9,110 8 1,240 13.61%58 14,1717377 7.5862068965517210 67 6948 2 114 6.02 1.14 3.3833 0.46 65380.34 37.57575757575760.68 0.09 0 1.240.01 0.47 0.65 0.12 222 1920 6956 5370 30101910 3904990 332810330 590150 2150 990 2 6280 20195 3530 2165 360 295 75 230 49510,300 30 10 870 1 9,7108.45%360 1,60015,016230 16.48% 77930 14,447 11,960 183 620 5.18%270 16,2257119 3.22222222222222341 124 6956 1.924.69208211143695 5.37 1.6 3.0133 0.59 79610.39 18.78787878787880.99 0.33 0.02 0.620.15 0.36 0.23 0.03 223 1910 6964 5000 28201900 3905310 333090270 41090 2250 570 2 6790 110195 3600 2135 410 205 45 270 2859,730 0 55 750 2 10,3007.71%205 1,09014,477135 10.58% 663 0 15,126 12,640 250 680 5.38%236 16,9636887 3.1779661016949224 18 6964 1.9145.4166666666667 5 1.09 2.8233 0.41 84190.39 20.60606060606060.57 0.27 0.11 0.680.09 0.41 0.27 0 224 1900 6998 5290 28401930 3405660 34316060 37010 1850 200 1 5230 20340 2860 3 60 390123.333333333333 10 19066.666666666666710,030 1906.66666666666667 410 1 10,7504.09%390 59014,467190 5.49% 34190 15,211 9,940 85 770 7.75%70 14,5376883 5.8571428571428634 63 6998 1.917.3529411764706 5.29 0.59 2.8434 0.37 67030.34 22.64705882352940.2 0.06 0.02 0.770.01 0.39 0.19 0.19 225 1880 7024 5010 28401920 3405400 34313050 360 0 1900 510 1 4990 100340 2830 1 50 370 360 0 130 5109,730 10 100 390 1 10,4504.01%370 97014,468130 9.28% 310 15,270 9,720 199 510 5.25%24 14,4526885 26 16.2594 7024 1.8837.3076923076923 5.01 0.97 2.8434 0.36 68850.34 0.51 0.0515 0.1 0.51 0 0.37 0.13 0.01 226 1900 7030 4990 28301930 3705380 343110130 37010 1930 410 1 5660 70370 3160 1130 370 370 10 200 4109,720 20 70 510 1 10,4205.25%370 85014,452200 8.16% 9420 15,274 10,750 192 590 5.49%34 15,2116885 22 1585 7030 1.938.6363636363636 4.99 0.85 2.8334 0.37 69980.37 17.35294117647060.41 0.13 0.07 0.590.01 0.37 0.2 0.02 35 1350 2970 1380 400 290 20 1 400 290 20 5,700 710 12.46% 8,515 120 35 3864 20.2857142857143 0.71 0.4 0.29 0.02 227 1910 7063 4990 28101930 5905810 3270990 38020 450 8 73.7550 123.751 3802.5 4509,710 501,600 11,01016.48% 88014,447 7.99% 183 15,336 106 341 6956 4.6920821114369569 7063 1.9112.7536231884058 4.99 0.88 2.81 0.38 0.59 0.45 0.99 0.05 0.02 36 1460 3930 2080 550 970 0 2 275 485 0 7,470 1,520 20.35% 11,446 18 36 5041 42.2222222222222 1.52 0.55 0.97 0 228 1970 7071 5710 34201950 4805750 15403160 400780 390 3 0160 513.3333333333332 200260 19511,100 2,8000 10,86025.23% 79015,718 7.27%1,323 15,350 14 567 7507 4.9382716049382712 7071 1.9765.8333333333333 5.71 0.79 3.42 0.4 0.48 0.39 1.54 0 0.78 36 1750 4830 2620 560 1130 130 1 560 1,130 130 9,200 1,820 19.78% 13,707 507 36 6216 50.5555555555556 1.82 0.56 1.13 0.13 229 2080 7115 6270 34401910 5505440 11503260 400110 330 6 91.6666666666667220 191.6666666666671 18.3333333333333400 33011,790 2201,810 10,61015.35% 95015,783 8.95% 481 15,419 73 341 7616 5.3079178885630552 7115 2.0818.2692307692308 6.27 0.95 3.44 0.4 0.55 0.33 1.15 0.22 0.11 36 1800 4930 2570 350 90 0 2 175 45 0 9,300 440 4.73% 14,172 1 36 6504 12.2222222222222 0.44 0.35 0.09 0 230 1910 7116 6100 34901950 4505740 3240750 390520 470 2 60225 1375 390260 47011,500 601,720 10,93014.96% 92015,788 8.42% 809 15,429 152 456 7620 3.77192982456147 7116 1.91131.428571428571 6.1 0.92 3.49 0.39 0.45 0.47 0.75 0.06 0.52 7119 1980 5980 363290 390 2130 410 6400 10 3460 1 450 390 400 410 0 10 1 11,250 450 810 400 7.20% 0 15,450 11,990 21 850 7.09% 15,791 45 6 7119 18 0.81 36 0.39 7642 23.61111111111110.41 0.01 0.85 0.45 0.4 0 231 2110 6180 3430 340 50 0 1 340 50 0 11,720 390 3.33% 15,786 2 4 7620 97.5 2.11 6.18 3.43 0.34 0.05 0 7119 1920 5370 373010 390 1300 330 3180 150 1360 1 430 390 390 330 20 150 3 143.33333333333310,300 870 130 6.666666666666678.45% 15,016 5,840 779 840 14.38% 8,564 270 139 7119 3.22222222222222 0.87 37 0.39 3877 22.70270270270270.33 0.15 0.84 0.43 0.39 0.02 232 1960 5890 3330 360 170 0 1 360 170 0 11,180 530 4.74% 15,787 11 77 7599 6.88311688311688 1.96 5.89 3.33 0.36 0.17 0 7141 1920 5630 373280 410 1320 450 3340 30 1510 2 410 205 380 225 190 15 1 10,830 410 890 380 8.22% 190 15,513 6,170 63 980 15.88% 9,407 24 74 7141 37.0833333333333 0.89 37 0.41 4190 26.48648648648650.45 0.03 0.98 0.41 0.38 0.19 233 2060 6420 3480 410 360 0 3 136.666666666667 120 0 11,960 770 6.44% 15,787 2 225 7608 3.42222222222222 2.06 6.42 3.48 0.41 0.36 0 7148 1850 5800 373290 470 1460 940 3740 70 2130 1 570 470 1120 940 20 70 1 10,940 570 1,480 1,120 13.53% 20 15,529 7,330 192 1,710 23.33% 11,431 101 50 7148 14.6534653465347 1.48 37 0.47 5041 46.21621621621620.94 0.07 1.71 0.57 1.12 0.02 234 2060 6390 3490 340 90 0 1 340 90 0 11,940 430 3.60% 15,789 12 144 7642 2.98611111111111 2.06 6.39 3.49 0.34 0.09 0 7148 1920 5690 373370 380 1560 360 4460 0 2360 2 510 190 710 180 0 0 2 10,980 255 740 355 6.74% 0 15,526 8,380 5 1,220 14.56% 12,732 2 3 7148 370 0.74 37 0.38 5658 32.9729729729730.36 0 1.22 0.51 0.71 0 235 2130 6400 3460 450 400 0 2 225 200 0 11,990 850 7.09% 15,791 6 36 7642 23.6111111111111 2.13 6.4 3.46 0.45 0.4 0 7149 1850 5730 383340 380 1830 380 4730 0 2520 1 550 380 790 380 0 0 1 10,920 550 760 790 6.96% 0 15,533 9,080 7 1,340 14.76% 13,902 1 11 7149 760 0.76 38 0.38 6342 35.26315789473680.38 0 1.34 0.55 0.79 0 236 2040 6340 3490 440 440 0 2 220 220 0 11,870 880 7.41% 15,803 14 31 7631 28.3870967741935 2.04 6.34 3.49 0.44 0.44 0 7163 2000 5790 383250 410 2190 520 6460 60 3470 3 510136.666666666667 810173.333333333333 200 20 1 11,040 510 990 810 8.97% 200 15,566 12,120 143 1,520 12.54% 16,336 14 49 7163 70.7142857142857 0.99 38 0.41 8123 0.52 40 0.06 1.52 0.51 0.81 0.2 237 2090 6570 3550 630 1200 80 8 78.75 150 10 12,210 1,910 15.64% 15,971 488 245 7760 7.79591836734694 2.09 6.57 3.55 0.63 1.2 0.08 7167 1930 5620 393220 340 1560 50 4390 10 2410 1 370 340 100 50 10 10 1 10,770 370 400 100 3.71% 10 15,572 8,360 24 480 5.74% 12,729 4 34 7167 100 0.4 39 0.34 5655 12.30769230769230.05 0.01 0.48 0.37 0.1 0.01 238 2190 6420 3540 350 160 0 1 350 160 0 12,150 510 4.20% 15,969 4 2 7760 255 2.19 6.42 3.54 0.35 0.16 0 7170 1940 5640 393390 440 1660 540 4840 0 2360 1 500 440 790 540 60 0 1 10,970 500 980 790 8.93% 60 15,572 8,860 0 1,350 15.24% 12,827 39 187 7170 25.1282051282051 0.98 39 0.44 5715 34.61538461538460.54 0 1.35 0.5 0.79 0.06 239 2090 6250 3470 380 190 0 1 380 190 0 11,810 570 4.83% 15,969 8 16 7760 35.625 2.09 6.25 3.47 0.38 0.19 0 7171 1980 5670 393360 400 1940 400 5640 0 3390 2 440 200 540 200 0 0 0 11,010 800 7.27% 15,578 10,970 6 980 8.93% 15,572 1 0 7171 800 0.8 39 0.4 7170 25.12820512820510.4 0 0.98 0.44 0.54 0 240 2090 6240 3520 600 1580 220 7 85.7142857142857 225.714285714286 31.4285714285714 11,850 2,400 20.25% 16,044 919 295 7789 8.13559322033898 2.09 6.24 3.52 0.6 1.58 0.22 7198 1960 5940 393310 400 1980 390 6180 0 3520 1 380 400 180 390 0 0 1 11,210 380 790 180 7.05% 0 15,647 11,680 0 560 4.79% 16,227 139 15 7198 5.68345323741007 0.79 39 0.4 8018 14.35897435897440.39 0 0.56 0.38 0.18 0 241 2110 6500 3510 400 460 10 5 80 92 2 12,120 870 7.18% 16,049 59 54 7804 16.1111111111111 2.11 6.5 3.51 0.4 0.46 0.01 7200 2030 6050 423290 480 1350 990 3440 100 1650 1 420 480 380 990 20 100 2 11,370 210 1,570 190 13.81% 10 15,647 6,440 217 820 12.73% 9,870 69 50 7200 22.7536231884058 1.57 42 0.48 4379 19.52380952380950.99 0.1 0.82 0.42 0.38 0.02 242 1990 6240 3520 360 170 0 2 180 85 0 11,750 530 4.51% 16,049 6 26 7792 20.3846153846154 1.99 6.24 3.52 0.36 0.17 0 7200 1950 6100 423290 480 1750 750 4880 0 2670 1 380 480 390 750 50 0 2 11,340 190 1,230 195 10.85% 25 15,656 9,300 9 820 8.82% 13,846 48 260 7200 25.625 1.23 42 0.48 6268 19.52380952380950.75 0 0.82 0.38 0.39 0.05 243 2000 6370 3460 420 760 140 2 210 380 70 11,830 1,320 11.16% 16,054 425 71 7791 18.5915492957746 2 6.37 3.46 0.42 0.76 0.14 7201 1970 5680 443380 430 1480 510 3990 0 2020 1 430 430 430 510 10 0 31 13.870967741935511,030 13.8709677419355940 0.322580645161298.52% 15,659 7,490 3 870 11.62% 11,484 1 67 7201 940 0.94 44 0.43 5066 19.77272727272730.51 0 0.87 0.43 0.43 0.01 244 2120 6310 3510 430 370 0 3 143.333333333333 123.333333333333 0 11,940 800 6.70% 16,063 23 103 7768 7.76699029126214 2.12 6.31 3.51 0.43 0.37 0 7221 1960 5950 453540 360 1350 230 3510 40 1710 0 490 740 230 6 81.666666666666711,450 123.333333333333630 38.33333333333335.50% 15,708 6,570 65 1,460 22.22% 10,385 32 163 7221 19.6875 0.63 45 0.36 4574 32.44444444444440.23 0.04 1.46 0.49 0.74 0.23 245 2080 6390 3450 420 640 110 4 105 160 27.5 11,920 1,170 9.82% 16,110 417 172 7803 6.80232558139535 2.08 6.39 3.45 0.42 0.64 0.11 7237 1910 5630 453400 500 1980 970 5980 60 3290 1 390 500 410 970 10 60 1 10,940 390 1,530 410 13.99% 10 15,752 11,250 196 810 7.20% 15,450 20 21 7237 76.5 1.53 45 0.5 7119 0.97 18 0.06 0.81 0.39 0.41 0.01 246 1970 6040 3420 350 60 0 1 350 60 0 11,430 410 3.59% 16,111 1 6 7803 68.3333333333333 1.97 6.04 3.42 0.35 0.06 0 7237 1920 5670 453400 350 1910 50 5710 0 3440 2 390 175 240 25 30 0 2 10,990 195 400 120 3.64% 15 15,756 11,060 3 660 5.97% 15,822 3 110 7237 133.333333333333 0.4 45 0.35 7272 14.66666666666670.05 0 0.66 0.39 0.24 0.03 247 1970 6120 3430 410 380 20 3 136.666666666667 126.666666666667 6.66666666666667 11,520 810 7.03% 16,149 92 94 7822 8.61702127659574 1.97 6.12 3.43 0.41 0.38 0.02 7237 1970 6090 453410 410 2180 470 6510 0 3620 2 450 205 580 235 110 0 2 11,470 225 880 290 7.67% 55 15,746 12,310 10 1,140 9.26% 16,410 4 228 7237 220 0.88 45 0.41 8157 25.33333333333330.47 0 1.14 0.45 0.58 0.11 248 2040 6330 3430 340 50 0 1 340 50 0 11,800 390 3.31% 16,149 2 2 7822 195 2.04 6.33 3.43 0.34 0.05 0 7238 1970 6070 463420 330 1220 60 3020 0 1440 31 39010.6451612903226 2801.93548387096774 40 0 1 11,460 390 390 280 3.40% 40 15,757 5,680 9 710 12.50% 8,595 8 127 7238 48.75 0.39 46 0.33 3884 15.43478260869570.06 0 0.71 0.39 0.28 0.04 249 2110 6320 3500 430 420 80 3 143.333333333333 140 26.6666666666667 11,930 930 7.80% 16,061 566 172 7836 5.40697674418605 2.11 6.32 3.5 0.43 0.42 0.08 249 2110 7238 6320 35001980 4305680 463330420 080 1730 0 3 4760143.3333333333330 2630 6140 430 26.66666666666670 450 11,9300 70 0 930 1 10,9907.80%430 16,0610 450 56670 15,757 9,120 0 950 10.42%172 13,5197836 5.406976744186050 328 7238 2.11 6.32 0 3.546 0 60620.43 20.65217391304350 0.42 0 0.950.08 0.43 0.45 0.07 250 2160 6390 3510 370 200 0 1 370 200 0 12,060 570 4.73% 16,061 0 2 7836 285 2.16 6.39 3.51 0.37 0.2 0 250 2160 7241 6390 35101930 3705880 473380200 430 0 2270 630 1 6560 60370 3470 1200 430 430 0 440 63012,060 0 60 570 1 11,1904.73%430 1,12016,061440 10.01% 0 0 15,766 12,300 224 870 7.07%2 16,3967836 4 28544 7241 2.16 280 6.39 1.12 3.5147 0.43 81520.37 18.51063829787230.63 0.2 0.06 0.87 0 0.43 0.44 0 251 2130 6380 3530 400 390 0 1 400 390 0 12,040 790 6.56% 16,055 6 2 7836 395 2.13 6.38 3.53 0.4 0.39 0 251 2130 7241 6380 35301870 4005600 483370390 460 0 1820 520 1 4590 0400 2580 2390 340 230 0 130 26012,040 20 0 790 8 10,8406.56%42.5 98016,05516.25 9.04% 62.5 15,770 8,990 4 490 5.45%2 13,6807836 2 395107 7241 2.13 490 6.38 0.98 3.5348 0.46 61820.4 10.20833333333330.52 0.39 0 0.49 0 0.34 0.13 0.02 252 2100 6400 3660 350 110 0 1 350 110 0 12,160 460 3.78% 16,071 16 13 7845 35.3846153846154 2.1 6.4 3.66 0.35 0.11 0 252 2100 7244 6400 36601940 3506130 483400110 340 0 1950 70 1 6100 0350 3290 2110 480 170 0 750 12,16035 0 0 460 3 11,4703.78%160 41016,071250 3.57% 16 0 15,766 11,340 10 1,230 10.85%13 15,6567845 35.384615384615419 9 7244 2.121.5789473684211 6.4 0.41 3.6648 0.34 72000.35 0.07 25.6250.11 0 1.23 0 0.48 0.75 0 253 2340 6660 3510 340 70 0 1 340 70 0 12,510 410 3.28% 16,075 8 12 7845 34.1666666666667 2.34 6.66 3.51 0.34 0.07 0 253 2340 7272 6660 35101910 3405710 49344070 390 0 1230 240 1 2920 30340 1330 1 70 420 390 0 380 24012,510 10 30 410 6 11,0603.28% 70 63.333333333333366016,075 1.666666666666675.97% 8 15,822 5,480 110 810 14.78%12 78458,645 34.166666666666745 50 7272 2.3414.6666666666667 6.66 0.66 3.5149 0.39 39190.34 16.5306122448980.24 0.07 0.03 0.81 0 0.42 0.38 0.01 254 1990 6050 3370 360 180 0 1 360 180 0 11,410 540 4.73% 16,075 4 8 7845 67.5 1.99 6.05 3.37 0.36 0.18 0 254 1990 7328 6050 33701990 3605960 503410180 1070 0 1980 3600 1 6050 300360 3520 1180 430 1,070 0 590 3,60011,410 50 300 540 2 11,3604.73%215 4,97016,075295 43.75% 425 15,600 11,550 2,394 1,070 9.26%8 16,2847845 3110 67.5286 7328 1.991.59807073954984 6.05 4.97 3.3750 1.07 80550.36 3.6 21.40.18 0.3 1.07 0 0.43 0.59 0.05 255 2110 6230 3480 370 110 10 1 370 110 10 11,820 490 4.15% 16,088 35 31 7848 15.8064516129032 2.11 6.23 3.48 0.37 0.11 0.01 255 2110 7377 6230 34802000 3706020 513380110 34010 2290 90 1 6690 10370 3610 1110 500 340 10 790 11,82090 30 10 490 1 11,4004.15%500 44016,088790 3.86% 3530 15,631 12,590 40 1,320 10.48%31 17,0457848 15.806451612903258 135 7377 2.117.58620689655172 6.23 0.44 3.4851 0.34 84610.37 25.88235294117650.09 0.11 0.01 1.320.01 0.5 0.79 0.03 256 2020 6070 3360 330 50 0 1 330 50 0 11,450 380 3.32% 16,092 4 4 7850 95 2.02 6.07 3.36 0.33 0.05 0 256 2020 7379 6070 33602050 3306250 52339050 430 0 1910 560 1 5440 170330 3260 8 50 400 53.75 0 330 11,45070 220 21.25 380 1 11,6903.32%400 1,16016,092330 9.92% 4220 15,643 10,610 431 950 8.95%4 15,4197850 177 9573 7379 2.026.55367231638418 6.07 1.16 3.3652 0.43 71150.33 18.26923076923080.56 0.05 0.17 0.95 0 0.4 0.33 0.22 257 2000 6130 3420 340 40 0 1 340 40 0 11,550 380 3.29% 16,091 13 13 7849 29.2307692307692 2 6.13 3.42 0.34 0.04 0 257 2000 7507 6130 34201970 3405710 53342040 480 0 1420 1540 1 3730 780340 1860 3 40 430 160 0 590513.33333333333311,550 70 260 380 3 143.33333333333311,1003.29% 196.6666666666672,80016,091 23.333333333333325.23% 13 15,718 7,010 1,323 1,090 15.55%13 10,9647849 29.2307692307692567 175 7507 4.938271604938272 6.13 2.8 3.4253 0.48 48380.34 20.56603773584911.54 0.04 0.78 1.09 0 0.43 0.59 0.07 258 2140 6340 3540 320 60 10 1 320 60 10 12,020 390 3.24% 16,096 5 5 7850 78 2.14 6.34 3.54 0.32 0.06 0.01 258 2140 7599 6340 35401960 3205890 53333060 36010 1790 170 1 4420 0320 2550 6 60 420 60 10 50028.333333333333312,020 0 0 390 1 11,1803.24%420 53016,096500 4.74% 5 0 15,787 8,760 11 920 10.50%5 13,7057850 77 14178 7599 2.146.88311688311688 6.34 0.53 3.5453 0.36 62170.32 17.35849056603770.17 0.06 0 0.920.01 0.42 0.5 0 259 2000 7608 6160 33402060 3606420 543480130 41010 1540 360 1 3850 0360 2030 2130 400 205 10 310 18011,500 170 0 500 2 11,9604.35%200 77016,096155 6.44% 685 15,787 7,420 2 880 11.86%26 11,4837872 19.2307692307692225 24 7608 3.422222222222222 6.16 0.77 3.3454 0.41 51200.36 16.29629629629630.36 0.13 0 0.880.01 0.4 0.31 0.17 260 2130 7616 6270 35002080 4506270 543440340 55030 2110 1150 4 6500 110112.5 3510 1 85 400 5507.5 460 1,15011,900 10 110 820 2 11,7906.89%200 1,81016,118230 15.35% 148 5 15,783 12,120 481 870 7.18%91 16,0497921 9.01098901098901341 59 7616 2.135.30791788856305 6.27 1.81 3.554 0.55 78040.45 16.11111111111111.15 0.34 0.11 0.870.03 0.4 0.46 0.01 261 1970 7620 6180 34801910 3806100 553490220 450 0 1770 750 2 4480 520190 2530 1110 380 450 0 460 75011,630 70 520 600 8 11,5005.16%47.5 1,72016,12257.5 14.96% 8.758 15,788 8,780 809 910 10.36%17 13,7177920 35.2941176470588456 423 7620 1.973.7719298245614 6.18 1.72 3.4855 0.45 62450.38 16.54545454545450.75 0.22 0.52 0.91 0 0.38 0.46 0.07 262 2170 7620 6220 34402110 3406180 56343080 340 0 1150 50 1 2880 0340 1270 3 80 400113.333333333333 0 40016.666666666666711,830 70 0 420 1 11,7203.55%400 39016,123400 3.33% 170 15,786 5,300 2 870 16.42%6 79208,349 4 14870 7620 2.17 97.5 6.22 0.39 3.4456 0.34 37990.34 15.53571428571430.05 0.08 0 0.87 0 0.4 0.4 0.07 263 2120 7631 6370 39502040 3806340 563490410 44020 1160 440 1 3050 0380 1320 1410 370 440 20 280 44012,440 40 0 810 1 11,8706.51%370 88016,157280 7.41% 3440 15,803 5,530 14 690 12.48%13 79298,426 62.307692307692331 153 7631 2.1228.3870967741935 6.37 0.88 3.9556 0.44 38230.38 12.32142857142860.44 0.41 0 0.690.02 0.37 0.28 0.04 264 2570 7642 7180 35602060 3706390 563490240 34020 1380 90 2 3570 0185 1750 2120 530 170 101070 13,31045 20 0 630 7 75.714285714285711,9404.73% 152.85714285714343016,192 2.857142857142863.60% 55 15,789 6,700 12 1,620 24.18%32 10,4817955 14419.687544 7642 2.572.98611111111111 7.18 0.43 3.5656 0.34 46150.37 28.92857142857140.09 0.24 0 1.620.02 0.53 1.07 0.02 265 2400 7642 6630 36302130 4006400 563460270 45010 1490 400 1 3800 0400 2010 2270 390 225 10 280 20012,660 30 0 680 5 11,9905.37% 78 85016,203 56 7.09% 11 6 15,791 7,300 6 700 9.59%5 11,4767960 36 136188 7642 2.423.6111111111111 6.63 0.85 3.6356 0.45 50700.4 0.4 12.50.27 0 0.70.01 0.39 0.28 0.03 266 2220 7760 6320 35202090 3306570 56355050 630 0 2260 1200 1 6480 80330 3620 8 50 390 78.75 0 350 15012,060 20 10 380 2 12,2103.15%195 1,91016,208175 15.64% 2110 15,971 12,360 488 760 6.15%23 16,2757971 16.5217391304348245 157 7760 2.227.79591836734694 6.32 1.91 3.5256 0.63 80580.33 13.57142857142861.2 0.05 0.08 0.76 0 0.39 0.35 0.02 267 2130 7760 6450 35502190 3506420 583540100 350 0 1320 160 1 3330 0350 1560 1100 430 350 0 450 16012,130 100 0 450 2 12,1503.71%215 51016,211225 4.20% 1150 15,969 6,210 4 980 15.78%23 79549,465 19.56521739130432 140 7760 2.13 255 6.45 0.51 3.5558 0.35 42050.35 16.89655172413790.16 0.1 0 0.98 0 0.43 0.45 0.1 268 2150 7760 6280 35302090 3606250 583470230 38030 1560 190 1 4230 0360 2280 1230 570 380 301090 19011,960 100 0 620 3 11,8105.18%190 363.33333333333357016,225 33.33333333333334.83% 124 15,969 8,070 8 1,760 21.81%33 12,3797961 18.787878787878816 379 7760 2.15 35.625 6.28 0.57 3.5358 0.38 55320.36 30.34482758620690.19 0.23 0 1.760.03 0.57 1.09 0.1 269 2150 7768 6330 34702120 3306310 58351090 430 0 1610 370 1 4370 0330 2370 7 90 43061.4285714285714 0 38052.857142857142911,950 40 0 420 4 11,9403.51%107.5 80016,228 95 6.70% 310 16,063 8,350 23 850 10.18%7 12,7617966 103 17360 7768 2.157.76699029126214 6.33 0.8 3.4758 0.43 57080.33 14.65517241379310.37 0.09 0 0.85 0 0.43 0.38 0.04 270 1990 7789 6040 35502090 3406240 58352060 600 0 2000 1580 1 6020 220340 3380 5 60 340 120 0 90 31611,580 10 44 400 1 11,8503.45%340 2,40016,229 90 20.25% 110 16,044 11,400 919 440 3.86%1 15,6317967 295 40040 7789 1.998.13559322033898 6.04 2.4 3.5558 0.6 73770.34 7.586206896551721.58 0.06 0.22 0.44 0 0.34 0.09 0.01 271 2060 7791 6160 35202000 4106370 593460370 420 0 1200 760 2 2930 140205 1390 2185 400 210 0 240 38011,740 10 70 780 3 133.33333333333311,8306.64% 1,32016,231 80 3.3333333333333311.16% 22 16,054 5,520 425 650 11.78%117 80028,773 6.6666666666666771 66 7791 2.0618.5915492957746 6.16 1.32 3.5259 0.42 39950.41 11.01694915254240.76 0.37 0.14 0.65 0 0.4 0.24 0.01 272 2120 7792 7000 36901990 3506240 603520110 360 0 1760 170 1 4850 0350 2660 2110 400 180 0 460 12,81085 250 0 460 1 11,7503.59%400 53016,233460 4.51% 2250 16,049 9,270 6 1,110 11.97%9 13,8488003 51.111111111111126 140 7792 2.1220.3846153846154 7 0.53 3.6960 0.36 62540.35 0.17 18.50.11 0 1.11 0 0.4 0.46 0.25 273 2000 7803 6220 36102080 3706390 613450210 42010 1870 640 1 4890 110370 2780 3210 530 140 10 960213.33333333333311,830 36.666666666666780 590 3 176.66666666666711,9204.99% 1,17016,255320 26.66666666666679.82% 76 16,110 9,540 417 1,570 16.46%28 14,4838013 21.0714285714286172 254 7803 6.802325581395352 6.22 1.17 3.6161 0.42 66690.37 25.73770491803280.64 0.21 0.11 1.570.01 0.53 0.96 0.08 274 1990 7803 5940 35101970 4606040 613420550 35020 2320 60 4 6780 0115 3820 137.54 460 87.5 5 490 11,44015 30 1,0300 1 11,4309.00%460 41016,239490 3.59% 11630 16,111 12,920 1 980 7.59%150 17,7847969 6.866666666666676 115 7803 1.9968.3333333333333 5.94 0.41 3.5161 0.35 88110.46 16.06557377049180.06 0.55 0 0.980.02 0.46 0.49 0.03 275 1990 7804 5980 34602110 5006500 623510640 40020 1380 460 5 3520 10100 1770 1128 510 400 4 860 46011,430 70 101,160 1 12,12010.15%510 87016,236860 7.18% 13570 16,049 6,670 59 1,440 21.59%166 10,3148013 6.9879518072289254 233 7804 1.9916.1111111111111 5.98 0.87 3.4662 0.4 45430.5 23.22580645161290.46 0.64 0.01 1.440.02 0.51 0.86 0.07 276 1980 7822 6180 35201970 3806120 633430180 410 0 1370 380 1 3590 20380 1710 3180 520136.666666666667 0 990126.66666666666711,680 1206.66666666666667 560 1 11,5204.79%520 81016,227990 7.03% 15120 16,149 6,670 92 1,630 24.44%39 10,4618018 14.358974358974494 394 7822 1.988.61702127659574 6.18 0.81 3.5263 0.41 46150.38 25.87301587301590.38 0.18 0.02 1.63 0 0.52 0.99 0.12 277 1980 7822 6270 37402040 3806330 633430380 340 0 1900 50 1 6040 0380 3560 1380 350 340 0 180 11,99050 30 0 760 1 11,8006.34%350 39016,227180 3.31% 030 16,149 11,500 2 560 4.87%12 16,3628018 63.33333333333332 145 7822 1.98 195 6.27 0.39 3.7463 0.34 81430.38 8.888888888888890.05 0.38 0 0.56 0 0.35 0.18 0.03 278 2280 7836 6180 34402110 3706320 643500240 43030 1430 420 1 3890 80370 2100 3240 550143.333333333333 301140 14011,900 26.666666666666790 640 1 11,9305.38%550 93016,2481,140 7.80% 13790 16,061 7,420 566 1,780 23.99%103 11,4018027 6.21359223300971172 438 7836 2.285.40697674418605 6.18 0.93 3.4464 0.43 50320.37 0.4227.81250.24 0.08 1.780.03 0.55 1.14 0.09 279 1980 7836 6050 35202160 4306390 663510590 37050 2290 200 5 6760 086 3790 1118 390 370 10 410 20011,550 60 1,0700 1 12,0609.26%390 57016,284410 4.73% 28660 16,061 12,840 0 860 6.70%50 17,6728055 2 21.4358 7836 1.98 285 6.05 0.57 3.5266 0.37 87650.43 13.0303030303030.2 0.59 0 0.860.05 0.39 0.41 0.06 280 2160 7836 6300 35002130 3406380 67353070 40010 1580 390 1 4360 0340 2380 1 70 560 400 10 630 39011,960 80 0 420 1 12,0403.51%560 79016,294630 6.56% 1880 16,055 8,320 6 1,270 15.26%4 12,9238057 2 105257 7836 2.16 395 6.3 0.79 3.567 0.4 57520.34 18.9552238805970.39 0.07 0 1.270.01 0.56 0.63 0.08 281 2000 7845 6210 35702100 3706400 673660230 35020 1680 110 2 4630 0185 2460 1115 480 350 10 640 11011,780 0 0 620 1 12,1605.26%480 46016,326640 3.78% 60 0 16,071 8,770 16 1,120 12.77%25 13,4828080 13 24.817 7845 35.38461538461542 6.21 0.46 3.5767 0.35 59980.37 16.71641791044780.11 0.23 0 1.120.02 0.48 0.64 0 282 2260 7845 6480 36202340 3906660 683510350 34020 1820 70 3 5090 0130 2830116.6666666666671 690 6.66666666666667340 1570 12,36070 80 0 760 1 12,5106.15%690 41016,2751,570 3.28% 15780 16,075 9,740 8 2,340 24.02%56 14,3108058 13.571428571428612 254 7845 2.2634.1666666666667 6.48 0.41 3.6268 0.34 66000.39 34.41176470588240.07 0.35 0 2.340.02 0.69 1.57 0.08 283 2170 7845 6650 35801990 3906050 693370280 36010 1930 180 2 5810 0195 3270 1140 380 360 5 450 18012,400 50 0 680 1 11,4105.48%380 54016,275450 4.73% 450 16,075 11,010 4 880 7.99%15 15,3368067 45.33333333333338 106 7845 2.17 67.5 6.65 0.54 3.5869 0.36 70630.39 12.75362318840580.18 0.28 0 0.880.01 0.38 0.45 0.05 284 2270 7848 6630 36202110 3606230 693480250 37010 2030 110 2 6050 10180 3290 1125 480 370 5 990 11012,520 100 10 620 4 11,8204.95%120 49016,307247.5 4.15% 6025 16,088 11,370 35 1,570 13.81%25 15,6478090 31 24.8217 7848 2.2715.8064516129032 6.63 0.49 3.6269 0.37 72000.36 22.75362318840580.11 0.25 0.01 1.570.01 0.48 0.99 0.1 285 1930 7849 6140 34002000 5306130 703420840 34030 1900 40 4 5290 132.50 2840 1210 340 3407.5 60 11,47040 10 1,4000 2 11,55012.21%170 38016,314 30 3.29% 121 5 16,091 10,030 13 410 4.09%90 14,4678090 15.555555555555613 34 7849 1.9329.2307692307692 6.14 0.38 3.470 0.34 68830.53 5.857142857142860.04 0.84 0 0.410.03 0.34 0.06 0.01 286 2160 7850 6390 36202020 4206070 713360420 33020 2000 50 3 6370 0140 3460 1140 420 6.66666666666667330 760 12,17050 140 0 860 1 11,4507.07%420 38016,321760 3.32% 91140 16,092 11,830 4 1,320 11.16%77 16,0548109 11.16883116883124 425 7850 2.16 95 6.39 0.38 3.6271 0.33 77910.42 18.59154929577460.05 0.42 0 1.320.02 0.42 0.76 0.14 287 1890 7850 6090 36102140 3306340 723540160 32010 1290 60 2 3580 10165 1590 1 80 370 320 5 280 11,59060 60 10 500 1 12,0204.31%370 39016,321280 3.24% 2060 16,096 6,460 5 710 10.99%18 81099,860 27.77777777777785 234 7850 1.89 78 6.09 0.39 3.6172 0.32 43810.33 9.861111111111110.06 0.16 0.01 0.710.01 0.37 0.28 0.06 1720 4770 2540 410 630 110 2 205 315 55 9,030 1,150 12.74% 13,587 380 72 6048 15.9722222222222 1.15 0.41 0.63 0.11 288 2180 7872 6360 36002000 3406160 723340130 360 0 130 2 10170 1 65 360 0 13012,140 10 470 11,5003.87% 50016,321 4.35% 0 16,096 6 4 8109 26 117.5 7872 2.1819.2307692307692 6.36 0.5 3.6 0.36 0.34 0.13 0.13 0.01 0 74 1760 4470 2510 660 1370 40 1 660 1,370 40 8,740 2,070 23.68% 14,021 182 74 6454 27.972972972973 2.07 0.66 1.37 0.04 289 2190 7920 6460 34701970 5106180 3480810 380200 220 3 0170 4270 66.666666666666795 12,12055 1,5200 11,63012.54% 60016,336 5.16% 49 16,122 8 38 8123 17 40 7920 2.1935.2941176470588 6.46 0.6 3.47 0.38 0.51 0.22 0.81 0 0.2 74 1800 4720 2750 440 670 110 1 440 670 110 9,270 1,220 13.16% 14,148 254 74 6499 16.4864864864865 1.22 0.44 0.67 0.11 290 2180 7920 6300 35202170 4006220 3440240 34040 80 1 0400 2240 170 40 12,00040 0 680 11,8305.67% 42016,336 3.55% 2 16,123 1 2 8123 6 340 7920 2.18 70 6.3 0.42 3.52 0.34 0.4 0.08 0.24 0 0.04 75 1390 3560 1880 470 520 210 1 470 520 210 6,830 1,200 17.57% 10,735 33 75 4742 16 1.2 0.47 0.52 0.21 291 2170 7921 6490 35502130 3506270 3500240 45020 340 1 30350 1240 450 20 34012,210 30 610 11,9005.00% 82016,298 6.89% 192 16,118 148 241 8130 2.5311203319502191 7921 2.179.01098901098901 6.49 0.82 3.55 0.45 0.35 0.34 0.24 0.03 0.02 7929 2120 6370 773950 380 1960 410 5890 20 3330 1 360 380 170 410 0 20 1 12,440 360 810 170 6.51% 0 16,157 11,180 34 530 4.74% 15,787 13 11 7929 62.3076923076923 0.81 77 0.38 7599 6.883116883116880.41 0.02 0.53 0.36 0.17 0 292 2230 6680 3540 390 400 0 1 390 400 0 12,450 790 6.35% 16,299 1 6 8130 131.666666666667 2.23 6.68 3.54 0.39 0.4 0 7954 2130 6450 773550 350 2160 100 6390 0 3620 2 420 175 420 50 20 0 1 12,130 420 450 420 3.71% 20 16,211 12,170 11 860 7.07% 16,321 23 91 7954 19.5652173913043 0.45 77 0.35 8109 11.16883116883120.1 0 0.86 0.42 0.42 0.02 293 2210 6480 3620 340 120 0 1 340 120 0 12,310 460 3.74% 16,299 0 2 8130 230 2.21 6.48 3.62 0.34 0.12 0 7955 2570 7180 783560 370 1620 240 4230 20 2310 1 670 370 1530 240 10 20 1 13,310 670 630 1,530 4.73% 10 16,192 8,160 55 2,210 27.08% 12,927 32 169 7955 19.6875 0.63 78 0.37 5797 28.33333333333330.24 0.02 2.21 0.67 1.53 0.01 294 1970 6040 3640 370 190 0 1 370 190 0 11,650 560 4.81% 16,299 0 14 8118 40 1.97 6.04 3.64 0.37 0.19 0 7960 2400 6630 793630 400 1220 270 2950 10 1410 1 510 400 950 270 90 10 2 12,660 255 680 475 5.37% 45 16,203 5,580 11 1,550 27.78% 8,995 5 304 7960 136 0.68 79 0.4 4036 19.6202531645570.27 0.01 1.55 0.51 0.95 0.09 295 2160 6290 3410 350 80 0 1 350 80 0 11,860 430 3.63% 16,317 26 10 8122 43 2.16 6.29 3.41 0.35 0.08 0 7961 2150 6280 803530 360 1820 230 5120 30 2680 1 680 360 1490 230 100 30 1 11,960 680 620 1,490 5.18% 100 16,225 9,620 124 2,270 23.60% 14,224 33 350 7961 18.7878787878788 0.62 80 0.36 6540 0.23 28.375 0.03 2.27 0.68 1.49 0.1 296 1900 6040 3560 350 180 30 1 350 180 30 11,500 560 4.87% 16,362 145 63 8143 8.88888888888889 1.9 6.04 3.56 0.35 0.18 0.03 7966 2150 6330 803470 330 1890 90 6650 0 3810 1 440 330 640 90 90 0 1 11,950 440 420 640 3.51% 90 16,228 12,350 3 1,170 9.47% 17,847 7 456 7966 60 0.42 80 0.33 8819 0.09 14.625 0 1.17 0.44 0.64 0.09 297 2270 6560 3470 430 440 0 3 143.333333333333 146.666666666667 0 12,300 870 7.07% 16,396 44 47 8152 18.5106382978723 2.27 6.56 3.47 0.43 0.44 0 7967 1990 6040 813550 340 1750 60 4670 0 2520 1 360 340 170 60 0 0 4 11,580 90 400 42.5 3.45% 0 16,229 8,940 1 530 5.93% 13,625 1 3 7967 400 0.4 81 0.34 6149 6.543209876543210.06 0 0.53 0.36 0.17 0 298 2180 6510 3620 450 580 110 2 225 290 55 12,310 1,140 9.26% 16,410 228 45 8157 25.3333333333333 2.18 6.51 3.62 0.45 0.58 0.11 7969 1990 5940 873510 460 1760 550 4590 20 2580 1 690 460 1440 550 350 20 5 11,440 138 1,030 288 9.00% 70 16,239 8,930 116 2,480 27.77% 13,668 150 356 7969 6.86666666666667 1.03 87 0.46 6184 28.50574712643680.55 0.02 2.48 0.69 1.44 0.35 299 2060 6410 3550 380 380 310 4 95 95 77.5 12,020 1,070 8.90% 16,780 370 175 8329 6.11428571428571 2.06 6.41 3.55 0.38 0.38 0.31 7971 2220 6320 893520 330 1760 50 4690 0 2580 2 490 165 560 25 180 0 1 12,060 490 380 560 3.15% 180 16,208 9,030 21 1,230 13.62% 13,931 23 35 7971 16.5217391304348 0.38 89 0.33 6432 13.82022471910110.05 0 1.23 0.49 0.56 0.18 300 2020 6430 3670 370 350 60 3 123.333333333333 116.666666666667 20 12,120 780 6.44% 16,951 307 155 8410 5.03225806451613 2.02 6.43 3.67 0.37 0.35 0.06 8002 2060 6160 903520 410 1930 370 6140 0 3400 1 530 410 840 370 30 0 1 11,740 530 780 840 6.64% 30 16,231 11,470 22 1,400 12.21% 16,314 117 121 8002 6.66666666666667 0.78 90 0.41 8090 15.55555555555560.37 0 1.4 0.53 0.84 0.03 301 2250 6790 3600 410 270 0 3 136.666666666667 90 0 12,640 680 5.38% 16,963 18 33 8419 20.6060606060606 2.25 6.79 3.6 0.41 0.27 0 8003 2120 7000 913690 350 2130 110 6270 0 3500 1 450 350 340 110 30 0 1 12,810 450 460 340 3.59% 30 16,233 11,900 2 820 6.89% 16,118 9 148 8003 51.1111111111111 0.46 91 0.35 7921 9.010989010989010.11 0 0.82 0.45 0.34 0.03 302 2250 6640 3880 410 210 10 2 205 105 5 12,770 630 4.93% 16,968 31 20 8425 31.5 2.25 6.64 3.88 0.41 0.21 0.01 8013 2000 6220 933610 370 1660 210 4240 10 2280 4 360 92.5 240 52.5 40 2.5 5 11,830 72 590 48 4.99% 8 16,255 8,180 76 640 7.82% 12,578 28 202 8013 21.0714285714286 0.59 93 0.37 5615 6.881720430107530.21 0.01 0.64 0.36 0.24 0.04 303 2030 6380 3650 400 330 40 2 200 165 20 12,060 770 6.38% 16,988 34 18 8435 42.7777777777778 2.03 6.38 3.65 0.4 0.33 0.04 8013 1990 5980 943460 500 1650 640 4520 20 2350 5 520 100 1190 128 180 4 1 11,430 520 1,160 1,190 10.15% 180 16,236 8,520 135 1,890 22.18% 13,134 166 699 8013 6.98795180722892 1.16 94 0.5 5904 20.10638297872340.64 0.02 1.89 0.52 1.19 0.18 304 2290 6690 3610 500 790 30 3 166.666666666667 263.333333333333 10 12,590 1,320 10.48% 17,045 135 51 8461 25.8823529411765 2.29 6.69 3.61 0.5 0.79 0.03 8018 1980 6180 943520 380 1970 180 6120 0 3430 1 410 380 380 180 20 0 2 11,680 205 560 190 4.79% 10 16,227 11,520 15 810 7.03% 16,149 39 92 8018 14.3589743589744 0.56 94 0.38 7822 8.617021276595740.18 0 0.81 0.41 0.38 0.02 305 1970 6520 3740 390 400 0 1 390 400 0 12,230 790 6.46% 17,044 1 5 8460 158 1.97 6.52 3.74 0.39 0.4 0 8018 1980 6270 963740 380 1370 380 3490 0 1740 1 510 380 560 380 80 0 3 11,990 170 186.666666666667760 26.66666666666676.34% 16,227 6,600 0 1,150 17.42% 10,039 12 259 8018 63.3333333333333 0.76 96 0.38 4460 11.97916666666670.38 0 1.15 0.51 0.56 0.08 306 2030 6390 3700 400 210 0 2 200 105 0 12,120 610 5.03% 17,050 6 8 8460 76.25 2.03 6.39 3.7 0.4 0.21 0 8027 2280 6180 993440 370 2310 240 6920 30 3790 1 350 370 270 240 60 30 2 11,900 175 640 135 5.38% 30 16,248 13,020 137 680 5.22% 17,729 103 256 8027 6.21359223300971 0.64 99 0.37 8732 6.868686868686870.24 0.03 0.68 0.35 0.27 0.06 307 2030 6520 3740 410 680 140 4 102.5 170 35 12,290 1,230 10.01% 17,456 500 226 8632 5.44247787610619 2.03 6.52 3.74 0.41 0.68 0.14 8055 1980 6050 1003520 430 1740 590 4390 50 2370 5 660 86 1600 118 170 10 2 11,550 330 1,070 800 9.26% 85 16,284 8,500 286 2,430 28.59% 12,952 50 533 8055 21.4 1.07 100 0.43 5837 0.59 24.3 0.05 2.43 0.66 1.6 0.17 308 2070 6570 3760 360 140 10 1 360 140 10 12,400 510 4.11% 17,464 12 8 8632 63.75 2.07 6.57 3.76 0.36 0.14 0.01 8057 2160 6300 1013500 340 1850 70 5800 10 3290 1 470 340 940 70 70 10 4 11,960 117.5 420 235 3.51% 17.5 16,294 10,940 18 1,480 13.53% 15,529 4 192 8057 105 0.42 101 0.34 7148 14.65346534653470.07 0.01 1.48 0.47 0.94 0.07 309 2260 6820 3760 350 90 10 1 350 90 10 12,840 450 3.50% 17,472 12 11 8633 40.9090909090909 2.26 6.82 3.76 0.35 0.09 0.01 8058 2260 6480 1033620 390 1650 350 4450 20 2460 2 540 195 1030 175 60 10 3 12,360 180 343.333333333333760 6.15% 20 16,275 8,560 157 1,630 19.04% 13,280 56 246 8058 13.5714285714286 0.76 103 0.39 5928 15.82524271844660.35 0.02 1.63 0.54 1.03 0.06 310 2280 6740 3770 350 140 0 1 350 140 0 12,790 490 3.83% 17,473 1 2 8633 245 2.28 6.74 3.77 0.35 0.14 0 310 2280 8067 6740 37702170 3506650 1033580140 390 0 2120 280 1 6310 10350 3510 3140 430 130 0 37093.333333333333312,790 3.333333333333330 490 2 12,4003.83%215 68017,473185 5.48% 1 0 16,275 11,940 4 800 6.70%2 16,0638633 15 24523 8067 2.2845.3333333333333 6.74 0.68 3.77103 0.39 77680.35 7.766990291262140.28 0.14 0.01 0.8 0 0.43 0.37 0 311 1860 6590 3840 460 320 0 2 230 160 0 12,290 780 6.35% 17,478 5 4 8635 195 1.86 6.59 3.84 0.46 0.32 0 311 1860 8080 6590 38402000 4606210 1033570320 370 0 2280 230 2 6180 20230 3440 2160 370 185 0 240 11512,290 30 10 780 2 11,7806.35%185 62017,478120 5.26% 515 16,326 11,900 60 640 5.38%4 16,2488635 25 195137 8080 1.86 24.8 6.59 0.62 3.84103 0.37 80270.46 6.213592233009710.23 0.32 0.02 0.64 0 0.37 0.24 0.03 312 2020 6590 3740 350 180 0 1 350 180 0 12,350 530 4.29% 17,472 16 13 8634 40.7692307692308 2.02 6.59 3.74 0.35 0.18 0 312 2020 8090 6590 37402270 3506630 1043620180 360 0 2290 250 1 6930 10350 3860 2180 360 180 0 80 12512,350 0 5 530 3 12,5204.29%120 26.666666666666762017,472 4.95% 16 0 16,307 13,080 60 440 3.36%13 18,0318634 40.769230769230825 5 8090 2.02 24.8 6.59 0.62 3.74104 0.36 88350.35 4.230769230769230.25 0.18 0.01 0.44 0 0.36 0.08 0 313 2270 6730 3750 380 200 0 1 380 200 0 12,750 580 4.55% 17,479 7 8 8634 72.5 2.27 6.73 3.75 0.38 0.2 0 313 2270 8090 6730 37501930 3806140 1073400200 530 0 1410 840 1 3460 30380 1920 4200 520 132.5 0 870 21012,750 30 7.5 580 1 11,4704.55%520 1,40017,479870 12.21% 730 16,314 6,790 121 1,420 20.91%8 10,9688634 90 72.5190 8090 2.2715.5555555555556 6.73 1.4 3.75107 0.53 48450.38 13.27102803738320.84 0.2 0.03 1.42 0 0.52 0.87 0.03 314 2040 6540 3750 350 190 10 1 350 190 10 12,330 550 4.46% 17,473 10 13 8633 42.3076923076923 2.04 6.54 3.75 0.35 0.19 0.01 314 2040 8109 6540 37502160 3506390 1093620190 42010 1750 420 1 4480 20350 2330 3190 510 140 10 650 14012,330 2406.66666666666667 550 1 12,1704.46%510 86017,473650 7.07% 10240 16,321 8,560 91 1,400 16.36%13 12,9908633 42.307692307692377 309 8109 2.0411.1688311688312 6.54 0.86 3.75109 0.42 57810.35 12.84403669724770.42 0.19 0.02 1.40.01 0.51 0.65 0.24 315 2280 6780 3740 370 270 0 2 185 135 0 12,800 640 5.00% 17,479 12 17 8635 37.6470588235294 2.28 6.78 3.74 0.37 0.27 0 315 2280 8109 6780 37401890 3706090 1133610270 330 0 1560 160 2 4050 10185 2250 2135 640 165 01620 12,80080 180 5 640 1 11,5905.00%640 50017,4791,620 4.31% 12180 16,321 7,860 20 2,440 31.04%17 12,4028635 37.647058823529418 826 8109 2.2827.7777777777778 6.78 0.5 3.74113 0.33 55160.37 21.59292035398230.16 0.27 0.01 2.44 0 0.64 1.62 0.18 316 2280 6690 3760 350 200 0 1 350 200 0 12,730 550 4.32% 17,473 10 13 8634 42.3076923076923 2.28 6.69 3.76 0.35 0.2 0 316 2280 8109 6690 37602180 3506360 1143600200 340 0 1560 130 1 4270 0350 2290 2200 620 170 01430 12,73065 160 0 550 1 12,1404.32%620 47017,4731,430 3.87% 10160 16,321 8,120 0 2,210 27.22%13 11,9528634 42.30769230769234 770 8109 2.28 117.5 6.69 0.47 3.76114 0.34 52980.35 19.38596491228070.13 0.2 0 2.21 0 0.62 1.43 0.16 317 2310 6920 3790 350 270 60 2 175 135 30 13,020 680 5.22% 17,729 256 99 8732 6.86868686868687 2.31 6.92 3.79 0.35 0.27 0.06 317 2310 8118 6920 37901970 3506040 1153640270 37060 1160 190 2 2790 0175 1200 3135 390123.333333333333 30 33063.333333333333313,020 70 0 680 1 11,6505.22%390 56017,729330 4.81% 25670 16,299 5,150 0 790 15.34%99 87327,954 6.8686868686868714 163 8118 2.31 40 6.92 0.56 3.79115 0.37 36290.35 6.86956521739130.19 0.27 0 0.790.06 0.39 0.33 0.07 318 2010 6690 3710 320 60 0 1 320 60 0 12,410 380 3.06% 17,737 8 8 8738 47.5 2.01 6.69 3.71 0.32 0.06 0 318 2010 8122 6690 37102160 3206290 117341060 350 0 1860 80 1 5330 0320 2980 1 60 470 350 01020 12,41080 360 0 380 1 11,8603.06%470 43017,7371,020 3.63% 8360 16,317 10,170 26 1,850 18.19%8 14,6828738 10 47.5459 8122 2.01 43 6.69 0.43 3.71117 0.35 67660.32 15.81196581196580.08 0.06 0 1.85 0 0.47 1.02 0.36 319 2300 6840 3820 600 1430 390 9 66.6666666666667 158.888888888889 43.3333333333333 12,960 2,420 18.67% 17,761 546 370 8765 6.54054054054054 2.3 6.84 3.82 0.6 1.43 0.39 319 2300 8123 6840 38202190 6006460 11714303470 510390 2060 810 9 616066.6666666666667200 3520158.8888888888891 410 43.3333333333333510 370 81012,960 0 2002,420 1 12,12018.67%410 1,52017,761370 12.54% 546 0 16,336 11,740 49 780 6.64%370 16,2318765 6.5405405405405438 22 8123 2.3 40 6.84 1.52 3.82117 0.51 80020.6 6.666666666666670.81 1.43 0.2 0.780.39 0.41 0.37 0 320 1880 8123 6710 38102180 3306300 125352070 40010 1890 240 1 5280 40330 3130 1 70 650 400 101740 24012,400 230 40 410 3 216.66666666666712,0003.31% 68017,760580 76.66666666666675.67% 1 16,336 10,300 2 2,620 25.44%4 15,0828765 2 102.5536 8123 1.88 340 6.71 0.68 3.81125 0.4 69480.33 0.24 20.960.07 0.04 2.620.01 0.65 1.74 0.23 321 2290 8130 6830 38002170 3806490 1283550380 350 0 1780 240 2 4820 20190 2520 1190 450 350 0 780 24012,920 80 20 760 2 12,2105.88%225 61017,758390 5.00% 240 16,298 9,120 192 1,310 14.36%4 14,1858765 241 190314 8130 2.292.53112033195021 6.83 0.61 1283.8 0.35 65380.38 0.2410.2343750.38 0.02 1.31 0 0.45 0.78 0.08 322 2290 8130 6860 38202230 3706680 130354090 390 0 1640 400 1 4370 0370 2170 1 90 440 390 0 760 40012,970 90 0 460 4 12,4503.55%110 79017,758190 6.35% 22.54 16,299 8,180 1 1,290 15.77%4 12,1078765 6 115403 8130 2.29131.666666666667 6.86 0.79 3.82130 0.39 53720.37 9.923076923076920.4 0.09 0 1.29 0 0.44 0.76 0.09 323 2290 8130 6760 37902210 3906480 1313620410 34060 1800 120 2 4580 0195 2670 1205 540 340 301020 12012,840 90 0 860 3 12,3106.70%180 46017,672340 3.74% 35830 16,299 9,050 0 1,650 18.23%66 13,9908765 13.0303030303032 752 8130 2.29 230 6.76 0.46 3.79131 0.34 64260.39 12.59541984732820.12 0.41 0 1.650.06 0.54 1.02 0.09 324 2300 8143 6800 37401900 3506040 1333560150 350 0 1400 180 1 3690 30350 1820 1150 530 350 0 810 18012,840 240 30 500 3 176.66666666666711,5003.89% 56017,673270 4.87% 180 16,362 6,910 145 1,580 22.87%1 10,8398766 63 500125 8143 2.38.88888888888889 6.8 0.56 3.74133 0.35 47910.35 11.87969924812030.18 0.15 0.03 1.58 0 0.53 0.81 0.24 325 2320 8152 6780 38202270 4606560 1333470490 43030 1590 440 4 4140 0115 2390 122.53 550143.3333333333337.51130146.66666666666712,920 80 0 980 2 12,3007.59%275 87017,784565 7.07% 11540 16,396 8,120 44 1,760 21.67%61 12,7018811 16.065573770491847 401 8152 2.3218.5106382978723 6.78 0.87 3.82133 0.43 56400.46 13.23308270676690.44 0.49 0 1.760.03 0.55 1.13 0.08 326 1930 8157 6720 38102180 3506510 1343620130 45010 1860 580 1 4670 110350 2590 2130 380 225 10 430 29012,460 70 55 490 2 12,3103.93%190 1,14017,789215 9.26% 1135 16,410 9,120 228 880 9.65%26 13,6228815 18.846153846153845 293 8157 1.9325.3333333333333 6.72 1.14 3.81134 0.45 61410.35 6.567164179104480.58 0.13 0.11 0.880.01 0.38 0.43 0.07 327 2320 8329 6790 38202060 3506410 1363550200 380 0 1920 380 1 5160 310350 3030 4200 650 95 01570 12,93095 150 77.5 550 3 216.66666666666712,0204.25% 523.3333333333331,07017,791 8.90% 250 16,780 10,110 370 2,370 23.44%2 14,8768817 175 275270 8329 2.326.11428571428571 6.79 1.07 3.82136 0.38 68720.35 17.42647058823530.38 0.2 0.31 2.37 0 0.65 1.57 0.15 328 1890 8410 6650 38102020 4406430 1383670640 37090 1760 350 3 4530146.66666666666760 2530213.3333333333333 460123.333333333333 30 680116.66666666666712,350 220 201,170 1 12,1209.47%460 78017,847680 6.44% 456220 16,951 8,820 307 1,360 15.42%80 13,8998819 155 14.625298 8410 1.895.03225806451613 6.65 0.78 3.81138 0.37 63560.44 9.855072463768120.35 0.64 0.06 1.360.09 0.46 0.68 0.22 329 1790 8419 6750 38202250 4106790 1393600310 41010 1960 270 2 5940 0205 3310 3155 400136.666666666667 5 390 12,36090 0 0 730 2 12,6405.91%200 68017,850195 5.38% 3 0 16,963 11,210 18 790 7.05%16 15,6478821 33 45.6250 8419 1.7920.6060606060606 6.75 0.68 3.82139 0.41 71980.41 5.683453237410070.27 0.31 0 0.790.01 0.4 0.39 0 330 2310 8425 6840 37402250 5806640 14013303880 410450 1220 210 5 3150 10116 1450 2266 390 205 90 260 10512,890 140 2,3605 4 12,77018.31%97.5 63018,036 65 4.93% 62435 16,968 5,820 31 790 13.57%254 89378,773 9.2913385826771620 70 8425 2.31 31.5 6.84 0.63 3.74140 0.41 39670.58 5.642857142857140.21 1.33 0.01 0.790.45 0.39 0.26 0.14 331 2290 8435 6930 38602030 3606380 144365080 400 0 2060 330 1 6390 40360 3490 2 80 340 200 0 90 16513,080 0 20 440 1 12,0603.36%340 77018,031 90 6.38% 5 0 16,988 11,940 34 430 3.60%104 15,7898835 4.2307692307692318 12 8435 2.2942.7777777777778 6.93 0.77 3.86144 0.4 76420.36 2.986111111111110.33 0.08 0.04 0.43 0 0.34 0.09 0 332 2050 8460 6670 38001970 3906520 1453740230 390 0 2320 400 1 6980 0390 3900 3230 670 130 01320133.33333333333312,520 310 0 620 1 12,2304.95%670 79018,0301,320 6.46% 1310 17,044 13,200 1 2,300 17.42%6 18,1468835 103.3333333333335 154 8460 2.05 158 6.67 0.79 1453.8 0.39 89560.39 15.86206896551720.4 0.23 0 2.3 0 0.67 1.32 0.31 333 2340 8460 6990 38602030 3706390 1503700140 400 0 1990 210 1 5940 0370 3510 1140 460 400 0 550 21013,190 20 0 510 1 12,1203.87%460 61018,030550 5.03% 220 17,050 11,440 6 1,030 9.00%2 16,2398835 8 255116 8460 2.34 76.25 6.99 0.61 3.86150 0.4 79690.37 6.866666666666670.21 0.14 0 1.03 0 0.46 0.55 0.02 334 1860 8461 6780 38302290 3906690 1553610250 500 0 2020 790 1 6430 30390 3670 2250 370 250 0 350 39512,470 60 15 640 2 12,5905.13%185 1,32018,032175 10.48% 230 17,045 12,120 135 780 6.44%5 16,9518838 51 128307 8461 1.8625.8823529411765 6.78 1.32 3.83155 0.5 84100.39 5.032258064516130.79 0.25 0.03 0.78 0 0.37 0.35 0.06 335 2300 8632 6930 38502030 3706520 1613740180 41010 1750 680 1 4740 140370 2610 4180 620 102.5 101380 17013,080 100 35 560 1 12,2904.28%620 1,23018,0301,380 10.01% 4100 17,456 9,100 500 2,100 23.08%3 13,5538839 186.666666666667226 430 8632 2.35.44247787610619 6.93 1.23 3.85161 0.41 60610.37 13.04347826086960.68 0.18 0.14 2.10.01 0.62 1.38 0.1 336 2320 8632 6980 39002070 6706570 16613203760 360310 1990 140 5 5980 10134 3460 1264 500 360 62 640 14013,200 20 102,300 1 12,40017.42%500 51018,146640 4.11% 15420 17,464 11,430 12 1,160 10.15%145 16,2368956 15.86206896551728 135 8632 2.32 63.75 6.98 0.51 1663.9 0.36 80130.67 6.987951807228920.14 1.32 0.01 1.160.31 0.5 0.64 0.02 337 2290 8633 6920 38502260 4206820 1723760180 35020 2080 90 1 6390 10420 3450 1180 420 350 20 640 13,06090 110 10 620 1 12,8404.75%420 45018,148640 3.50% 10110 17,472 11,920 12 1,170 9.82%7 16,1108959 88.571428571428611 417 8633 2.2940.9090909090909 6.92 0.45 3.85172 0.35 78030.42 6.802325581395350.09 0.18 0.01 1.170.02 0.42 0.64 0.11 338 2310 8633 6890 38802280 4006740 1723770180 35010 2110 140 1 6320 0400 3500 1180 430 350 10 420 14013,080 80 0 590 2 12,7904.51%215 49018,148210 3.83% 840 17,473 11,930 1 930 7.80%2 16,0618959 2 295566 8633 2.31 245 6.89 0.49 3.88172 0.35 78360.4 5.406976744186050.14 0.18 0 0.930.01 0.43 0.42 0.08 339 2300 8633 6900 38002040 3606540 1753750150 35010 2060 190 1 6410 10360 3550 2150 380 175 10 380 13,00095 310 5 520 1 12,3304.00%380 55018,149380 4.46% 27310 17,473 12,020 10 1,070 8.90%7 16,7808960 74.285714285714313 370 8633 2.342.3076923076923 6.9 0.55 1753.8 0.35 83290.36 6.114285714285710.19 0.15 0.01 1.070.01 0.38 0.38 0.31 340 2000 8634 6570 38102020 3606590 1773740140 35010 2050 180 1 6250 0360 3390 1140 430 350 10 560 18012,380 170 0 510 2 12,3504.12%215 53018,150280 4.29% 185 17,472 11,690 16 1,160 9.92%2 15,6438960 13 255431 8634 40.76923076923082 6.57 0.53 3.81177 0.35 73790.36 6.553672316384180.18 0.14 0 1.160.01 0.43 0.56 0.17 341 2280 8634 6760 38002270 3606730 1833750200 38040 1410 200 1 3610 0360 1950 1200 690 380 401600 20012,840 460 0 600 1 12,7504.67%690 58018,1881,600 4.55% 150460 17,479 6,970 7 2,750 39.45%27 11,3098967 22.22222222222228 667 8634 2.28 72.5 6.76 0.58 1833.8 0.38 49830.36 15.02732240437160.2 0.2 0 2.750.04 0.69 1.6 0.46 342 1990 8634 6710 38202280 3706690 1893760150 350 0 1350 200 1 3310 0370 1730 1150 460 350 0 730 20012,520 250 0 520 9 51.111111111111112,7304.15% 81.111111111111155018,191 27.77777777777784.32% 3 17,473 6,390 10 1,440 22.54%5 89709,720 13 104485 8634 1.9942.3076923076923 6.71 0.55 3.82189 0.35 43360.37 7.619047619047620.2 0.15 0 1.44 0 0.46 0.73 0.25 343 2280 8635 6860 38301860 3506590 189384090 46010 1500 320 1 4130 0350 2140 2 90 620 230 101550 16012,970 370 0 450 1 12,2903.47%620 78018,1961,550 6.35% 5370 17,478 7,770 5 2,540 32.69%1 11,8148971 4 450869 8635 2.28 195 6.86 0.78 3.83189 0.46 52400.35 13.43915343915340.32 0.09 0 2.540.01 0.62 1.55 0.37 344 2280 8635 6870 39102280 4306780 1933740270 370 0 1340 270 1 3410 0430 1690 1270 510 370 0 990 27013,060 160 0 700 2 12,8005.36%255 64018,196495 5.00% 080 17,479 6,440 12 1,660 25.78%15 10,3158984 46.666666666666717 586 8635 2.2837.6470588235294 6.87 0.64 3.91193 0.37 45490.43 8.601036269430050.27 0.27 0 1.66 0 0.51 0.99 0.16 345 2350 8732 6930 38902310 4106920 1963790270 350 0 1130 270 2 2770 60205 1170 2135 380 175 0 370 13513,170 100 30 680 1 13,0205.16%380 68018,204370 5.22% 8100 17,729 5,070 256 850 16.77%13 89917,756 52.307692307692399 341 8732 2.356.86868686868687 6.93 0.68 3.89196 0.35 34870.41 4.336734693877550.27 0.27 0.06 0.85 0 0.38 0.37 0.1 8738 2010 6690 2013710 320 1590 60 4310 0 2290 1 600 320 1230 60 60 0 2 12,410 300 380 615 3.06% 30 17,737 8,190 8 1,890 23.08% 12,324 8 230 8738 47.5 0.38 201 0.32 5491 9.402985074626870.06 0 1.89 0.6 1.23 0.06 8765 2300 6840 2063820 600 1270 1430 3140 390 1520 9 44066.6666666666667 660158.888888888889 15043.3333333333333 1 12,960 440 2,420 660 18.67% 150 17,761 5,930 546 1,250 21.08% 9,332 370 684 8765 6.54054054054054 2.42 206 0.6 4158 6.067961165048541.43 0.39 1.25 0.44 0.66 0.15 8765 1880 6710 2193810 330 1100 70 2770 10 1190 1 390 330 230 70 30 10 4 12,400 97.5 410 57.5 3.31% 7.5 17,760 5,060 1 650 12.85% 7,819 4 135 8765 102.5 0.41 219 0.33 3554 2.968036529680370.07 0.01 0.65 0.39 0.23 0.03 8765 2290 6830 2243800 380 1130 380 2560 0 1140 2 380 190 420 190 100 0 1 12,920 380 760 420 5.88% 100 17,758 4,830 2 900 18.63% 7,469 4 424 8765 190 0.76 224 0.38 3336 4.017857142857140.38 0 0.9 0.38 0.42 0.1 8765 2290 6860 2253820 370 2060 90 6420 0 3480 1 410 370 360 90 0 0 1 12,970 410 460 360 3.55% 0 17,758 11,960 4 770 6.44% 15,787 4 2 8765 115 0.46 225 0.37 7608 3.422222222222220.09 0 0.77 0.41 0.36 0 8765 2290 6760 2263790 390 2030 410 6520 60 3740 2 410 195 680 205 140 30 3 136.66666666666712,840 226.666666666667860 46.66666666666676.70% 17,672 12,290 358 1,230 10.01% 17,456 66 500 8765 13.030303030303 0.86 226 0.39 8632 5.442477876106190.41 0.06 1.23 0.41 0.68 0.14 8766 2300 6800 2363740 350 1910 150 5000 0 2820 1 390 350 270 150 90 0 2 12,840 195 500 135 3.89% 45 17,673 9,730 1 750 7.71% 14,477 1 663 8766 500 0.5 236 0.35 6887 3.177966101694920.15 0 0.75 0.39 0.27 0.09 8811 2320 6780 2373820 460 1140 490 2730 30 1340 4 430 115 590 122.5 190 7.5 5 12,920 86 980 118 7.59% 38 17,784 5,210 115 1,210 23.22% 8,235 61 439 8811 16.0655737704918 0.98 237 0.46 3754 5.105485232067510.49 0.03 1.21 0.43 0.59 0.19 8815 1930 6720 2373810 350 1700 130 4370 10 2050 1 630 350 1370 130 120 10 1 12,460 630 490 1,370 3.93% 120 17,789 8,120 11 2,120 26.11% 11,327 26 575 8815 18.8461538461538 0.49 237 0.35 4996 8.945147679324890.13 0.01 2.12 0.63 1.37 0.12 8817 2320 6790 2413820 350 2170 200 6490 0 3550 1 350 350 240 200 20 0 1 12,930 350 550 240 4.25% 20 17,791 12,210 2 610 5.00% 16,298 2 192 8817 275 0.55 241 0.35 8130 2.531120331950210.2 0 0.61 0.35 0.24 0.02 8819 1890 6650 2453810 440 2090 640 6570 90 3550 3 630146.666666666667 1200213.333333333333 80 30 1 12,350 630 1,170 1,200 9.47% 80 17,847 12,210 456 1,910 15.64% 15,971 80 488 8819 14.625 1.17 245 0.44 7760 7.795918367346940.64 0.09 1.91 0.63 1.2 0.08 8821 1790 6750 2513820 410 1390 310 3440 10 1930 2 610 205 1560 155 460 5 1 12,360 610 730 1,560 5.91% 460 17,850 6,760 3 2,630 38.91% 10,708 16 955 8821 45.625 0.73 251 0.41 4714 10.47808764940240.31 0.01 2.63 0.61 1.56 0.46 8835 2290 6930 2543860 360 2310 80 6840 0 3740 5 580 72 1330 16 450 0 1 13,080 580 440 1,330 3.36% 450 18,031 12,890 5 2,360 18.31% 18,036 104 624 8835 4.23076923076923 0.44 254 0.36 8937 9.291338582677160.08 0 2.36 0.58 1.33 0.45 8835 2050 6670 2703800 390 1920 230 5370 0 3010 1 390 390 330 230 150 0 5 12,520 78 620 66 4.95% 30 18,030 10,300 1 870 8.45% 15,016 6 779 8835 103.333333333333 0.62 270 0.39 7119 3.222222222222220.23 0 0.87 0.39 0.33 0.15 8835 2340 6990 2953860 370 2090 140 6240 0 3520 1 600 370 1580 140 220 0 1 13,190 600 510 1,580 3.87% 220 18,030 11,850 2 2,400 20.25% 16,044 2 919 8835 255 0.51 295 0.37 7789 8.135593220338980.14 0 2.4 0.6 1.58 0.22 8838 1860 6780 3413830 390 1910 250 4990 0 2810 1 590 390 990 250 20 0 1 12,470 590 640 990 5.13% 20 18,032 9,710 2 1,600 16.48% 14,447 5 183 8838 128 0.64 341 0.39 6956 4.692082111436950.25 0 1.6 0.59 0.99 0.02 8839 2300 6930 3413850 370 2080 180 6270 10 3440 1 550 370 1150 180 110 10 1 13,080 550 560 1,150 4.28% 110 18,030 11,790 4 1,810 15.35% 15,783 3 481 8839 186.666666666667 0.56 341 0.37 7616 5.307917888563050.18 0.01 1.81 0.55 1.15 0.11 8937 2310 6840 3703740 580 2300 1330 6840 450 3820 1 600 580 1430 1,330 390 450 1 12,890 600 2,360 1,430 18.31% 390 18,036 12,960 624 2,420 18.67% 17,761 254 546 8937 9.29133858267716 2.36 370 0.58 8765 6.540540540540541.33 0.45 2.42 0.6 1.43 0.39 8956 2320 6980 4563900 670 1910 1320 6100 310 3490 5 450 134 750 264 520 62 1 13,200 450 2,300 750 17.42% 520 18,146 11,500 154 1,720 14.96% 15,788 145 809 8956 15.8620689655172 2.3 456 0.67 7620 3.77192982456141.32 0.31 1.72 0.45 0.75 0.52 8959 2290 6920 5673850 420 1970 180 5710 20 3420 1 480 420 1540 180 780 20 1 13,060 480 620 1,540 4.75% 780 18,148 11,100 10 2,800 25.23% 15,718 7 1,323 8959 88.5714285714286 0.62 567 0.42 7507 4.938271604938270.18 0.02 2.8 0.48 1.54 0.78 8959 2310 6890 11023880 400 1570 180 4290 10 2300 1 760 400 2100 180 580 10 1 13,080 760 590 2,100 4.51% 580 18,148 8,160 8 3,440 42.16% 12,428 2 2,555 8959 295 0.59 1102 0.4 5561 3.121597096188750.18 0.01 3.44 0.76 2.1 0.58 8960 2300 6900 31103800 360 1990 150 5960 10 3410 1 1070 360 3600 150 300 10 1 13,000 1,070 520 3,600 4.00% 300 18,149 11,360 27 4,970 43.75% 15,600 7 2,394 8960 74.2857142857143 0.52 3110 0.36 7328 1.598070739549840.15 0.01 4.97 1.07 3.6 0.3 8960 2000 6570 31393810 360 1030 140 2640 10 1070 1 1050 360 2470 140 1060 10 2 12,380 525 510 1,235 4.12% 530 18,150 4,740 1 4,580 0.966244725738397 7,047 2 7,047 -0.1678241044083558960 0.689878474422275255 0.51 3139 0.36 3139 1.459063395985980.14 0.01 4.58 1.05 2.47 1.06 8967 2280 6760 3800 360 200 40 1 360 200 40 12,840 600 4.67% 18,188 150 27 8967 22.2222222222222 0.6 0.36 0.2 0.04 8970 1990 6710 3820 370 150 0 1 370 150 0 12,520 520 4.15% 18,191 3 5 8970 104 0.52 0.37 0.15 0 8971 2280 6860 3830 350 90 10 1 350 90 10 12,970 450 3.47% 18,196 5 1 8971 450 0.45 0.35 0.09 0.01 8984 2280 6870 3910 430 270 0 1 430 270 0 13,060 700 5.36% 18,196 0 15 8984 46.6666666666667 0.7 0.43 0.27 0 8991 2350 6930 3890 410 270 0 2 205 135 0 13,170 680 5.16% 18,204 8 13 8991 52.3076923076923 0.68 0.41 0.27 0 13.31 seconds. Incremental analysis takes between 0.37 and 4.97 seconds. The mean analysis times are 9.75 seconds and 0.96 seconds, with standard deviations of 2.29 and 0.61 seconds, respectively. Incremental analysis takes between 3.06% and 43.75% of the time of a full analysis. The mean ratio between incremental and full analysis is 10.56%. Thus, incremental analysis gives huge performance gains. RQ3) Figures 3.1c and 3.1d shows incremental analysis times per revision, ordered by LOC and changed LOC, respectively. The size of a project does not seem to influence incremental analysis time (correlation coefficient −0.18), but the size of the change does. This is the expected outcome, but more experiments will be needed RQ4) There were 137 revisions which affected only a single file. Incremen- tal analysis takes between 0.37 and 1.12 seconds. There is only one revision where incremental analysis takes longer than one second. The mean incremental analysis time is 0.56 seconds. All analysis times would be acceptable response times in an interactive IDE setting, where analysis is performed in the background without blocking the user interface. Single responses which take slightly more than one second would still be acceptable, if regular responses are fast. Furthermore, changes between two revisions are more coarse grained and should require more re-evaluation than changes in an editing scenario. 3.6.3 Threats to validity An important threat to external validity is that we analyzed only WebDSL applications and only two of them. We are convinced that WebDSL’s name and type analysis is representative for other languages, but our evaluation cannot generalize beyond WebDSL and its sublanguages. Furthermore, other WebDSL applications, particularly those of different size, might show different characteristics. Additional threats are the large distance between revisions and the correctness of revisions. In real-time editing scenarios, distances might be much smaller and revisions might switch between correct and erroneous states. We believe that smaller distances would only be in the benefit of incremental analysis. Erroneous revisions should not affect parse and collection times but evaluation times, which tend to be small. A threat to internal validity is file size. Incremental analysis re-parses and re-collects changed files. Independent of the actual changes inside a file, file size alone can influence parse and collection times. However, we believe that this does not influence the conclusions from any of our research questions. Regarding construct validity, we measured performance using wall-clock time only and control JIT compilation with a warm-up phase. By running the garbage collector between analysis runs, we ensured a similar amount of memory available to all analyses. However, the semantic index and the task engine store large amounts of data (13 MB in the worst case) and may experience garbage collection pauses.

3.7 Related Work We give an overview of other approaches for incremental name and type analysis.

54 3.7.1 IDEs and Language Workbenches IDEs such as Eclipse typically lack a generic framework for the development of incremental analyses, but provide manual implementations of incremental analysis and compilation for popular languages such as Java or C#. Some language workbenches automatically derive incremental analyses. In SugarJ [37], extensions inherit the incremental behaviour of SugarJ, which uses the module system of Java to provide incremental compilation on file-level, but lacks name and type analysis of its host language Java. Xtext [12] leverages incremental analysis and compilation from the Eclipse JDT to user-defined languages, as long as they map to Java concepts. The JDT performs only local analyses on edit and global analyses on save. MPS [159] does not require name binding due to its projectional nature. It supports incremental type analysis but lacks a framework for other incremental analyses. In general, language workbenches lack frameworks for developing incremental analyses. 3.7.2 Attribute Grammars Attribute Grammars (AGs) [83] provide a formal way of specifying the semantics of a context-free language, including name and type analysis. One of the first incremental attribute evaluators is proposed in [25]. It only evaluates changed attributes and propagates evaluation to affected attributes. A similar incremental evaluation algorithm is shown in [164, 165] for ordered attributed grammars [73]. In [122, 124, 121, 72], extensions to propagation are shown that stop propagation if an attribute value is unchanged from its previous attribution. Similar to attribute grammars, our approach exploits static dependencies, caching, and change propagation. Similar to ordered attribute grammars, we assume an evaluation order of tasks. Though tasks can be cyclic, we just do not evaluate them. While attributes are (re-)evaluated in visits to the tree, our collection separates tasks from the tree and they are (re-)evaluated independent of the tree. As a consequence, we do not require incremental parsing techniques and are not restricted to editing modes. For name analysis, attribute grammars typically pass environments throughout the tree. Incremental name analysis suffers from this as a single change in the environment requires a full reevaluation of the aggregated environment and all dependent attributes. In our approach, we have a predefined notion of an environment, the semantic index, which is globally maintained. It enables fine-grained dependency tracking for name and type analysis tasks solely based on changing entries, not on changing environments. 3.7.3 Reference Attribute Grammars A popular extension to attribute grammars is the addition of reference attributes. These simplify the specification of algorithms that require non-local information, including name resolution. Door Attribute Grammars [55, 54] extend attribute grammars with reference attributes and door objects which facilitate analysis of object-oriented languages. A similar but more general extension is shown in [120].

Chapter 3. A Task Engine for Incremental Name and Type Analysis 55 Reference Attribute Grammars (RAGs) [56] are a generalization of door attribute grammars where the door objects are removed. In [130], an incremental evaluator for reference attributed grammars is shown which is used by the JastAdd [33] meta-compilation system. JastAdd also adds parametrized attributes which allow attributes to be parametrized, forming a mapping. The approach is compared to traditional attribute grammars in [131] and shows that the use of reference attribute grammars reduces the number of affected attributes for name and type analysis significantly. Our approach has two mechanisms similar to reference attributes. First, we can refer to binding instances by URIs and can look up their properties in the semantic index. Second, properties and tasks can refer to arbitrary other tasks. Reference attribute grammars discover dependencies during evaluation. We detect inter-task dependencies after collection. This already helps in establishing an ordering for evaluation. Only dependencies from properties to tasks are discovered during evaluation. Similar to ordinary attribute grammars, reference attribute grammars also do not provide a solution for aggregate attributes. Some attribute grammar formalisms take a functional approach to evaluation. In [115] attributes are evaluated using visit-functions with memoization. A more general extension to attribute grammars is the higher order attribute grammar [157, 139] for which an incremental evaluator is presented in [156]. Similar to this approach, our approach employs a global cache and uses hash consing to efficiently share tasks and to make look-ups into the cache extremely fast. Tasks can also be seen as functions, but the evaluation strategy differs. Visit-functions are still applied on subtrees while tasks are completely separated from the tree. 3.7.4 Other Approaches Pregmatic [13] is an incremental program environment generator that uses extended affix grammars for specification. It uses an incremental propagation algorithm similar to the one used by attribute grammar approaches which were discussed earlier. Instead of separating parsing and semantic analysis, all evaluation is done during parse-time which differs significantly from our parse, collect and evaluate approach. Incremental Rewriting [103] describes efficient algorithms for incrementally rewriting programs based on algebraic specifications. An algorithm for incrementally evaluating functions on aggregated values is also shown. The approach does not support non-local dependencies, making specification of name binding less intuitive as it requires copying of information.

3.8 Conclusion We have proposed an approach for incremental name and type analysis in two phases, collection and deferred evaluation of analysis tasks. The collection is instantiated with language-specific name binding and type rules and incremental on file level. Unchanged files are neither re-parsed nor re-traversed. The evaluation phase is incremental on task level. When a file changes, all tasks

56 that are affected by this change are reevaluated. This might include dependent tasks from other files. Tasks execute low-level instructions for name resolution and type analysis, and can form a basis for the definition of declarative meta-languages at a higher level of abstraction. For example, we map declarative name binding and scope rules expressed in NaBL to an instantiation of the presented approach. We implemented the approach as part of the Spoofax language workbench. It frees language engineers from the burden of manually implementing incremental analysis. We applied the implementation to WebDSL and empirical evaluation has shown this analysis to be responsive to changes in analyzed programs and suitable to the interactive requirements of an IDE setting. Acknowledgements This research was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Specific Languages) and by a research grant from Oracle Labs. We would like to thank Lennart Kats for his contribution to the start of NaBL and to Spoofax’ incremental analysis project. We would also like to thank Karl Kalleberg for valuable discussions on the interpretation of name binding and scoping rules.

Chapter 3. A Task Engine for Incremental Name and Type Analysis 57 58 Reﬂection: Incremental Name and Type Analysis, Bootstrapping, and Spoofax Core

We reﬂect on the applications and expressiveness of NaBL, TS, and the task engine, which we will refer to as NaBL for brevity. Furthermore, we provide 4 a vision for bootstrapping the meta-languages of Spoofax, and describe our work on Spoofax Core to make bootstrapping, other research, and application in industry feasible.

Incremental Name and Type Analysis We have applied NaBL to develop the static analysis of Green-Marl [63], a DSL for graph analysis, initially developed at Stanford University, but now part of the Parallel Graph AnalytiX (PGX)[94] project at Oracle Labs. With Green- Marl, graph analysis programs can be concisely written in terms of graph analysis concepts such as nodes, vertices, properties of nodes and vertices, and depth/breadth-first traversals. The PGX runtime can then automatically parallelize these graph analyses. We have also applied NaBL in the 2015-2016 edition of the Compiler Con- struction lab [145], where students develop a full version of the MiniJava [9] language, including name and type analysis and corresponding editor services. Limitations in Expressiveness. NaBL can be used to model many interesting name and type analysis patterns, even for complicated DSLs such as Green- Marl. However, there are several limitations in expressiveness. For example, certain kind of let bindings such as let*, which are found in some functional languages, cannot be modeled because of a lack of control over scopes. Method overriding and overloading, where there are multiple method definitions a method call can refer to, and a selection must be made between those definitions based on parameter types, are not well supported. Finally, more comprehensive type systems with nonlocal inference are not supported. Scope Graphs. Further work on improving the expressiveness has been done by other researchers in the context of Scope Graphs [109], a language-independent theory for program binding structure and name resolution. With this approach, a scope graph is first constructed from an abstract syntax tree using language- specific rules. Then, references in the scope graph are resolved to definitions using a language-independent resolution process. Name resolution is specified in terms of a concise, declarative and language-independent resolution calculus, while the actual name resolution is performed by a resolution algorithm that is sound and complete with respect to the calculus. Hendrik van Antwerpen et al. refine and extend the scope graph framework to a full framework for static semantic analysis [5]. The framework is based

59 on a language of constraints, which support uniting type checking and name resolution. A language-speciﬁc extraction function translates an abstract syntax tree to a set of constraints, which are then solved by a constraint solver, which is proven to be sound. Although not discussed in [5], the extraction function can be generated from NaBL2, a meta-DSL for specifying name and type analysis in terms of syntax-directed constraint generation rules. The expressiveness of the framework was further extended in [6] by viewing scopes as types, enabling models of the internal structure of non-simple types such as structural records and generic classes. Statix, a new meta-DSL was introduced, to enable speciﬁcation of static semantics using this new framework. The main difference between NaBL and scope graphs, is that scope graph approaches are more expressive, include the formal semantics of the calculi, and include algorithms that are sound in relation to the calculi. However, making these algorithms incremental and scalable; and providing good error messages, semantic completions, and other editor services; are still open problems.

Bootstrapping We have self-applied NaBL and TS and the SDF [153] meta-DSL (for syntax specification), providing incremental name and type analysis and editor services for these meta-languages. These meta-languages all depend on each other. That is, NaBL’s syntax is specified in SDF, its name binding in itself, and its type analysis in TS. Similarly, SDF and TS are specified in NaBL and each other. Therefore, we performed an ad-hoc form of bootstrapping, where for each meta-language, we apply its own compiler and the compiler of other meta-languages to the sources of the meta-language to generate code that implements parts of that meta-language. We then commit this generated code to source control, serving as a baseline for building the meta-language. However, this ad-hoc form of bootstrapping is problematic, because commit- ting generated (possibly binary) artifacts in source control creates additional load on source control storage. Furthermore, forgetting to commit the generated code may break the compilation of the meta-language. Finally, since we are applying all meta-language compilers only once, we cannot find cascading defects that are only evident after multiple applications. Our vision to solve these bootstrapping problems is to do fixpoint bootstrapping, to version and separately store meta-language compilers (binaries), and to support using multiple versions of meta-language compilers simultaneously in an interactive system. We want to evaluate this approach by application to Spoofax and its meta-languages. However, using Spoofax as a vehicle for research turned out to be a problem for several reasons.

Spoofax Problem Analysis First of all, Spoofax was dependent on the Eclipse IDE. It was not possible to run the Spoofax language workbench, or any language created with Spoofax,

60 private void registerWithImp(Language language) { final int TRIES = 10; final int SLEEP = 500;

for(inti=0; i < TRIES; i++) { try{ LanguageRegistry.registerLanguage(language); return; } catch (ConcurrentModificationException e) { // Loop try{ Thread.sleep(SLEEP); } catch (InterruptedException e2) { throw new RuntimeException(e2); } } } }

Listing 4.1: Unclear concurrency and dynamic language loading in older versions of Spoofax. outside of the Eclipse IDE, making it hard to develop, build, test, and deploy Spoofax. One of the defining features of Spoofax is dynamic language loading [75], where a language under development can be dynamically reloaded in the IDE without restarting, with the changes being immediately visible. However, Spoofax used IMP [24] for its Eclipse editor services, which does not support dynamic language loading, and thus required a hack to get this working: a single IMP language which acted as all Spoofax dynamic languages, with the actual language being identified at runtime based on the extension of a file. Furthermore, there were two ways to load languages: statically from an Eclipse plugin, and dynamically at runtime, which resulted in different code paths for all editor services. The concurrency model of Spoofax was unclear, leading to code as found in listing 4.1 in several places. Finally, Spoofax’s code was very tightly coupled, making to modify a part without breaking other parts. Besides these problems, we also missed two features crucial for bootstrapping meta-languages. We need to be able to version the compilers of languages, and to have explicit dependencies between languages. For example, to build NaBL version 2, we need SDF version 3’s compiler, NaBL version 1’s compiler, and TS version 1’s compiler.

Spoofax Core Given this problem analysis; and the need to do bootstrapping, other research, and better application in industry; we need to develop a better tooling foundation. Therefore, we have reimplemented Spoofax, dubbed Spoofax Core, which solves these problems and adds missing features. This was a joint effort

Chapter 4. Reﬂection: Incremental Name and Type Analysis and Bootstrapping 61 Spoofax 2.0

Sunshine Eclipse Eclipse IntelliJ Maven adapter adapter adapter adapter adapter

Stratego runtime Meta Spoofax Core Spoofax-meta Core lang. JSGLR

MetaBorg Core MetaBorg-meta Core

Figure 4.1: Spoofax Core component architecture. with Vlad Vergu, Hendrik van Antwerpen, Daniel Pelsmaeker, Martijn Dwars, Eduardo Souza Amorim, and Sebastian Erdweg. Spoofax Core is a platform-independent Java library, meaning that it can be integrated into any Java application, making custom integrations possible. We provide ready-made adapters for various platforms such as the command-line (called Sunshine), the Eclipse and IntelliJ IDEs, and the Maven build system. The language workbench environment, also called the meta-environment, is separated from the runtime environment so that languages can be used without a dependency on the meta-environment. Figure 4.1 is a depiction of this architecture Spoofax Core can be developed, built, tested, and deployed from the command-line on Windows, Linux, and MacOS, allowing anyone to work on Spoofax. To ensure that Spoofax works, and to provide ready-made packages for using Spoofax, our Jenkins build farm continuously builds, tests, and deploys Spoofax Core and its adapters. Spoofax Core is loosely coupled, enabling parts of Spoofax to be changed without breaking other parts. We use the Guice [48] dependency injection framework to hook up these loosely coupled parts as a separate concern, while also allowing several parts to be freely extended. Dynamic language loading is supported as a separate concern, with the rest of Spoofax only dealing with the concept of a language that is already loaded. Furthermore, we support concurrent environments such as IDEs by being thread-safe when its concurrency invariants are upheld. Finally, we have full support for versioning languages and explicit versioned dependencies between languages. With this new foundation, we continue our research into bootstrapping in the next chapter (chapter 5).

62 Bootstrapping Domain-Specific Meta-Languages in Language Workbenches Abstract 5 It is common practice to bootstrap compilers of programming languages. By using the compiled language to implement the compiler, compiler developers can code in their own high-level language and gain a large-scale test case. In this chapter, we investigate bootstrapping of compiler-compilers as they occur in language workbenches. Language workbenches support the development of compilers through the application of multiple collaborating domain-specific meta-languages for defining a language’s syntax, analysis, code generation, and editor support. We analyze the bootstrapping problem of language workbenches in detail, propose a method for sound bootstrapping based on fixpoint compilation, and show how to conduct breaking meta-language changes in a bootstrapped language workbench. We have applied sound bootstrapping to the Spoofax language workbench and report on our experience.

5.1 Introduction A bootstrapped compiler can compile its own source code, because the compiler is written in the compiled language itself. Such bootstrapping yields four main advantages: 1. A bootstrapped compiler can be written in the compiled high-level language. 2. It provides a large-scale test case for detecting defects in the compiler and the compiled language. 3. It shows that the language’s coverage is sufficient to implement itself. 4. Compiler improvements such as better static analysis or the generation of faster code applies to all compiled programs, including the compiler itself. Compiler bootstrapping is common practice nowadays. For example, the GCC compiler for the C language is a bootstrapped compiler; its source code is written in C and it can compile itself. More generally for a language L, a bootstrapped compiler Lc should apply to its own definition Ld such that Ld ∈ L and Lc(Ld) = Lc. Language workbenches [39] are compiler-compilers that provide high-level meta-languages for defining domain-specific languages (DSLs) and their compilers. Thus, users of a language workbench implement the compiler Lc of language L not in L but in a high-level meta-language M such that Ld ∈ M and Mc(Ld) = Lc. Thus, bootstrapping of Lc is no longer required, which is good since many DSLs have limited expressiveness and are often ill-suited for

63 compiler development. What we desire instead is bootstrapping of a language workbench’s compiler- compiler Mc. We want to use our meta-languages for implementing our meta- language compilers, thus inheriting the benefits of bootstrapping stated above: high-level meta-language implementation, large-scale test case, meta-language coverage, and improvement dissemination. In short, bootstrapping of language workbenches supports meta-language development. However, bootstrapping of language workbenches also entails three novel challenges: • Most language workbenches provide separate meta-languages M1..n for describing the different language aspects such as syntax, analysis, code generation, and editor support. Thus, to build the defintion of any one meta- i 1..n language compiler Md, multiple meta-language compilers Mc are necessary 1..n i i such that Mc (Md) = Mc. This entails intricate dependencies that sound language-workbench bootstrapping needs to handle. • Most language workbenches provide an integrated development environment (IDE). Typically, language workbenches generate or instantiate this IDE based on the definition of the meta-languages. In this setup, the meta- language developer needs to restart the IDE whenever the definition of a meta-language is changed. However, to support bootstrapping, the definition of meta-language compilers should be available within the IDE and no restart should be required to generate and load the new bootstrapped meta- language compilers [88]. Importantly, since meta-language changes can be defective, it also needs to be possible to rollback to an older meta-language version if bootstrapping fails. • Since meta-languages in language workbenches depend on one another, it can become difficult to implement breaking changes that require the simultaneous modification of a meta-language and existing client code. For example, renaming a keyword in one meta-language can require modifications in the compilers of the other meta-languages. To preserve changeability, we need to support implementing such breaking changes in a bootstrapped language workbench. We present a solution to these challenges based on versioning and fixpoint bootstrapping of meta-language compilers. That is, we iteratively self-apply meta-language compilers to derive new versions until no change occurs. For this to work, we identified properties that meta-language compilers need to satisfy: explicit cross-language dependencies, deterministic compilation, and comparability of compiler binaries. To support meta-language engineers, we describe how to build interactive environments on top of fixpoint bootstrapping. Finally, we discuss how to implement and bootstrap breaking changes in the context of fixpoint bootstrapping. To confirm the validity of our approach, we have implemented fixpoint bootstrapping for the Spoofax language workbench [75]. We use our implementation to successfully bootstrap eight meta-languages. We present our experience with seven changes to the meta-languages. We describe how we implemented the changes, how bootstrapping helped us to detect defects, and

64 how we handled breaking changes. We are the ﬁrst to describe a method for bootstrapping the meta-languages of a language workbench. In summary, we make the following contributions: • We present a detailed problem analysis and requirements for languageworkbench bootstrapping (section 5.2). • We describe a sound bootstrapping method based on ﬁxpoint meta-language compilation (section 5.3). • We explain how to build bootstrapping-aware interactive environments (section 5.4). • We investigate support for implementing breaking changes in a bootstrapped language workbench (section 5.5). • We validate our approach by realizing it in Spoofax and by investigating seven bootstrapping changes (section 5.6).

5.2 Problem Analysis To get a better understanding of bootstrapping in the context of language workbenches, we analyze the problem of bootstrapping in more detail. This problem analysis will help us answer why we need bootstrapping in the first place, and what is required to do bootstrapping in the context of language workbenches. 5.2.1 Bootstrapping Example First, we need a more realistic example that shows the complexities of bootstrapping language workbenches. As an example, we use the SDF and Stratego meta-languages from the Spoofax language workbench. SDF [153] is a meta- language for specifying syntax of a language. Stratego [18] is a meta-language for specifying term transformations. SDF and Stratego are bootstrapped by self specification and mutual specification. That is, SDF’s syntax is specified in SDF, and its transformations in Stratego. Stratego’s syntax is specified in SDF, and its transformations in Stratego. SDF also contains several generators. SDF contains a pretty-printer generator PP-gen that generates a pretty-printer based on the layout and concrete syntax in a syntax specification [158]. A pretty-printer (sometimes called an unparser) is the inverse of a parser. It takes a parsed abstract syntax tree (AST) and pretty- prints it back to a string. The generated pretty-printer is a Stratego program that performs this function. Besides generating a pretty-printer, SDF contains a signature generator Sig-gen that generates signatures for the nodes occurring in the AST. Since these signatures serve as a basis for AST transformations in Stratego, SDF describes these signatures in Stratego syntax and pretty-prints them using the generated Stratego pretty-printer. Overall, our scenario entails various complex dependencies across languages. In the remainder of this section, we focus on the following dependency chain: • The pretty-printer generator translates SDF ASTs into Stratego ASTs and

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 65 thus requires the SDF and Stratego signatures. • The SDF signatures are generated by the signature generator using the Stratego pretty-printer. • The Stratego pretty-printer is generated by the pretty-printer generator, from the Stratego syntax deﬁnition. • The pretty-printer generator is implemented as a Stratego program within SDF.

We want to apply bootstrapping to SDF and Stratego to detect defective changes to a language’s implementation. In order to illustrate the difficulties of bootstrapping in the context of language workbenches, we will deliber- ately introduce a defect in the implementation of the pretty-printer generator. Normally, a pretty-printer needs to align with the parser such that parse(pretty-print(ast)) = ast. We break the the pretty-printer generator to violate this equation by generating pretty-printers that print superfluous semicolons. This is an obvious way to sabotage the pretty-printer generator and will cause parse failures when parsing a pretty-printed string. We expect bootstrapping to detect this defect. Figure 5.1 shows an iterative bootstrapping attempt with relevant dependencies, illustrating code examples, and an explanation for each bootstrapping iteration. We start with a baseline of language implementations. We introduce the defect in the pretty-printer generator and start rebuilding the whole system in Iteration 1 using the baseline. However, despite the defect, all components build fine in Iteration 1. This is because it takes multiple iterations for the defect to propagate through the system before it produces an error. In our example, the defective pretty-printer generator (Iteration 1) generates a broken pretty-printer (2), which is used by the signature generator (3), which then generates signatures in Stratego syntax but with superfluous semicolons (4). All defects remain undetected until the build of PP-gen or Sig-gen in Iteration 4 fails because of parse errors in the signatures. Our example illustrates multiple points. First, dependencies between components in a language workbench are complex, circular, and across languages. Second, language bootstrapping yields a significant test case for language implementations and can successfully detect defects. Third, a single build is insufficient because many defects only materialize after multiple iterations of rebuilding. This example is still far removed from the complexity that language workbenches face in practice. For example, Spoofax features eight interdependent meta-languages and SDF alone has seven generators that uses pretty-printers from four other meta-languages.

5.2.2 Requirements

Based on our example above, we derive requirements for sound bootstrapping support in language workbenches.

66 SDF0 STR0 Baseline Correct baseline, consisting of SDF and Stratego (STR), with a working Stratego pretty-printer (PP). PP-gen Sig-gen PP pp-Decl : Signature(t) -> [ H([SOpt(HS(), "0")], [S("signature")]), t ]

SDF1 STR1 Bootstrapping iteration 1 SDF The language engineer changes PP-gen, the pretty-printer generator. In this example, they introduce a defect into the pretty-printer generator, that generates pretty-printers that print superﬂuous semicolons. Sig SDF The defect is not observed, because the defect PP-gen has not been used yet. String(s) -> List(Op("S", [List([Str(s)])]))

PP-gen Sig-gen PP String(s) -> List(Op("S", [List([Str(s), Str(";")])]))

SDF2 STR2 Bootstrapping iteration 2 SDF PP is now a defect Stratego pretty-printer, generated with the defect PP-gen from SDF1, but is not included in Sig-gen yet. The defects are not observed, because the defect PP has not been used yet.

Sig SDF pp-Decl : Signature(t) -> [ H([SOpt(HS(), "0")], [S("signature"), S(";")]), t ]

PP-gen Sig-gen PP

SDF3 STR3 Bootstrapping iteration 3 SDF The defect Stratego pretty-printer from STR2 is now included in Sig-gen. However, SDF signatures (Sig) are still correct, since they are generated by the correct signature generator from SDF2, which imports a correct Stratego pretty-printer from STR1. Sig SDF The defects are not observed, because the defect Sig-gen has not been used yet. module signatures/sorts/Sorts-sig signature constructors : String -> Sort PP-gen Sig-gen PP Sort : Sort -> Symbol ParameterizedSort : Sort * List(Symbol) -> Symbol

SDF4 Bootstrapping iteration 4 SDF Defect SDF signatures, generated with the defect signature generator from SDF3. These signatures are not valid Stratego code. The defects are observed because PP- gen and Sig-gen use signatures that cannot be parsed.

Sig module; signatures/sorts/Sorts-sig signature; constructors; :; String ->; Sort Sort :; Sort ->; Symbol PP-gen Sig-gen ParameterizedSort :; Sort *; List(;Symbol); ->; Symbol

language generator G generates C, from source ﬁles written in meta-language M, using correct generator uses previously generated component I G M C hidden defect generated component generates

source ﬁles written in observed defect M meta-language M I

Figure 5.1: Bootstrapping ﬂow for bootstrapping SDF and Stratego with a defective pretty-printer generator. In each iteration, SDF and Stratego are compiled, based on their previous versions. For example, SDF2 is compiled with SDF1 and STR1. In the fourth iteration, bootstrapping fails because of a parse error, which can be traced back to the change which introduces a defect into the pretty-printer generator in the ﬁrst iteration. Source code on the right belongs to underlined/bold components on the left.

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 67 Sound Bootstrapping. In our example, we needed 4 bootstrapping iterations to find a failure caused by the defective pretty-printer. In general, there is no way to know how many iterations are necessary until a defect materializes or after how many iterations it is safe to stop. Therefore, for sound bootstrapping it is required to iterate until reaching a fixpoint, that is, until the build stabilizes. To determine if a fixpoint has been reached, we must be able to compare the binaries that meta-languages generate. We have reached a fixpoint if the generated binaries in iteration k + 1 are identical to the binaries generated in iteration k. Since the binaries are the same, further rebuilds after reaching a fixpoint cannot change the implementation or detect new defects. A further requirement for fixpoint bootstrapping is that compilers must be deterministic. That is, when calling a compiler with identical source files, the compiler must produce identical binaries. Bootstrapping always requires a baseline of meta-language binaries to kickstart the process. Bootstrapping uses the baseline only to rebuild the meta- languages in the first bootstrapping iteration. After that, bootstrapping uses the bootstrapped binaries. Finally, the bootstrapping system should be general; it should work for any meta-language in the language workbench.

Interactive Bootstrapping Environment. Besides having a bootstrapping system that satisﬁes the requirements above, we also need to support bootstrapping in the interactive environments of language workbenches. In particular, an interactive environment needs to provide operations that (1) start a bootstrapping attempt, (2) load a new baseline into the environment after bootstrapping succeeded, (3) roll back to an existing baseline after bootstrapping failed, and (4) cancel non-terminating bootstrapping attempts. Loading a baseline needs to be such that subsequent bootstrapping attempts use the new baseline. When bootstrapping fails, a rollback to the existing baseline is required such that the defect causing the failure can be ﬁxed and a new bootstrapping attempt can be started. All operations should work within the same language workbench environment, without requiring a restart of the environment, or a new environment to be started.

Bootstrapping Breaking Changes. Bootstrapping helps to detect changes that break a language implementation. However, sometimes breaking changes are desirable, for example, to change the syntax of a meta-language. If we change the syntax deﬁnition of some language L and the code written in L simultaneously, bootstrapping fails to parse the source code in Iteration 1 because the baseline only supports the old syntax of L. If we change the syntax deﬁnition of L but leave the code written in L unchanged, bootstrapping fails to parse the source code in Iteration 2 because the parser of L generated in Iteration 1 only supports the new syntax of L. The bootstrapping environment should provide operations for bootstrapping breaking changes.

68 5.3 Sound Bootstrapping

Compiling or bootstrapping a meta-language is a complex operation that requires application of generators from many meta-languages, to a meta- language that consist of sources in several meta-languages. Therefore, we would like to find a general compilation and bootstrapping algorithm. We describe a method for sound bootstrapping that fulfills the requirements from the previous section. As a first step towards compilation and bootstrapping, we introduce a general model for meta-language definitions and products. Using the model, we describe a general compilation algorithm that compiles a meta-language definition into a meta-language product. Finally, we show how to perform fixpoint bootstrapping operations based on the model and compilation algorithm. We use the bootstrapping scenario from fig. 5.1 as a running example in this section.

5.3.1 Language Deﬁnitions and Products

As a first step towards bootstrapping, we introduce a general model for language definitions and products. We require such a model to describe a general compilation and bootstrapping algorithm for meta-languages. Listing 5.1 shows the model encoded in Haskell. In this subsection, we explain the model. In later subsections, we explain the compilation and fixpoint bootstrapping algorithm.

Language. First of all, we use a unique name to identify each meta-language Lang of a language workbench, such as SDF and Stratego. However, a name alone is not enough to uniquely distinguish meta-languages. Multiple versions of the same meta-language exist when bootstrapping, for example, a baseline version of SDF and the first bootstrapping iteration of SDF. Therefore, we also use a version to identify a language, LangID in the model. We denote a language L with version 1 as L1. For example, with versioning, we can uniquely identify different versions of SDF and Stratego: SDF0, STR0, SDF1, STR1. Bootstrapping applies the generators of a meta-language to the definition of its own and other meta-languages. Therefore, it is important to distinguish a meta-language definition from a meta-language product, which results from compiling the definition. The example in fig. 5.1 does not make this distinction to reduce its complexity, but we require this distinction here in order to precisely define compilation and bootstrapping.

Language Definition. Each language definition LangDef defines a specific ver- 1 sion of a language. We denote the definition of language L at version 1 as Ld. The definition consists of source artifacts written in different meta-languages (field alang of Artifact). To compile a language definition, we need to know what external artifacts and generators it requires. To this end, a language definition defines artifact and generator dependencies on previous versions of itself or on specific versions of other languages. We use these dependencies during compilation.

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 69 -- Model for languages, language definitions with sources, and language products with artifacts and generators. type Version= Int type Lang= String data LangID= LangID { name :: Lang, version :: Version} data Artifact= Artifact { aname :: String, alang :: Lang, acontent :: String} data LangDef= LangDef { dlang :: LangID, dsources :: [Artifact], dartDeps :: [LangID], dgenDeps :: [LangID]} data Generator= Generator { gname :: String, gsource :: Lang, gtarget :: Lang, generate :: Artifact -> Artifact} data LangProd= LangProd { plang :: LangID, partifacts :: [Artifact], pgenerators :: [Generator]} type Baseline=[LangProd]

getProd:: LangID -> Baseline -> LangProd getProd lang baseline = fromJust $ find (\prod -> lang == plang prod) baseline

-- Compile. Sort languages by generator source/target and run relevant generators against relevant artifacts. compile:: LangDef -> Baseline -> LangProd compile def baseline = createLangProd def (runGens sorted gens inputs) where inputs = dsources def ++ [ a | l <- dartDeps def, a <- partifacts (getProd l baseline) ] gens = [ g | l <- dgenDeps def, g <- pgens (getProd l baseline) ] sorted = topsort [ l | LangIDl_ <- dgenDeps def ] [ (gsource g,gtarget g) | g <- gens]

runGens::[Lang] -> [Generator] -> [Artifact] -> [Artifact] runGens[] gens inputs = inputs runGens (lang:langs) gens inputs = runGens langs gens (inputs ++ runGensFor lang gens inputs)

runGensFor:: Lang -> [Generator] -> [Artifact] -> [Artifact] runGensFor lang gens inputs = [ generate g a | g <- gens, a <- inputs, gsource g == lang, alang a == lang ]

createLangProd:: LangDef -> [Artifact] -> LangProd -- Implemented by the LWB.

-- Fixpoint bootstrap language definitions with a baseline. -- Update versions in the first iteration, then fixpoint. bootstrap:: Version -> [LangDef] -> Baseline -> (Baseline,[LangDef]) bootstrap v defs baseline = let firstBuild = [ compile (setVersion v def) baseline | def <- defs ] in bootstrapFixpoint (prepareFixpoint v defs) firstBuild

bootstrapFixpoint::[LangDef] -> Baseline -> (Baseline,[LangDef]) bootstrapFixpoint defs baseline = let newBaseline = [ compile def baseline | def <- defs ] in if baseline == newBaseline then (newBaseline, defs) else bootstrapFixpoint defs newBaseline

setVersion:: Version -> LangDef -> LangDef setVersionv(LangDef(LangIDl_) srcs gdeps adeps) = LangDef(LangID l v) srcs gdeps adeps

prepareFixpoint:: Version -> [LangDef] -> [LangDef] prepareFixpoint v defs = [ prepareFixpointDef v bootstrappedLangs def | def <- defs ] where bootstrappedLangs = [ l | LangDef(LangIDl_)___ <- defs ]

prepareFixpointDef:: Version -> [Lang] -> LangDef -> LangDef prepareFixpointDef v langs (LangDef(LangIDl_) srcs adeps gdeps) = LangDef(LangID l v) srcs [ updateDep v langs dep | dep <- adeps ] [ updateDep v langs dep | dep <- gdeps ]

updateDep:: Version -> [Lang] -> LangID -> LangID

Listing 5.1: Model for sound bootstrapping, with algorithms for compilation and ﬁxpoint bootstrapping, encoded in Haskell.

70 Language Product. A language product LangProd models a compiled meta- 1 1 language definition. We denote the product of compiling Ld as Lp. A product exports artifacts and generators. A generator Generator transforms artifacts of some source language into artifacts of some target language. For example, Sig-gen in SDF transforms SDF artifacts into Stratego artifacts, or fails if the SDF artifacts are invalid, which we model as a dynamic exception of function generate. Example. Language definitions and products model the dependencies required to compile a definition into a product, which we describe in the next 1 subsection. For example, SDFd requires the application of generator Sig-gen 0 1 0 of SDFp, whereas STRd requires the application of generator PP-gen of SDFp. 1 0 Moreover, SDFd requires the pretty-print table artifact PP of STRp. 5.3.2 Compilation Before we can bootstrap multiple meta-language definitions against a baseline of meta-language products, we must first be able to compile a single meta- language definition. We describe the compilation algorithm that compiles a single language definition using the model from above. Function compile takes a language definition and a baseline of language products, and produces a new language product from the definition. The basic idea of the algorithm is to run the required generators on the source artifacts and the required external artifacts. This yields new generated artifacts that we package into a language product using createLangProd. We first collect all generator inputs, which are the source artifacts (dsources def) of the definition and the required artifacts according to dependencies (dartDeps def). We use the baseline to resolve dependencies; function getProd finds the product of the required LangID. Similar to required artifacts, we collect the required generators according to dependencies (dgenDeps def). When running generators, we need to make sure to call them in the right order: A generator must run later if it consumes an artifact produced by 1 another generator. For example, SDFd requires the application of generator 1 Sig-gen, which produces Stratego code. But SDFd also requires the application 0 of the Stratego-compiler generator of STRp, which translates Stratego code into an executable. Thus, we must run Sig-gen before the Stratego compiler. To this end, we sort all languages topologically according to their source and target languages. Function runGenerators iterates over the sorted source languages and for each one applies all generators of the current source language lang. Function runGeneratorsFor finds all relevant generators g that take artifacts of lang as input and it finds all relevant artifacts a of lang. It then calls the generate function of all relevant generators g on all relevant artifacts a and collects and returns the generated artifacts. Function runGenerators passes the generated artifacts down when recursing to allow subsequent generators to compile them. If any generate function fails with a dynamic exception, the compilation fails. Finally, after generating all artifacts, we create a language product from the

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 71 language definition and the generated artifacts by calling createLangProd. This function must be implemented by the language workbench. We abstract over how a language workbench determines which artifacts to export and which generators to create from generated artifacts. For example, Spoofax determines which artifacts to export from a configuration file in the language definition, has built-in notions of generators to create based on generated artifacts, and allows a language definition to configure its own generators. 5.3.3 Fixpoint Bootstrapping We can now use compilation to define fixpoint bootstrapping. In general, there is no way to know how many bootstrapping iterations are required before it is safe to stop. Therefore, we iteratively bootstrap meta-languages until reaching a fixpoint. We define a general fixpoint bootstrapping algorithm using the model and compilation algorithm from above. Function bootstrap takes the version of the new baseline, a list of meta- language definitions, and an existing baseline, and it produces a new baseline of the given version. The basic idea of the algorithm is to compile meta- language definitions in iterations, until we reach a fixpoint. However, to avoid building against the old baseline repeatedly, we have to update the versions of the language definitions in the first iteration. In the first iteration, function bootstrap calls compile on modified definitions def where we have set the version to v. This yields a list of language products firstBuild that contains products of version v. We use this as starting point for fixpoint computation. In addition, we update the dependencies in defs using function prepareFixpoint, which updates versions and dependencies of all bootstrapped languages to v. To produce a new baseline, we repeat bootstrapping in bootstrapFixpoint until reaching a fixpoint. In each iteration, we compile all meta-language definition into meta-language products. If the new baseline is equal to the baseline from the previous iteration, we have reached a fixpoint and return the new baseline. To compare the language products of a baseline, we compare the name, version, artifacts, and generators of products. To compare generators, we need to compare the executables of generators (not modeled in Haskell). In practice, this boils down to comparing binary files byte-for-byte, ignoring nondeterministic metadata such as the creation date or the last modified date. We change meta-language versions each iteration in fig. 5.1 for illustrative purposes. However, bootstrap only changes versions once, to prevent baseline comparison from always failing because of version differences. Bootstrapping fails with a dynamic exception if any compile operation fails. Otherwise, our algorithm soundly produces a new baseline. In the next section, we explain how to manage baselines in interactive environments.

5.4 Interactive Bootstrapping A language workbench provides an interactive environment in which a language engineer can import language deﬁnitions, make changes to the deﬁni-

72 tions in interactive editors, compile them into a language products, and test the changed languages. This allows a language engineer to quickly iterate over language design and implementation. Likewise, a meta-language engineer wants to quickly iterate over meta-language design and implementation [88]. There- fore, we need to support running bootstrapping operations in the interactive language workbench environment. A language workbench manages an interactive environment with a language registry that manages all loaded language definitions and language products. The language registry loads language definitions and products dynamically, that is, while the environment is running without restarting the environment or starting a new one. The language workbench should react to loading, reloading, and unloading of language definitions and products, for example, by setting up file associations and updating editors. To support interactive development, meta-language compilation interacts with the language registry. Instead of receiving a baseline as argument, in an interactive environment function compile from the previous section uses the language registry to retrieve language products. Thus, a change to the registry affects subsequent compilations. Since bootstrapping relies on compile, in an interactive environment bootstrapping also interacts with the language registry. Instead of receiving a baseline as argument, in an interactive environment function bootstrap from the previous section uses the language registry to retrieve an initial baseline. Before calling compile in each iteration, bootstrap needs to load/reload the compiled products of version v. If bootstrapping succeeds, the new baseline stays in the language registry upon termination of bootstrap. But if bootstrapping fails with an exception, subsequent operations may not use the intermediate language products. To this end, bootstrap needs to rollback changes to the registry by unloading the language products of version v, and rolling back version changes in definitions. Based on these changes to the algorithms and the language registry, our bootstrapping model supports interactive environments. Specifically, we can start a bootstrapping attempt with bootstrap, load a new baseline into the registry, and rollback the registry after bootstrapping failed or was canceled by the user.

5.5 Bootstrapping Breaking Changes In the context of bootstrapping, a breaking change is a change to meta-language definitions such that fixpoint bootstrapping fails. Instead of treating such change as a whole, a breaking change needs to be decomposed into multiple smaller changes for which fixpoint bootstrapping succeeds. For example, changing a keyword in the SDF meta-language is a breaking change, because it will cause parse failures for SDF source files elsewhere. Changing a keyword and all usages of the old keyword is also a breaking change, because we use the old baseline on the first build, which does not support the new keyword. To perform such a breaking change, we need decompose it into smaller non-

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 73 breaking changes. Using such decomposition, fixpoint bootstrapping succeeds after each change. However, it is actually sufficient to only perform a full fixpoint bootstrap after the final change and to only find defects then. For the intermediate changes, it is enough to bootstrap a single iteration in order to construct a new baseline for the subsequent builds. To support this, we propose to extend the interactive environment with an additional bootstrapping operation that bootstraps a single iteration only. This will still find all defects in the final fixpoint bootstrapping, but intermediate defects may go unnoticed until then. A common breaking change that occurs when evolving a meta-language is the change of a feature F to F0. For example, this includes changing the syntax of Stratego or changing the Sig-gen generator in SDF. We can decompose 0 a change of feature F to F0 in M by (1) adding F as an alternative to F in M,(2) executing a single bootstrap iteration, (3) changing all source artifacts 0 written in M to use F instead of F,(4) executing a single bootstrap iteration, (5) removing F from M, and finally (6) performing a fixpoint bootstrap. We have successfully used this decomposition for changing features in our evaluation.

5.6 Evaluation To evaluate our bootstrapping method, we realized it in the Spoofax language workbench and bootstrapped Spoofax’s eight meta-languages. 5.6.1 Implementation We have implemented the model, general compilation algorithm, and general bootstrapping algorithm of our sound bootstrapping method in the interactive Eclipse environment of the Spoofax language workbench. With our implementation, a meta-language engineer can import meta-language definitions into Eclipse, make changes to the definitions, and run bootstrapping operations on the definitions to produce new baselines. The Eclipse console displays information about the bootstrapping process, e.g. when a new iteration starts, which artifacts were different during language product comparison, and any errors that occur during bootstrapping. When bootstrapping fails, changes are reverted, and the console shows observed errors. Bootstrapping can also be cancelled by cancelling the bootstrapping job. When bootstrapping succeeds, the new baseline and meta-language definitions are dynamically loaded, such that the meta-language engineer can start making changes to the definitions and run new bootstrapping operations. 5.6.2 Meta-languages To evaluate the bootstrapping method and implementation, we bootstrap Spoofax’s meta-languages. Spoofax currently consists of eight meta-languages: SDF2, SDF3, Stratego, ESV, NaBL, TS, NaBL2, and DynSem. The generator dependencies between these meta-languages are shown in fig. 5.2. Syntax used to be specified in the SDF2 [153] language, but we have since

74 Dynsem

ESV SDF2

Stratego

SDF3 NaBL2

TS NaBL

Figure 5.2: Generator dependencies between meta-languages. A generator dependency indicates that a meta-language requires (some) generators of a meta- language.

moved on to the more advanced SDF3 [158] language with syntax templates from which pretty-printers are automatically derived. SDF2 still exists because of compatibility reasons (some languages still use it) but also to use it as a target for generation. The SDF3 compiler generates SDF2, which the SDF2 compiler turns into a parse table. The SDF that we have been using as a running example in this paper is actually SDF3. ESV is a domain-specific meta-language for specifying editor services such as syntax coloring, outlines, and folding. Every meta-language (including ESV itself) uses ESV to specify its editor services, such that Spoofax can derive an editor for the meta-languages. Stratego [150, 18] is used for specifying term transformations and static semantics in several (meta)languages, and is also a common target for generation. For the name and type analysis domains, NaBL [89] is a domain-specific meta- language for specifying name analysis, and TS for specifying type analysis in terms of typing rules. NaBL2 is an evolution of NaBL that combines NaBL and TS in one language. Again, the older version of the language is kept for compatibility reasons. Finally, DynSem [147] is a meta-language for dynamic semantics specification through operational semantics. We have successfully bootstrapped these meta-languages with our bootstrapping implementation.

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 75 5.6.3 Bootstrapping Changes We evaluate our bootstrapping method and its implementation in Spoofax by bootstrapping changes to Spoofax’s meta-languages. We could not test our bootstrapping implementation against existing changes made to the meta- languages, because our bootstrapping implementation expects meta-languages to be in a specific format which existing meta-languages are not. Therefore, we converted the meta-languages to this format and constructed realistic and interesting changes for evaluation. We have logged the changes in the form of a Git repository1, which contains a readme file explaining how to view the repository. Each change is tagged with a version in the Git repository. For each tag, sources and binaries of the created baseline are available. Tags for fixpoint bootstrapping operation also include the bootstrapping log. We now go over each change scenario to explain what we changed, why we made the change, and any issues that occurred. Initial Bootstrap. To be able to bootstrap Spoofax’s meta-languages, we convert the meta-language definitions to work with our bootstrapping implementation. We successfully bootstrap the meta-languages by running a fixpoint bootstrapping operation. Version v2.1.0 is the first fixpoint bootstrap that includes all meta-languages, and will be used as a baseline for the next change. SDF2 in SDF3. SDF2 currently exports a handwritten pretty-printer, which is imported by SDF3 to pretty-print SDF2 source files. A handwritten pretty- printer is bad for maintenance because it must be manually changed whenever the syntax changes. However, a generated pretty-printer is automatically updated in conjunction with changes to the syntax, which reduces the maintenance effort. Therefore, we convert SDF2’s syntax to SDF3, such that SDF3’s pretty-printer generator, generates a pretty-printer to replace the handwritten one. This is a breaking change because SDF3 imports SDF2’s pretty printer, and we change the pretty-printer that SDF2 exports, so SDF3’s imports need to change. We decompose the change into three parts by applying the feature change decomposition shown previously: (1) we convert SDF2’s syntax to SDF3 and export the generated pretty-printer, while still exporting the handwritten pretty-printer, (2) we change SDF3 to use the generated pretty-printer from SDF2, and (3) we remove the handwritten pretty-printer from SDF2. We apply each bootstrapping operation in the same environment, to test the interactive language workbench environment. We first convert SDF2’s syntax to SDF3, and run a single iteration bootstrapping operation to produce baseline v2.1.1. However, we have converted the syntax of SDF2 wrongly, constructor names are supposed to be in lowercase to retain compatibility with existing SDF2 transformations. Furthermore, lowercase constructor names such as module conflict with Stratego’s syntax, which uses module as a reserved keyword. Therefore, we change SDF3 to support quotation marks in constructor names to support 'module, which does not

1https://github.com/spoofax-bootstrapping/bootstrapping

76 conflict with Stratego, and run a single iteration bootstrapping operation to produce baseline v2.1.2. We convert SDF2’s grammar again, with lowercase constructor names, and prefix reserved keywords with ' where needed, run a single iteration bootstrapping operation to create baseline v2.1.3. We use the newly generated signatures and pretty-printer from SDF2 in SDF3, run a fixpoint bootstrapping operation (to confirm that the pretty-printer works), which succeeds, and produces baseline v2.1.4. Finally, we clean up SDF2 by removing the handwritten pretty-printer. We also fix a bug in Stratego that causes some imports to loop infinitely during analysis, that was uncovered by the import dependencies between SDF2 and SDF3, which now mutually import each other. A fixpoint bootstrapping operation produces baseline v2.1.5. Stratego in SDF3. We convert Stratego’s syntax from SDF2 to SDF3 to also benefit from generated signatures and pretty-printers, instead of handwritten ones. This breaking change is decomposed in a similar way. However, we do not remove the handwritten pretty-printer yet, because multiple other meta-languages are using it, while we only change NaBL to use the generated one. We convert Stratego’s syntax to SDF3, run a single iteration bootstrapping operation to produce baseline v2.1.6. We change NaBL to use the newly generated Stratego signatures and pretty-printer, and run a fixpoint bootstrapping operation to produce baseline v2.1.7. Results. We were able to successfully bootstrap eight meta-languages with realistic changes, including complex breaking changes that required multiple bootstrapping steps. Bootstrapping is sound because it terminates, finds defects when we introduce them, and produces a baseline when bootstrapping succeeds. We were also able to run multiple bootstrapping operations in the interactive language workbench environment, where the baseline produced from bootstrapping is loaded into the environment and used to kickstart the next bootstrapping operation.

5.7 Related Work We now discuss related work on bootstrapping. 5.7.1 Bootstrapped General-Purpose Languages Most general-purpose programming languages are bootstrapped. We discuss early languages and compilers that were bootstrapped. The ﬁrst programming language to be bootstrapped in 1959 is the NELIAC [66] dialect of the ALGOL 58 language. The main advantage of bootstrapping the compiler is implementation in a higher-level language. In- stead of writing the compiler in assembly, it could be written in NELIAC itself, which is a much higher-level language than assembly. This allowed the compiler to be more easily be cross-compiled to the assembly of other machines, since the cross-compiled versions could be written in NELIAC.

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 77 Lisp was bootstrapped by creating a Lisp compiler written in Lisp, which was interpreted by an existing Lisp interpreter [142]. It is the first compiler that compiled itself by being interpreted by an existing interpreter of that language. The Pascal P compiler [110] is a compiler for the minimal subset of standard PASCAL that it can still compile itself. It generates object code for a hypotheti- cal stack computer SC. The first version of the compiler is written in assembly for SC. Using an assembler or interpreter for SC, the first compiler is compiled or executed. The compiler is bootstrapped by writing the compiler in itself, and compiling it with the existing compiler. The bootstrapped compiler is validated by comparing the assembled compiler binary and the bootstrapped compiler binary, a convention that we still apply to this day. 5.7.2 Bootstrapping We now look at literature on the art of bootstrapping itself. Tombstone diagrams (also called T-diagrams) are a graphical notation for reasoning about translators (compilers), first introduced in [16] and extended with interpreters and machines in [31]. T-diagrams are most commonly used to describe bootstrapping, cross-compilation, and other processes that require executing complex chains of compilers, interpreters, and machines [95]. T- diagrams are a useful tool for graphically reasoning about compilers and bootstrapping, orthogonal to the bootstrapping framework presented in this paper. Axiomatic bootstrapping [8] is an approach for reasoning about bootstrapping using axioms and equations between those axioms, to verify if a change to a compiler will result in a successful bootstrap. They present axioms for an interactive ML runtime and compiler, which compiles to native code, and needs to deal with multiple architectures, calling conventions, and binary for- mats. For example, instantiating the axioms and equations show that changing the calling convention of the compiler causes bootstrapping to fail, and that a special cross-compilation operation can bootstrap the change. Axioms are a useful tool to verify if a single bootstrapping iteration will work, and to reason about how a breaking change should be split up over multiple bootstrapping steps. However, axioms cannot be used to find hidden defects, such as the example from fig. 5.1, because those defects can manifest over an arbitrary number of iterations. Axioms also cannot be used to reason if a fixed point can be reached, for the same reason. Therefore, it is complementary to the bootstrapping framework presented in this paper. 5.7.3 Language Workbenches We now look at related work on bootstrapping in language workbenches. Xtext [12] is a framework and IDE for the development of programming languages and DSLs. Its Grammar and Xtend meta-languages are bootstrapped. However, Xtext does not support dynamic loading, so it is not possible to bootstrap the meta-languages in the language workbench environment. MPS [159] is a projectional language workbench. It has several meta- languages which are bootstrapped, but are read-only in the language work-

78 bench environment, meaning that they cannot be bootstrapped in the language workbench environment. MPS supports dependencies between languages through its powerful extension system. A meta-language can be extended, and that extension could be bootstrapped. However, MPS does not support versioning or undoing changes, making rollbacks impossible if defects are introduced. MetaEdit+ [78] is a graphical language workbench for domain-specific modeling. Some of its meta-languages are bootstrapped, and can be bootstrapped in the language workbench environment. When changes are applied, they immediately apply to the bootstrapped meta-language and other languages. If an applied change breaks the language, the change can be undone or aban- doned entirely to go back to a previous working state, after which the error can be fixed. However, they do not document their bootstrapping method, so it cannot be applied to other language workbenches. Enso¯ [97, 138] is a project to enable a software development paradigm based on interpretation and integration of executable specification languages. Enso’s¯ meta-languages are bootstrapped, but has no general framework for fixpoint bootstrapping or versioning of meta-languages. The meta-language engineer has to write code which handles fixpoint bootstrapping and versioning specifically for their meta-languages. Rascal [81] is a metaprogramming language and IDE for source code analysis and transformation. In the current version, Rascal’s parser is bootstrapped, but the rest is implemented as an interpreter in Java. Development versions include a new Rascal compiler which is completely bootstrapped. However, the Rascal IDE has no general support for fixpoint bootstrapping or versioning of languages. SugarJ [37, 35] is a Java-based extensible programming language that allows programmers to extend the base language with custom language features. In principle, SugarJ can be bootstrapped, because its compiler is written in Java and Java is a subset of SugarJ. However, in practice this was never done, and it is not obvious if that would actually work. Racket [146] is an extensible programming language in the Lisp/Scheme family, which can serve as a platform for language creation, design, and implementation. DrRacket [41, 42] is the Racket IDE. Racket is mostly bootstrapped, but the core of the compiler and interpreter are implemented in C. The parts of Racket that are written in Racket can be changed interactively in DrRacket, which affects subsequently running Racket programs. A defect introduced in Racket’s self definition may prevent bootstrapping to succeed, which requires a restart of the DrRacket IDE.

5.7.4 Staged Metaprogramming

Staged metaprogramming approaches such as MetaML [143], MetaOCaml [21], Mint [162], and LMS [125] provide typesafe run-time code generation, which ensures that generated code does not contain typing defects. However, these approaches do not provide support for bootstrapping.

Chapter 5. Bootstrapping Meta-DSLs in Language Workbenches 79 5.8 Conclusion Bootstrapping is an efficient means for detecting defects in compiler implementations and should be useful for language workbenches as well. However, bootstrapping compiler-compilers of language workbenches needs to handle the intricate interactions between meta-languages. Unfortunately, previous literature on bootstrapping ignores these intricacies. We present a sound method for meta-language bootstrapping. Given a baseline and updated meta-language definitions, our bootstrapping algorithm constructs a new baseline through fixpoint self-application of the meta-languages. We explain how our algorithms can be used in interactive environments and how to decompose breaking changes that occur when evolving meta-languages. We have implemented the approach in the Spoofax language workbench and evaluated it by successfully bootstrapping eight interdependent meta- languages, and report on our experience with bootstrapping two breaking changes. This makes Spoofax into a laboratory for meta-language design experimentation. Acknowledgements This research was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Specific Languages) and NWO VICI Project (639.023.206) (Language Designer’s Workbench).

80 Reﬂection: Language Workbench Pipelines

Our work on bootstrapping enables systematic bootstrapping of the meta- languages of language workbenches in an interactive metaprogramming system. However, while working on bootstrapping, we noticed that language workbenches implement a very complicated pipeline. A language workbench 6 pipeline builds (meta-)language specifications into (meta-)language products; parses, analyses, and transforms programs using these products; and provides feedback to programmers through editor services such as inline error messages, code styling, structure outlines, and code completion. Such a pipeline roughly flows as follows: • Meta-language specifications are bootstrapped using a baseline, to produce new meta-language products. • Meta-language programs (which specify a language) go through the pipeline: parse, analyze, transform, and provide feedback to the language developer, using meta-language products. • Language specifications are built into language products. • Programs go through the pipeline and provide feedback to programmers, using language products. • When programs are transformed or compiled into other programs that require more processing or feedback, the pipeline continues. To make an interactive (meta)programming system based on such a pipeline, it must be incremental, scalable, and easy to develop and maintain. While the pipeline was improved a lot with our work on Spoofax Core and bootstrapping, it still suffered from several problems which we now analyze.

Problem Analysis The pipeline of Spoofax overspecifies certain dependencies, causing a loss of incrementality. For example, when we change the name and type analysis specification of a language, rebuild the language specification into a language product, and then edit a program of that language, Spoofax will reparse the program even though the parser did not change. This is because Spoofax does not perform fine-grained dependency tracking, because the development effort to do so would be large, requiring incrementality techniques such as caching, cache invalidation, dependency tracking, and change detection. Conversely, Spoofax’s pipeline also underspecifies certain dependencies, causing loss of correctness. For example, when we change the NaBL meta- language, and then edit a language specification that uses NaBL, Spoofax will not automatically rebuild the NaBL meta-language, resulting in inconsistent behavior. The language developer must first manually build (bootstrap)

81 NaBL, and then build their own language. This is again because Spoofax does not perform fine-grained dependency tracking, but also because eagerly rebootstrapping the meta-languages for every change would take a lot of time. In some cases, Spoofax’s pipeline is incremental and correct. For example, the separate transformation (compilation) of the source files of the SDF meta- language into normalized SDF files for parse table generation, pretty-printer rules, and syntactic code completion rules, is incremental and correct. Sepa- rate compilation is simpler to implement because there are no dependencies between source files. However, to achieve this, we still manually implement an incremental pipeline with caching, invalidation, and change detection. Finally, Spoofax’s pipeline is implemented in five different formalisms: 1. A custom build system that parses, analyzes, and transforms programs, which is incremental only if the transformation is a separate transformation (i.e., does not depend on other files or modules). 2. A language registry which (re)loads languages dynamically, enabling live language development. 3. Maven POM files which instruct Maven to download Java dependencies and compile Java code. 4. The meta-language bootstrapping system from the previous chapter. 5. An incremental build system for building language specifications, based on Pluto [36], a sound and optimal incremental build system with dynamic dependencies. Because the pipeline is specified in five different formalisms, it is harder to understand and change, and several opportunities for incrementality are lost because different formalisms do not properly communicate. Vision Given this problem analysis, and the desire to develop correct and responsive interactive programming systems for language workbench pipelines, we need a systematic method for developing these pipelines. This method should have a single formalism in which pipelines can be concisely and directly expressed, without the pipeline developer having to explicitly think about incrementality. Pipelines specified in this formalism are correctly and incrementally executed, and scale down to low-impact changes and large inputs. In the next chapter (chapter 7), we show PIE, a formalism for describing pipelines, and a runtime for correctly and incrementally executing these pipelines. PIE reuses the incremental build algorithm from Pluto [36]. However, Pluto’s incremental build algorithm does not scale because it needs to traverse the entire dependency graph after each change, which becomes slow with many low-impact changes and large dependency graphs. In chapter 8 we solve this scalability problem with a new change-driven incremental build algorithm.

82 PIE: A DSL, API, and Runtime for Interactive Software Development Pipelines Abstract 7 Context. Software development pipelines automate essential parts of the software engineering processes, such as compiling and continuous integration testing. In particular, interactive pipelines, which process events in a live environment such as an IDE, require responsive results for low-latency feedback, and persistence to retain low-latency feedback between restarts. Inquiry. Developing an incrementalized and persistent version of a pipeline is one way to improve responsiveness, but requires implementation of dependency tracking, cache invalidation, and other complicated and error- prone techniques. Therefore, interactivity complicates pipeline development if responsiveness and persistency become responsibilities of the pipeline programmer, rather than being supported by the underlying system. Systems for programming incremental pipelines exist, but do not focus on ease of development, requiring a high degree of boilerplate, increasing development and maintenance effort. Approach. We develop PIE, a DSL, API, and runtime for developing interactive software development pipelines, where ease of development is a focus. The PIE DSL is a statically typed and lexically scoped language. PIE programs are compiled to programs implementing the API, which the PIE runtime executes in an incremental and persistent way. Knowledge. PIE provides a straightforward programming model that enables direct and concise expression of pipelines without boilerplate, reducing the development and maintenance effort of pipelines. Compiled pipeline programs can be embedded into interactive environments such as code editors and IDEs, enabling timely feedback at a low cost. Grounding. Compared to the state of the art, PIE reduces the code required to express an interactive pipeline by a factor of 6 in a case study on syntax- aware editors. Furthermore, we evaluate PIE in two case studies of complex interactive software development scenarios, demonstrating that PIE can handle complex interactive pipelines in a straightforward and concise way. Importance. Interactive pipelines are complicated software artifacts that power many important systems such as continuous feedback cycles in IDEs and code editors, and live language development in language workbenches. New pipelines, and evolution of existing pipelines, is frequently necessary. Therefore, a system for easily developing and maintaining interactive pipelines, such as PIE, is important.

83 7.1 Introduction A pipeline is a directed acyclic graph of processors in which data flows from the output of one processor to the input of its succeeding processors. Pipelines are ubiquitously used in computer hardware and software. E.g., in hardware, CPUs contain instruction pipelines that allow interleaved execution of multiple instructions that are split into fixed stages. Software pipelines compose software components by programmatically connecting their input and output ports (e.g., UNIX pipes). In software development, pipelines are used to automate parts of the software engineering process, such as building software systems via build scripts, or continuously testing and integrating the composition of subsystems. Such pipelines are suitable for batch-processing, and often run isolated on remote servers without user interaction. Interactive software development pipelines build software artifacts, but react instantly to changes in input data and provide timely feedback to the user. Typical examples are continuous editing of source code in an IDE, providing feedback through editor services such as syntax highlighting; selective re- execution of failing test cases in the interactive mode of a build system during development; or development of languages in a language workbench [39]. Interactive pipelines focus on delivering timely results when processing an event, such that the user can subsequently act on the results. Furthermore, an interactive software development pipeline should persist its state on non-volatile memory so that a session can be restarted without re-execution. Especially in the context of an IDE, restarting the development environment should not trigger re-execution of the entire pipeline, especially if pipeline steps are costly, such as advanced static analyses [141]. Interactivity complicates the development of pipelines, if timeliness and persistency become responsibilities of the pipeline programmer, rather than being supported by the underlying system. Developing an incrementalized version of an expensive operation is one way to reduce the turnaround time when re-executing the operation. However, implementing support for incrementality in a pipeline is typically complicated and error-prone. Similarly, persisting the result of expensive operations reduces the turnaround time when restarting a session, but requires tedious management of files or a database. Furthermore, when persistency is combined with incrementality, dependency tracking and invalidation is required, which is also complicated and error-prone. Therefore, an expressive system for easily developing correct incremental and persistent interactive software development pipelines is required. One system that partially achieves this is Pluto [36], a sound and optimal incremental build system. Pluto supports dynamic dependencies, meaning that dependencies to files and other build steps are created during build execution (as opposed to before or after building), enabling both increased incrementality through finer-grained dependencies, and increased expressiveness. While Pluto focusses on build systems, it is well suited for expressing correct incremental and persistent pipelines. However, ease of development is not a focus of Pluto, as pipelines are implemented as Java classes, requiring significant

84 boilerplate which leads to an increase in development and maintenance effort. Furthermore, persistence in Pluto is not fully automated because pipeline developers need to manually thread objects through pipelines to prevent hidden dependencies, and domain-specific features such as file operations are not first class. These are open problems that we would like to address. In this paper, we introduce PIE, a DSL, API, and runtime for programming interactive software development pipelines, where ease of development is a focus. The PIE DSL provides a straightforward programming model that enables direct and concise expression of pipelines, without the boilerplate of encoding incrementality and persistence in a general-purpose language, reducing development and maintenance effort. The PIE compiler transforms high-level pipeline programs into programs implementing the PIE API, resulting in pipeline programs that can be incrementally executed and persisted to non-volatile memory to survive restarts with the PIE runtime. Compiled pipeline programs can be embedded in an interactive environment such as an IDE, combining coarse grained build operations with fine-grained event processing. To summarize, the paper makes the following contributions:

• The PIE language, a DSL with high-level abstractions for developing interactive software development pipelines without boilerplate. • The PIE API for implementing foreign pipeline functions, and as a compilation target for the DSL, with reduced boilerplate. • The PIE runtime that executes pipelines implemented in the API in an incremental and persistent way, which fully automates persistence and automatically infers hidden dependencies. • An evaluation of PIE in two critical case studies: (1) modeling of the pipeline of a language workbench in an IDE setting, and (2) a pipeline for incremental performance testing.

The PIE implementation is available as open source software [84].

Outline. The paper continues as follows. In section 7.2 we describe requirements for interactive software development pipelines, review the state of the art, and list open problems. In section 7.3 we illustrate PIE by example. In section 7.4 we describe the PIE API and runtime. In section 7.5 we describe the syntax, static semantics, and compilation of the PIE DSL in more detail. In sections 7.6 and 7.7 we present critical case studies of the application of PIE in an interactive language workbench and an interactive benchmarking setting. In section 7.8 we discuss related work. In section 7.9 we discuss directions for future work. Finally, we conclude in section 7.10.

7.2 Problem Analysis

In this section, we ﬁrst describe requirements for interactive software development pipelines, review the state of the art, and list open problems.

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 85 Error Code messages Editor

Text Generate Parse Parse Table

Figure 7.1: Example of an interactive software development pipeline, where text from a code editor is parsed, and parse error messages are displayed in the editor.

7.2.1 Requirements We first describe the requirements for interactive software development pipelines. In order to do so, we use the example pipeline from fig. 7.1 as the running example in this section. In this pipeline, a code editor parses its text buffers in order to display error messages interactively to the programmer. Parsing requires a parse table, which is generated by an external process and may change when a new version of a language is deployed, with new syntax that requires regeneration of the parse table. We identify the following requirements for interactive software development pipelines: Incrementality. A pipeline should attempt to recompute only what has been affected by a change. For example, when only a text buffer in the code editor changes, the pipeline reparses the text and new error messages are displayed, but the generated parse table is reused because it did not change. Correctness. Incremental pipeline executions must have the same results as from-scratch batch executions. For example, if the parse table does change, the pipeline also reparses text and displays new error messages. Persistence. Results of computation should be persisted to disk in order to enable incrementality after a restart of the pipeline. For example, if we restart the code editor, the parse table is retrieved from disk instead of requiring a lengthy recomputation. Expressiveness. In practice, pipelines are a lot more complex than the simple example shown here. It should be possible to express more complex pipelines as well. Ease of development. Pipelines are complex pieces of software, especially when the previous requirements are involved. Therefore, the development and maintenance effort of pipelines should be low. 7.2.2 State of the Art We now review the state of the art in interactive software development pipelines, and determine to what extent existing tools meet the requirements, focussing on build systems. Make [133], and systems with similar dependency management (e.g., Ninja, SCons, MSBuild, CloudMake, Ant), are tools for developing build systems based on declarative rules operating on files. These tools support incremental

86 provides file requires file requires build Text (buffer 1) Parse Error messages Generate Code syntax.sdf3 syntax.tbl Parse Table Editor Error Parse messages Text (buffer 2)

Figure 7.2: Pluto dependency graph created by executing the pipeline from fig. 7.1, where the code editor has 2 open text buffers. builds, but incrementality is limited to static file dependencies which are specified up front in the build rules. Because dependencies cannot be the result of computation, the dependencies must either be soundly overapproximated, which limits incrementality, or underapproximated, which is unsound. For example, a Makefile that determines the version of a Java source file, in order to parse it with the corresponding parse table file, must depend on all parse table files instead of a single one. Therefore, a system that supports a more expressive dependency mechanism is required. A detailed discussion of dependency expressiveness can be found in section 7.8, but in this section, we focus on the system with the highest dependency expressivity: Pluto. Pluto [36] is a sound and optimal incremental build system with support for dynamic dependencies. A build system in Pluto is implemented in terms of builders, which are functions that perform arbitrary computations and dynamically record dependencies to files and other builders during execution. Executing a builder with an input produces a build, containing an output object and recorded dependencies. Figure 7.2 illustrates the dependency graph Pluto produces when it executes the pipeline of fig. 7.1 where the code editor has two open text buffers. The dependency graph differs from the pipeline by containing builds (function calls) instead of builders (function definitions). For example, the pipeline has one parse builder, but two parse builds, one for each text buffer. We use this dependency graph to illustrate Pluto’s adherence to requirements for interactive pipelines: Incrementality. The code editor has two text buffers open, which have separate dependencies to a parse build. When one text buffer changes, only the corresponding parse build is recomputed. Therefore, Pluto supports fine-grained incrementality. Correctness. The parse builds depend on the parse table build, such that when the parse table is regenerated, the parse builds are re-executed, and new error messages are displayed in the editor. Pluto enforces this by performing hidden dependency detection. That is, if a build requires a file, without requiring the build that provides that file, Pluto marks this as an error and aborts execution. Persistence. While not shown in the dependency graph, builds are persisted

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 87 to disk to survive restarts. Expressiveness. Dependencies are recorded during build execution, allowing builds to depend on files or call other builds, based on results of computation. For example, when parsing Java code, the parse builder may choose to depend on a different parse table, based on whether we want to parse text of version 8 or 9 of Java. This greatly increases the expressiveness required for interactive software development pipelines. Ease of development. Pluto build systems are implemented in Java, requiring significant boilerplate. To summarize, Pluto provides a great foundation for implementing interactive software development pipelines, but does not cater to the pipeline developer because ease of development is not a focus, leading to a higher implementation and maintenance effort than necessary. 7.2.3 Open Problems The main problem is that Pluto build systems are not easy to develop. We list four concrete open problems. Boilerplate. Pipelines in Pluto are written in Java, which has a rigid and verbose syntax, requiring significant boilerplate. Pipelines are implemented as classes extending the Builder abstract class, as seen in listing 7.1. Such a class requires generics for specifying the input and output type, a factory and constructor enabling other builders to create instances of this builder to execute it, a persistentPath method for persistence, and finally a build method that performs the actual build computation. The Parse builder requires an inner class for representing multiple input values, which must correctly implement equals and hashCode, which Pluto uses to detect if an input has changed for incrementality. Finally, calling other builders through requireBuild is verbose, because the factory is referenced, and the result is unwrapped with .val. Semi-automated persistence. Pipeline developers are required to implement the persistentPath method of a builder and return a unique and deterministic filesystem path where the result of the builder and its input are persisted. It must be unique to prevent overlap with other builders or other inputs. For example, if the Parse builder persists results to the same file for different text buffers, it overwrites the persisted result of other builds. It must be deterministic such that the persisted file can later be found again. Since the OS filesystem is used for persistence, there are also limitations to which characters can be used in paths, and to how long a path can be. For example, on Windows, the current practical limit is 260 characters which is frequently reached with deeply nested paths, causing persistence to fail. Hidden dependencies. Hidden dependency detection is crucial for sound incremental builds, but is also cumbersome. In the pipeline in listing 7.1, we must construct a build request object for the parse table generator, pass that object to the Parse builder, and require it to depend on the parse table generator. This becomes tedious especially in larger and more complicated pipelines.

88 class GenerateTable extends Builder> { static BuilderFactory, GenerateTable> factory = BuilderFactoryFactory.of(GenerateTable.class, File.class);

GenerateTable(File syntaxFile) { super(syntaxFile); } @Override File persistentPath(File syntaxFile) { return new File("generate-table-" + hash(syntaxFile)); } @Override Out build(File syntaxFile) throws IOException { require(syntaxFile); File tblFile = generateTable(syntaxFile); provide(tblFile); return OutputPersisted.of(tblFile); } } class Parse extends Builder> { static class Input implements Serializable { File tblFile; String text; BuildRequest tblReq; Input(File tblFile, String text, BuildRequest tblReq) { this.tblFile = tblFile; this.text = text; this.tblReq = tblReq; } boolean equals(Object o) {/* omitted */ } int hashCode() {/* omitted */ } } @Override Out build(Input input) throws IOException { requireBuild(input.tblReq); require(input.tblFile); return OutputPersisted.of(parse(input.tblFile, input.text)); } /* ... other required code omitted ... */ } class UpdateEditor extends Builder> { @Override Out build(String text) throws IOException { File syntaxFile = new File("syntax.sdf3"); File tblFile = requireBuild(GenerateTable.factory, syntaxFile).val; BuildRequest tblReq = new BuildRequest(GenerateTable.factory, syntaxFile); return requireBuild(Parse.factory, new Parse.Input(tblReq, tblFile, text)); } /* ... other required code omitted ... */ }

Listing 7.1: The parsing pipeline implemented as Java classes in Pluto.

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 89 Editor (5) Text (1) (2) (3)

lexical.sdf normalize lexical.norm Messages (4) generate parse.tbl parse Tokens style Styling table contextfree.sdf normalize contextfree.norm AST

Figure 7.3: The example pipeline: (1) normalization of SDF syntax deﬁnition source modules, (2) generation of a parse table from the normalized modules comprising the deﬁnition for a language, (3) parsing the text of an editor using the parse table, (4) computing the styles for the parsed tokens, and (5) displaying the styling and error messages in an editor.

Missing domain-specific features. Finally, path (handle to file or directory) and list operations, which are prevalent in software development pipelines, are not first class in general-purpose languages such as Java. Solving these concrete problems requires a proper abstraction over interactive software development pipelines, which we present in subsequent sections.

7.3 PIE by Example To solve the open problems from the previous section, we introduce PIE: a DSL, API, and runtime for developing and executing interactive software development pipelines. Pipelines in PIE have minimal boilerplate, fully automated persistence, automatically infer hidden dependencies, and have domain-specific features such as path and list operations. In this section we illustrate PIE by means of an example that combines building and interaction. We discuss the example and the requirements for this pipeline, present the pipeline in the PIE DSL, and discuss its features and execution. Example Pipeline: Syntax-Aware Editors. As example we consider a code editor with syntax styling based on a syntax definition. The pipeline to support this use case is depicted by the diagram in fig. 7.3. It generates a parse table from a syntax definition, parses the program text of an editor, computes syntax styling for each token, and finally applies the computed syntax styling to the text in the editor. We want this pipeline to be interactive by embedding it into the IDE such that changes to the syntax definition as well as changes to the text in an editor are reflected in updates to syntax styling. The example in fig. 7.3 is representative for language workbenches [38, 39], which support edits on a language definition that are immediately reflected in the programming environment that is derived from it. Concretely, we instantiate the pipeline with components from the Spoofax language workbench [75]. We process an SDF [153, 158] syntax definition in two stages. First, syntax definition modules are separately transformed (normalized) to a core language. Next, the normalized modules comprising the syntax definition for a language are transformed to a parse table. The parse table is interpreted by a scannerless parser [152, 15] to parse the contents of an editor, returning an AST, token stream, and error messages. A syntax

90 func normalize(file: path, includeDirs: path*) -> path={ requires file;[requires dir with extension "sdf"| dir <- includeDirs]; val normFile= file.replaceExtension("norm"); val depFile= file.replaceExtension("dep"); exec(["sdf2normalized"] +"$file"+["-I$dir"| dir <- includeDirs] + "-o$normFile"+ "-d$depFile"); [requires dep by hash| dep <- extract-deps(depFile)]; generates normFile; normFile } func extract-deps(depFile: path) -> path* = foreign func generate-table(normFiles: path*, outputFile: path) -> path={ [requires file by hash| file <- normFiles]; exec(["sdf2table"] + ["$file"| file <- normFiles] + "-o$outputFile"); generates outputFile; outputFile } func exec(arguments: string*) -> (string, string) = foreign

data Ast= foreign{} data Token= foreign{} data Msg= foreign{} data ParseTable= foreign{} data Styling= foreign{} func table2object(text: string) -> ParseTable= foreign func parse(text: string, table: ParseTable) -> (Ast, Token*, Msg*) = foreign func style(tokenStream: Token*) -> Styling= foreign func update-editor(text: string) -> (Styling, Msg*) = { val sdfFiles=[./lexical.sdf, ./contextfree.sdf]; val normFiles=[normalize(file,[./include]) | file <- sdfFiles]; val parseTableFile= generate-table(normFiles, ./parse.tbl); val(ast, tokenStream, msgs) = parse(text, table2object(read parseTableFile)); (style(tokenStream), msgs) }

Listing 7.2: PIE DSL program for the pipeline illustrated in fig. 7.3. Identifiers of foreign functions and data types are omitted for brevity. highlighter annotates tokens in the token stream with styles. Integrated Pipelines with the PIE DSL. Listing 7.2 shows the pipeline program in the PIE DSL. We first explain what each function does, and then discuss the features and execution of PIE in more detail. The normalize function executes a command-line tool to normalize an SDF source file into a normalized version that is ready for parse table generation, and retrieves (dynamic) dependencies from the generated dependency (.dep) file, implemented by the extract-deps foreign function. The generate-table function executes a command-line tool on normalized files, creating a parse table file. The parse function, when given a parse table object, parses text into an AST, token stream, and error messages. The style function produces a styling based on a token stream, which can be used in source code editors for styling the text of the source code. Finally, the update-editor function defines the complete pipeline by composing all previously defined functions. Composing Pipelines with Functions. In PIE, pipelines are defined in terms of function definitions which are the reusable processors of the pipeline, and function calls that compose these processors to form a pipeline. Function calls

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 91 register a dynamic call dependency from caller to callee. Domain-Specific Types and Dependencies. Since build pipelines often interact with files and directories, PIE has native support for path types and several operations on paths. Path literals such as ./lexical.sdf provide an easy way to instantiate relative or absolute paths. The requires operation dynamically registers a path dependency from the current function call to the path, indicating that the function reads the path, whereas generates records a dependency indicating the function creates or writes to the path. The read operation reads the text of a given path, and also registers a path dependency. Path dependencies to directories can specify a filter such as with extension "sdf" to only create dependencies to files inside the directory that match the filter. Finally, path dependencies can specify how changes are detected. For example requires dep by hash indicates that a change is only detected when the hash of the file changes, instead of the (default) modification date, providing more fine grained dependency tracking.

Foreign Functions and Types. Some functions are foreign, indicating that they are implemented outside of the PIE DSL, either because they are outside of the scope of the DSL (e.g., text processing required for extract-deps), or because they require system calls. For example, exec is a foreign function that takes a list of command-line arguments, executes a process with those arguments, and returns its standard output and error text. Unlike read, exec is not first class, because it does not induce special (path) dependencies. PIE contains several built-in types such as string and bool, but foreign types can be defined to interface with existing types. Foreign data types are required to integrate with existing code, such as an editor that expects objects of type Styling and Msg, returned by foreign functions parse and style. Comprehensions. Pipelines frequently work with lists, which are natively supported in PIE by annotating a type with * multiplicity. Lists are instantiated with list literals between [], and concatenated using +. List comprehensions such as [f(elem) | elem <- elems] transform a list into a new list by applying a function f to each element of the list. Execution and IDE Integration. To execute a pipeline, we compile it into a program implementing the PIE API. We embed the compiled pipeline, together with the PIE runtime, into an IDE such as Eclipse. When an editor in Eclipse is opened or changed, it calls the update-editor function through the PIE runtime with the text from the editor. The PIE runtime then incrementally executes (and persists the results of) the pipeline and returns Styling and Msg objects, which Eclipse displays in the editor. Because the results of the pipeline are persisted, a restart of Eclipse does not require re-execution of the pipeline. This becomes especially important with larger pipelines. Solutions to Open Problems. PIE solves the open problems listed in section 7.2. First of all, PIE minimizes boilerplate by enabling direct expression of pipelines in the PIE DSL through function definitions, function calls, foreign data types and functions, and path dependencies. The compiler of the DSL generates

92 the corresponding boilerplate. Furthermore, PIE supports fully automated persistence. There is no need to specify where the result of a function call is stored. The PIE runtime stores results automatically based on the function name and input arguments. It persists the function arguments, return value, and dependencies in a key-value store, preventing filesystem issues. Hidden call dependencies are automatically inferred. In other words, when a function requires files that are generated by another function, the first function does not explicitly need to call the latter. For example, generate-table requires files that normalize generates, but does not need to explicitly call normalize to record a call dependency which keeps the required files up-to-date. The PIE runtime infers these dependencies by keeping track of which function call generated a file, further reducing boilerplate. Note that this only infers call dependencies, not path dependencies, which still need to be declared by the pipeline programmer. Finally, the PIE DSL caters to the pipeline developer by including domain- specific features – such as path type and operations, list type and comprehensions, string and path interpolation, and tuples – to make pipeline development convenient. Solving these problems reduces the implementation and maintenance effort. The equivalent Pluto implementation for this pipeline requires 396 lines of Java code in 8 files (excluding comments and newlines), whereas the PIE implementation is over 6 times shorter by only requiring 62 lines of code in 2 files. The PIE code consists of 34 lines of PIE DSL code, and 28 lines of PIE API code for interfacing with foreign functions.

7.4 PIEAPI and Runtime In this section, we review the PIE API and runtime, and our reasons for not directly reusing the Pluto runtime. 7.4.1 API The PIE API is a Kotlin [71] library for implementing PIE function definitions on the JVM. Kotlin is a programming language with a focus on reducing verbosity and increasing extensibility compared to Java, while maintaining fully compatible with Java by running on the JVM. It shares many goals with Scala [34], but additionally focusses on fast compile times and simplicity. We chose to specify the API in Kotlin instead of Java, because it has a more flexible and concise syntax. The PIE API is heavily based on the Pluto API, but uses terminology from the pipeline domain (functions instead of builders), and requires less boilerplate. Listing 7.3 illustrates the parsing pipeline implemented in the PIE API.A pipeline function definition is implemented by creating a class which subtypes the Func interface and overrides the exec function. The exec function takes an input, is executed in an execution context ExecContext, and produces an output. The execution context enables calling other pipeline functions through the requireCall function, and recording of path dependencies through the require and generate functions, using Kotlin’s extension functions to make

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 93 typealias In = Serializable typealias Out = Serializable interface Func { fun ExecContext.exec(input: I): O } interface ExecContext{ fun> requireCall(clazz: KClass, input: I, stamper: OutputStamper = OutputStampers.equals): O fun require(path: PPath, stamper: PathStamper = PathStampers.modified) fun generate(path: PPath, stamper: PathStamper = PathStampers.hash) }

class GenerateTable: Func { override fun ExecContext.exec(syntaxFile: PPath): PPath { require(syntaxFile) val tableFile = generateTable(syntaxFile) generate(tableFile) return tableFile } } class Parse: Func { data class Input(val tableFile: PPath, val text: String): Serializable override fun ExecContext.exec(input: Input): ParseResult { require(input.tableFile) return parse(input.tableFile, input.text) } } class UpdateEditor: Func { override fun ExecContext.exec(text: String): ParseResult { val tableFile = requireCall(GenerateTable::class, path("syntax.sdf3")) return requireCall(Parse::class, Parse.Input(tableFile, text)) } }

Listing 7.3: The PIE Func API and implementation of a parsing pipeline in that API. these functions accessible without a qualiﬁer. The PIE runtime uses this execution context for dependency tracking and hidden dependency inference. Inputs of Func implementations must be immutable, Serializable, and have an equals and hashCode implementation. These properties are required so that the PIE runtime can assume objects do not change inside a cache, can persist objects to non-volatile memory, and can detect if an object has changed for incrementality. The types used in listing 7.3 all adhere to these properties. Fur- thermore, Kotlin’s data classes automatically implement equals and hashCode, reducing boilerplate for multiple input arguments. Outputs of functions must adhere to the same properties, with the exception that outputs can opt-out of serialization. Some outputs are in-memory object representations and cannot be serialized, are too large to be serialized, or are not immutable. PIE supports these kind of objects as outputs of function calls, by wrapping the output in a special class (OutTransient) which prevents serialization. PIE still caches these outputs in volatile memory. However, when the runtime is restarted (thus clearing the in-memory cache), and such an

94 output is requested by calling the function, PIE re-executes the function to recreate the output. Although it is possible to implement a full pipeline directly in this API, there is more boilerplate involved compared to writing the pipeline in the PIE DSL. Therefore, the API should only be used for implementing foreign functions, such as interfacing with a parse table generator and parser, or for executing system calls such as executing command-line tools. However, reduced boilerplate for implementing foreign functions reduces implementation and maintenance effort. 7.4.2 Runtime

The job of the PIE runtime is to execute a pipeline – represented as a set of Func implementations, from compiled PIE DSL code, and from foreign function implementations against the PIE API – in an incremental and persistent way. The runtime is largely based on the Pluto runtime, from which we inherit the sound and optimal incremental and persistent build algorithm. However, we incorporate fully automated persistence and hidden dependency inference in the PIE runtime. The runtime calls a Func by calling its exec function with an input argument, under an execution context. During execution, a function may call other functions, and record path dependencies, through the execution context, and finally return a value. After a function has been executed, the runtime persists the returned value and recorded dependency information in a key-value store, by mapping the function call (Func instance and input argument) to the returned value and dependency information. This mapping is used by the incremental build algorithm as a cache and for retrieving dependency information. We use the LMDB [140] key-value database, which persists to a single file on the filesystem, and is memory-mapped for fast read access. Therefore, we fully automate persistence, meaning that pipeline developers are freed from reasoning about persistence. To infer hidden dependencies, whenever a path (handle to file or directory) is generated, the runtime maps (in the key-value store) the path to the function call that generated the path. Whenever a path is required, the runtime consults the mapping to look up if that path was generated by a function call. If it was, then a function call dependency is inferred from the current executing function call to the function call that generated the path. For example, in listing 7.3, a call of GenerateTable generates the parse table file, which a call of Parse requires. The runtime then infers a dependency from the Parse call to the GenerateTable call. This is sound, because there may be at most one function call that generates a single path. We validate this property and abort execution when multiple function calls generate a single path. Therefore, we automatically infer hidden dependencies. 7.4.3 Reusing the Pluto Runtime We have implemented our own API and runtime, instead of reusing the Pluto runtime, for the following three reasons. First of all, we reimplemented parts

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 95 of the Pluto runtime in order to better understand Pluto’s incremental rebuild algorithm and concrete implementation. Second, we wanted to reduce boilerplate for writing foreign functions. Third, automated persistence would be hard to implement in Pluto, because Pluto requires every pipeline function to implement a persistentPath function (as seen in listing 7.1), which returns a unique filesystem path for persisting the result of executing a function with a particular input. We could generate a persistentPath implementation from the PIE DSL, but then foreign functions still need to manually implement this functions. Furthermore, filesystem paths may not contain certain characters, and have size limits (e.g., 260 characters on many Windows systems), which makes using files as a persistent storage complicated and error prone. Therefore, in the PIE runtime, we persist to a memory-mapped database.

7.5 PIELanguage

In this section, we present PIE’s language definition. We present PIE’s syntax specification, describe domain-specific language constructs, and briefly look at static semantics. Finally, we describe compilation from the PIE language to the API, providing incremental and persistent pipeline execution when executed with the PIE runtime.

7.5.1 Syntax Listing 7.4 shows PIE’s syntax through an EBNF grammar specification. PIE programs are composed of (foreign) function definitions and foreign data types at the top level. Its constructs can be categorized into base constructs that can be directly translated to a general purpose language, and special constructs for the domain of interactive software development pipelines that require a special translation. Base constructs include regular unary and binary operations, control flow, list comprehensions, value declarations and references, function definitions and calls, early return or failure, literals, and string interpolation. Special constructs include path types, path literals, dependencies (requires and generates), and operations (exists, read, list, and walk); foreign function definitions and calls; and foreign data definitions. We intentionally keep PIE’s constructs simple in order to support incrementality and persistence, with concise expression of pipelines, while still supporting a wide range of different pipelines. For example, PIE does not allow assignment or other forms of mutation, because mutation complicates incrementality support. Instead, immutability allows the dynamic semantics to perform caching for improved incrementality.

7.5.2 Static Semantics PIE is a statically typed and lexically scoped language. As base types, PIE has the unit type, booleans, integer, strings, paths, and user-deﬁned foreign data types. Types can be made optional (t?), into a list (t*), and composed into tuples ((t1, t2)). All data type and function deﬁnitions are explicitly typed, but types are inferred inside function bodies. Static type checks prevent

96 idchr = ?[a-zA-Z0-9-_]?; id ={idchr}; qid ={idchr | "."}; int = ["-"]{?[0-9]?};

func_head= id "(" {id ":"t, ","} ")" "->"t; func_def = "func" func_head "=" ("foreign" id|"foreign java" qid "#" id|e); data_def = "data" id [":" id] "foreign java" id "{" {"func" func_head} "}"; program ={func_def| data_def};

t = "unit"|"bool"|"int"|"string"|"path"| id|t "?" |t "*" | "(" {t, ","} ")";

e = "{" {e, ";"} "}" | "("e ")" | "!"e|e "!" |e ("==" | "!=" | "||" | "&&" | "+")e | "if" "("e ")"e ["else"e] | "["e "|" binder "<-"e "]" | "val" binder "="e | id| id "(" {e, ","} ")" |e "." id "(" {e, ","} ")" | "requires"e ["with" filter] ["by" stamper] | "generates"e ["by" stamper] | "exists"e | "read"e | "list"e ["with" filter] | "walk"e ["with" filter] | "return"e | "fail"e | "unit" | "true" | "false" | int | "null" | "(" {e, ","} ")" | "[" {e, ","} "]" | '"' {?~[\"\$\n\r]? | '\\$' | '\\"' | "$" id | "${"e "}"} '"' | ["."] "/" {?~[\n\r\$\,\;\]\)\ ]? | '\\ ' | '\\$' | "$" id | "${"e "}"};

binder= bind | "(" {bind, ","} ")"; bind= id| id ":"t;

filter = ("regex" | "pattern" ["s"] | "extension" ["s"])e;

stamper = "exists" | "modified" | "hash";

Listing 7.4: PIE’s syntax deﬁnition in a dialect of EBNF. mistakes in the pipeline from appearing at runtime. For example, it is not possible to call a pipeline function with an argument of the wrong type, as PIE’s type checker will correctly mark this as a type error. Name binding prevents mistakes such as duplicate deﬁnitions and unresolved references.

7.5.3 Compilation

To execute a PIE program with the PIE runtime, we compile it to a Kotlin program implementing the PIE API. We compile every function deﬁnition in the program to a class implementing Func, with corresponding input and output types, and compile its function to the exec method. Multiple function arguments, as well as tuple types, are translated into an immutable data class, implementing the required equals, and hashCode functions, and the Serializable interface. Function calls are compiled to requireOutput calls on the execution context, which records a function call dependency and incrementally executes that function. Path dependencies are translated to require and generate calls on the execution context, which records path dependencies, and which infers hidden

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 97 dependencies when requiring a generated file. Path dependencies can use different stampers, which instruct the PIE runtime as to how generated and required paths are checked for changes during incremental execution. The exists stamper checks that a file or directory exists, modified compares the modification date of a file or directory, and hash compares the hash of a file, or the hashes of all files in a directory. The exists, read, list, and walk path operations are translated to function calls of built-in functions that perform these tasks and register the corresponding path dependencies. For example, the walk construct recursively walks over files and directories in a top-down fashion, returns them, and registers dependencies for each visited directory. Some path constructs also accept a filter that filters down the visited files and directories. For example, a requires on a directory with a filter only creates path dependencies for files and directories that are accepted by the filter. A regular expression, ANT pattern, or file extension filter can be used. Other constructs (ones that do not affect incrementality or persistence) are compiled directly to Kotlin expressions. For example, list comprehensions are translated to maps.

7.6 Case Study:Spoofax Language Workbench We evaluate PIE using two critical [43] case studies that are representative for the domain of interactive software development pipelines. In this section we discuss a case study in the domain of language workbenches. In the next section we discuss a case study in the domain of benchmarking. Spoofax [75] is a language workbench for developing textual programming languages. Spoofax supports simultaneous development of a language definition and testing the programming environment generated from that language definition. This requires complex pipelines, including bootstrapping of languages [86]. In this case study we evaluate the feasibility of implementing the Spoofax pipeline using PIE. In the Spoofax ecosystem, a programming language is specified in terms of multiple high-level declarative meta-language definitions, where each meta- language covers a language-independent aspect (e.g., separate syntax definition [153], name binding rules [5, 109, 89], or the dynamic semantics definition of a programming language [147]). Subsequently, Spoofax generates a complete implementation of a programming language, given all the meta-language definitions. Dividing a programming language implementation into linguistic abstractions in terms high-level meta-language definitions is the key enabler for maintainability of a language, however it complicates the necessary (interactive) software development pipelines. Spoofax supports interactive language development in the Eclipse IDE, including developing multiple language specifications side-by-side. In contrast to a regular IDE that solely processes changes of source files in the source language, Spoofax additionally comes with support for interactive software development pipelines that respond to language specification changes. For example, changes to the syntax specification are reflected by reparsing source files of the language. In order to achieve this goal, Spoofax will: (1) execute

98 a pipeline to regenerate the language implementation based on the language specification, (2) reload the updated language implementation into the language registry, and (3) execute a pipeline for all open source files of the changed language. The pipeline for source files will: (1) parse the source file into an AST and token stream, (2) generate syntax styling based on the token stream, (3) show parse errors (if any) and apply syntax styling, and (4) analyze and transform the source file. 7.6.1 Pipeline Re-Implementation We have implemented Spoofax’s management of multiple languages, parsing, and syntax-based styling with the PIE pipeline that is illustrated in listing 7.5. This is an extension to the example pipeline of section 7.3, but is still a subset of the complete pipeline due to space constraints. We omit the foreign keyword for brevity. Language Specification Management. The first part of the pipeline is used to manage multiple language specifications. The LangSpec data type represents a language specification, which has a file extension and configuration required for syntax specification and styling. The Workspace type represents a workspace with multiple language specifications, which has a list of relevant file extensions, and a function to get the LangSpec for a path based on its extension. The aforementioned data types are similar to classes by binding function definitions to them. In this particular case their implementations are foreign (i.e., implemented in a JVM language), but registered in PIE in order for using them in an interactive software development pipeline. An instance of the Workspace (which contains LangSpecs) is created by the getWorkspace function from a configuration file. Interfacing with foreign functions and data types is a key enabler for embedding PIE pipelines in other programs, while still benefiting from domain-specific features such as dependency tracking. Parse Table Generation, Parsing, and Styling. The second part implements parsing. There are several foreign data and function definitions which bind to Spoofax’s tools. For example, sdf2table takes a specification in the SDF meta- language, and produces a ParseTable which can be used to parse programs with the jsglrParse function. The parse function takes as input the text to parse and the language specification containing the syntax specification mainFile to derive a parser from, creates a parse table for the language specification, and uses that to parse the input text. Parsing returns a product type containing the Ast, Tokens, and error Messages. Since parsing can fail, the AST and tokens are annotated with ? multiplicity to indicate that they are nullable (optional). The third part implements syntax-based styling, similarly to parsing. Processing Files in the IDE. The fourth part combines parsing and styling to process a single string or file and return the error messages and styling, which we can display in the Eclipse IDE. The fifth and sixth parts interface with the Eclipse IDE, by providing functions to keep an Eclipse project and editor up-to-date. A project is kept up-to-date by walking over the relevant files of

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 99 // 1) Language specification and workspace management data LangSpec={ func syntax() -> path; func startSymbol() -> string; func styling() -> path } data Workspace={ func extensions() -> string*; func langSpec(path) -> LangSpec } func createWorkspace(string, path) -> Workspace func getWorkspace(root: path) -> Workspace={ val text= read(root+ "/workspace.cfg"); createWorkspace(text, root) } // 2) Creating parse tables and parsing data ParseTable{} data Ast{} data Token{} data Msg{} func sdf2table(path) -> ParseTable func jsglrParse(string, string, ParseTable) -> (Ast?, Token*?, Msg*) func parse(text: string, langSpec: LangSpec) -> (Ast?, Token*?, Msg*) = { val mainFile= langSpec.syntax(); requires mainFile; val startSymbol= langSpec.startSymbol(); val table= sdf2table(mainFile); jsglrParse(text, startSymbol, table) } // 3) Syntax-based styling data SyntaxStyler{} data Styling{} func esv2styler(path) -> SyntaxStyler func esvStyle(Token*, SyntaxStyler) -> Styling func style(tokens: Token*, langSpec: LangSpec) -> Styling={ val mainFile= langSpec.styling(); requires mainFile; val styler= esv2styler(mainFile); esvStyle(tokens, styler) } // 4) Combine parsing and styling to process strings and files func processString(text: string, langSpec: LangSpec) -> (Msg*, Styling?) = { val(ast, tokens, msgs) = parse(text, langSpec); val styling= if(tokens != null) style(tokens, langSpec) else null; (msgs, styling) } func processFile(file: path, langSpec: LangSpec) -> (Msg*, Styling?) = processString(read file, langSpec) // 5) Keep files of an Eclipse project up-to-date func updateProject(root: path, project: path) -> (path, Msg*, Styling?)* = { val workspace= getWorkspace(root); val relevantFiles= walk project with extensions workspace.extensions(); [updateFile(file, workspace) | file <- relevantFiles] } func updateFile(file: path, workspace: Workspace) -> (path, Msg*, Styling?) = { val langSpec= workspace.langSpec(file); val(msgs, styling) = processFile(file, langSpec); (file, msgs, styling) } // 6) Keep an Eclipse editor up-to-date func updateEditor(text: string, file: path, root: path) -> (Msg*, Styling?) = { val workspace= getWorkspace(root); val langSpec= workspace.langSpec(file); processString(text, langSpec) }

Listing 7.5: Spoofax pipeline in PIE, with support for developing multiple language speciﬁcations, parsing, syntax styling, and embedding into the Eclipse IDE.

100 the project, and returning the messages and styling for each ﬁle which are displayed in Eclipse. An editor is kept up-to-date by processing the text in the editor.

7.6.2 Analysis

In this section we discuss the observations we made while re-implementing the incremental software development pipeline of Spoofax in PIE. Overall, the re-implementation improves on the areas mentioned below.

Canonical Pipeline Formalism. The main benefit over the old pipeline of Spoofax is that the PIE re-implementation is written in a single and concise formalism that is easier to understand and maintain. The old pipeline of Spoofax is comprised of code and configuration in four different formalisms: 1) Maven Project Object Model (POM) file that describes the compilation of Java source code, 2) an incremental build system using the Pluto [36] Java API and runtime that builds language specifications, 3) a custom (partially incremental) build system for building and bootstrapping meta-languages, and 4) a custom language registry that manages multiple language specifications. Incrementality and persistence are only partially supported, and implemented and maintained explicitly. In contrast, the PIE pipeline is specified as a single formalism in a readable, concise, and precise way, without having to implement incrementality and persistence explicitly.

Exact (Dynamic) Dependencies. Spoofax’s old pipeline emits dependencies that are either overapproximated or underapproximated, resulting in poor incrementality and therefore longer execution times. For example, in Spoofax, changing the styling specification will trigger parsing, analysis, compilation, and styling for all editors, even though only recomputation of the styling is required (i.e., sound overapproximation). On the other hand, changing the syntax specification will not trigger reparsing of files that are not open in editors (i.e., unsound underapproximation). In the PIE pipeline, these problems do not occur because of the implicit incrementality of function calls, and the right path dependencies. For example, the parse function creates several dependencies which enable incremental recomputation. When the input text, mainFile path, contents of the mainFile, or the startSymbol changes, the function is recomputed. Furthermore, the function creates a parse table, which is a long-running operation. However, because of incremental recomputation and persistence, the parse table is computed once, and after that only when the syntax specification changes.

Support for Complex Pipeline Patterns. Due to space constraints, listing 7.5 omits the parts necessary for using Spoofax’s name binding language and constraint solver, interfacing with existing Spoofax languages, and bootstrapping languages, but our re-implementation does support the aforementioned features. The full implementation can be found online [85].

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 101 func main(jmhArgs: string*) -> path* = { val jar= build(); val pkg= "io.usethesource.criterion"; val javaSrcDir= ./src/main/java/io/usethesource/criterion; val benchs:(string, string, path*)* = [ // Benchmarks name, pattern, classes ("set","$pkg.JmhSetBenchmarks.(*)\$",[javaSrcDir+"/JmhSetBenchmarks.java"]) ,("map","$pkg.JmhMapBenchmarks.(*)\$",[javaSrcDir+"/JmhMapBenchmarks.java"]) ]; val subjs:(string, string, path*)* = [ // Subjects name, identifier, libs ("clojure", "VF_CLOJURE",[./lib/clojure.jar]) ,("champ", "VF_CHAMP",[./lib/champ.jar]) ,("scala", "VF_SCALA",[./lib/scala.jar]) ,("javaslang", "VF_JAVASLANG",[./lib/javaslang.jar]) ,("unclejim", "VF_UNCLEJIM",[./lib/unclejim.jar]) ,("dexx", "VF_DEXX",[./lib/dexx.jar]) ,("pcollections", "VF_PCOLLECTIONS",[./lib/pcollections.jar]) ]; [run_benchmark(jar, jmhArgs, bench, subj) | bench <- benchs, subj <- subjs] } func build() -> path={ val pomFile= ./pom.xml; requires pomFile; [requires file| file <- walk ./src with extensions["java", "scala"]]; exec(["mvn", "verify", "-f","$pomFile"]); val jar= ./target/benchmarks.jar; generates jar; jar } func run_benchmark(jar: path, jmhArgs: string*, bench:(string, string, path*), subj:(string, string, path*)) -> path={ val(bName, bId, bDeps) = bench;[requires dep| dep <- bDeps]; val(sName, sId, sDeps) = subj;[requires dep| dep <- sDeps]; val csv= ./results/${bName}_${sName}.csv; requires jar by hash; exec(["java", "-jar","$jar"] + bId+["-p", "subject=$sId"] + jmhArgs+ ["-rff","$csv"]); generates csv; csv }

Listing 7.6: Incremental performance benchmarking pipeline in PIE.

7.7 Case Study:Live Performance Testing In this section we evaluate PIE on a case study for continuously monitoring the performance of a set of libraries. Specifically we use a snapshot of the Criterion benchmark suite [136] that measures the performance of immutable hash-set/map data structures on the JVM. The snapshot of Criterion was submitted as a well-documented artifact to accompany the findings of a research paper [137]. Under the hood, Criterion uses the Java Microbenchmarking Harness (JMH)[69] to execute benchmark suites against seven data structure libraries, producing Comma-Separated Values (CSV) files with statistical-relevant benchmarking data. Criterion uses bash scripts for orchestration, requiring to re-run all benchmarks whenever a benchmark or subject library changes. Those scripts are not able to exploit incrementality, which is tedious since benchmarking all combinations takes roughly two days, to produce statistically significant

102 outputs. We re-engineered the pipeline such that initially each subject and benchmark combination is tested in isolation, and then incrementally re-execute all benchmarks for a particular subject if and only if that subject changes. In case the implementation of a benchmark changes, all subjects are re-tested for that benchmark. Regardless of the scenario, the CSV result ﬁles are kept up-to-date for subsequent data visualization. We can apply such a pipeline on a local machine while developing the benchmarks for timely performance test results, or on a remote benchmarking server to minimize the amount of benchmarking work when something changes. While it is technically possible to write such an incremental pipeline in bash scripts, it would require a lot of manual work to implement, and will likely result in error-prone code. Fortunately, it is straightforward to write this pipeline in PIE. 7.7.1 Pipeline Re-Implementation

Listing 7.6 illustrates the benchmarking pipeline in PIE. The build function builds the benchmark and yields an executable JAR file ./target/benchmarks.jar, by invoking Maven on the POM file ./pom.xml. The build function requires all Java and Scala source files, to ensure that the JAR file is rebuilt as soon as a single source files changes. To produce a CSV result file, the run_benchmark function executes the JAR file with the necessary command-line arguments for the JMH library, including the combination of benchmark and subject. The tuples benchmark and subject both store unique name identifiers —that are later used for naming the CSV file— and references to files they are comprised of. These file references are used by PIE to create dependencies for incremental re-execution. Finally, main glues everything together by creating a list of benchmarks and subjects, running the benchmark with each combination of those, and by returning the up-to-date CSV files for subsequent data visualization. 7.7.2 Analysis Compared to the existing bash script, the PIE pipeline provides incremental and persistent execution, and static analysis. The main benefit of the PIE pipeline over the bash script is that it provides incremental execution by function calls and path dependency annotations. In bash, implementing an incremental pipeline requires the pipeline developer to explicitly encode dependency tracking, change detection, caching, and more, which is why the existing bash script is not incremental. In the PIE pipeline, incrementality comes from stating the requires and generates dependencies in each function, which is straightforward because it is clear what the dependencies of each function are. Furthermore, PIE performs static name and type analysis, before executing the pipeline, whereas bash has no static checks at all. This means that errors such as simple typographical errors, or appending a value of a wrong type to a list of strings, result in a static error in PIE which is easily fixed, but result in

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 103 otegnrtdMkfie.I contrast, In Makefiles. generated the to — 104 ok xlsvl ihfie n omn-iepoess enn hti is it that meaning processes, command-line and files with exclusively works uoaesrpsmypoaaedfcs—ta r nydtcal trntime run at detectable only are that — defects propagate may scripts Automake rvdsafaueoeve ftessesw ics hogotti section. this throughout discuss we systems the of overview feature a provides al .:Faueoeve of overview Feature 7.1: Table omo yai ahdpnece cle ieefcsi Mk) and OMake), in side-effects (called dependencies path dynamic of form a mechanism. tracking dependency execution. pipeline before pipelines, interactive than arbitrary flexible write mostly to less follow used is it that be Automake making processes cannot compilation but Makefiles. other patterns, and generates similar compilation that C Make towards of geared top on formalism Makefiles. i.e., other generalize, loading not allows does that that directive Make dependencies include dynamic timestamps. an of on detect form based commands to limited regeneration able a the require is supports not executes Make do re-execution, and that Upon files rules, unchanged graph. these dependency the from to graph according dependency static a [ Make Table systems. 7 build on focus a with work related discuss we section this In 7 bash. in errors run-time support, tial/limited o osbet eedo h euto ucincl,o oitraewith pipelines interface interactive to for or unsuitable call, it function a making of types, result and the functions on foreign depend to OMake possible Make, not like However, dependencies. these on based incrementality o Boilerplate Low Restartable Embeddable Incrementality Implicit Deps. File Dynamic Analysis Static Cross-platform . . 8 8 Mk [ OMake [ Automake . 1 R ata oanSeicBidAbstractions Build Domain-Specific Partial 133 elated sabidatmto olbsdo elrtv ue.Mk extracts Make rules. declarative on based tool automation build a is ] 62 sabidto ihaMk-iesna,btwt richer a with but syntax, Make-like a with tool build a is ] 98 leitsmn fMk’ hrcmn yitouiga introducing by shortcoming Make’s of many alleviates ] W ork osupport). no = Make Automake

PIE OMake PIE

u oalc fsai hcig ill-typed checking, static of lack a to Due . Tup PIE n eae ok( work related and PROM ssmlrt Mk nta ohsupports both that in OMake to similar is Nix Maven PIE Ant

ace uherr statically errors such catches Gradle Jenkins

ulsupport, full = Shake Pluto Fabricate Spark Reactive Prog.

par- = Workﬂow Lang. 7 .

1 PIE which require embedding into an interactive system. Tup [127] is a build tool with Make-like rules. Tup automatically infers required file dependencies by instrumenting the build process, providing more fine-grained dependencies than Make. However, the dependency on the input file and generated file must still be declared statically upfront. PROM [79] is a Prolog-based make tool where Make-like builds are specified declaratively and executed as Prolog terms, increasing expressiveness and ease of development. PROM’s update algorithm executes in two phases, where first a file-dependency graph is created, after which creation rules are executed to create new files, or to update out-of-date files. Because of these phases, PROM does not support dynamic discovery of dependencies during build execution. Nix [28, 30] is a purely functional language for building and deploying software. One of its applications is managing the configuration of the operating system NixOS [29]. Nix supports incremental execution of pipelines through cryptographic hashes of attributes and files, but must be explicitly initiated by the developer through the use of the mkDerivation function. While incrementality becomes explicit, Nix puts the burden on pipeline developers, whereas PIE supports incrementality implicitly. Furthermore, Nix is dynamically checked, meaning that name and type defects are reported at runtime, as opposed to before runtime with static checking in PIE. Maven [45] is a software dependency management and build tool, popular in the Java ecosystem. It features a fixed sequence of pipeline steps such as compile, package, and deploy, which are configured through an XML file. Maven is neither incremental nor interactive, requiring a full batch re-execution every time data in the pipeline changes. Ant [44] is a build automation tool, using XML configuration files for defining software development pipelines. Ant supports incrementality by inserting uptodate statements that check if a source file is up to date with its target file, making incrementality explicit, at the cost of burdening the developer. Ant does not provide static analysis. Gradle [67] is a build automation tool, programmable with the Groovy language, featuring domain-specific library functions to specify builds declaratively. Gradle supports incremental task execution through annotations that specify a task’s input/output variables, files, and directories. Like Make, dependencies have to be specified statically up-front, causing an overapproximation of dependencies. Jenkins is a continuous integration server which can be programmed with its Groovy pipeline and a set of domain-specific library functions [70]. Jenkins can detect changes to a (remote) source code repository to trigger re-execution of an entire build pipeline, however without support for incrementality. 7.8.2 Software Development Pipelines as a Library Subsequently discussed software pipeline solutions are available as a library (i.e., internal DSL) implemented in a general-purpose programming language. Unlike an external DSL solution such as PIE, those libraries do not support domain-specific syntax or error reporting in terms of the domain, instead

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 105 requiring encoding of domain concepts. Furthermore, it is hard, if not impossible, to restrict features of a programming language via a library, that heavily influence incrementality, such as mutable state. Finally, the compiler of the PIE DSL can be retargeted to a different environment or programming language, enabling a PIE pipeline to be embedded into different interactive environments without (or minimal) alteration. Shake [107, 108] is a Haskell library for specifying build systems. Unlike Make, required file dependencies can be specified during builds in Shake, supporting more complex dependencies and reducing overapproximation of dependencies. However, like Make, targets (generated file dependencies) have to be specified up-front. This means that it is not possible to specify builds where the names of generated files are decided dynamically. For example, the Java compiler generates a class file for each inner class in a source file, where its file names are based on the inner and outer class name. Therefore, the generated file dependencies of the Java compiler are decided dynamically, and cannot be specified in Shake. Pluto [36] is a Java library for developing incremental builds, which we have already discussed extensively in section 7.2. One difference between Pluto and PIE is that Pluto supports incremental cyclic builders, whereas PIE does not. We have opted not to implicitly support cycles for simplicity of the build algorithm, and because cycles typically do not appear in pipelines. Cycles can be handled explicitly in PIE by programming the cyclic computation inside a single pipeline function. Fabricate [64] is a Python library for developing incremental and parallel builds, that aims to automatically infer all file dependencies by tracing system calls. System call tracing is not cross-platform, only fully supporting Linux at the moment. PIE in contrast is cross-platform, because its runtime works on any operating system the JVM runs on. Apache Spark [7] is a big data processing framework where distributed datasets are transformed by higher-order functions. PIE is similar to Spark in that both create dependency graphs between calculations. For example, when transforming a dataset with Spark (e.g., with map or filter), the derived dataset depends on the parent dataset, such that the derived dataset is rederived when the parent changes. PIE differs from Spark in that PIE works with local data only, whereas Spark works with a distributed storage system required for big data processing. However, PIE supports arbitrary computations (as opposed to a fixed set of higher-order functions in Spark), and dynamic file dependencies. 7.8.3 General-Purpose Languages Reusing an existing general-purpose language, such as Java or Haskell, and giving it an incremental and persistent interpretation is not feasible for several reasons. It requires adding additional constructs to the language, such as path dependencies and operations, which require changes to the syntax, static semantics, dynamic semantics (compiler or interpreter) of the language. That requires at the very least being able to change the language, which is not always possible. Even when it is possible, language parsers, checkers, and compilers

106 are often large codebases that require significant effort to change. Furthermore, we also need to ensure that existing constructs work under incrementality. For example, mutable state in Java interferes with incrementality. 7.8.4 Reactive Programming Reactive programming is characterized by asynchronous data stream processing, where data streams form a pipeline by composing streams with a set of stream combinators. Reactive programming approaches come in the form of libraries implemented in general-purpose languages, such as Reactive Extensions [101], or as an extended language such as REScala [126]. Reactive programming approaches provide a form of incrementality where the reactive pipeline will rerun if any input signal changes. However, they do not cache outputs or prevent re-execution of pipeline steps when there are no changes. Note that reactive programming approaches operate in volatile memory only, whereas PIE’s runtime supports persistence (i.e., pause and resume) of pipeline executions. Preserving a pipeline’s state is of special importance in interactive environments such as IDEs, to support restarting the programming environment without re-triggering potentially expensive calculations. Furthermore, reactive programming approaches do not support (dynamic) file dependencies. 7.8.5 Workflow Languages Workflows, like pipelines, describe components (processors) and how data flows between these components. Workflow languages are DSLs which are used to model data analysis workflows [4], business process management [1], and model-to-model transformations [12], among others. The crucial differenti- ation between many workflow systems and software development pipelines, is that the former model manual steps that require human interaction, whereas the latter focuses on processors that perform general purpose computations.

7.9 Future Work We now discuss directions for future work. 7.9.1 First-Class Functions and Closures Currently, the PIE DSL does not support first-class functions and closures, for simplicity. The PIE runtime does support first-class functions, since function calls are immutable and serializable values which can be passed between functions and called. However, closures are not yet supported, because (again for simplicity), functions must be registered with the runtime before a pipeline is executed. To fully support first-class functions and closures, we must add them to the PIE DSL, and support closures in the runtime and API. This requires closures to be serializable, which the JVM supports. Closures from foreign functions must ensure not to capture mutable state, non-serializable values, or large objects graphs, as these can break incrementality. Spores [105] could be used to guarantee these properties for closures.

Chapter 7. PIE: A Framework for Interactive Software Development Pipelines 107 7.9.2 Live Pipelines PIE pipelines can dynamically evolve through the inputs into the pipeline: files on the filesystem such as configuration files, and objects passed through function calls such as editor text. However, the pipeline code itself currently cannot dynamically evolve at runtime. When the pipeline code itself is changed, the pipeline must be recompiled and reloaded. This process is relatively fast, because compiling PIE DSL code and restarting the JVM is fast, but can be improved nevertheless. Furthermore, dynamic evolution of pipelines at runtime is especially important if we want to apply PIE to live programming environments. While there are known solutions for compiling and reloading code in the JVM, such as using class loaders, it is unclear how to handle incrementality in the face of changes to the pipeline program. For example, if the normalize function in listing 7.2 is changed, all calls of normalize are potentially out-of- date and need to be re-executed, as well as all function calls that (transitively) call normalize. Similarly, foreign functions and data types can be changed, which require re-execution or even data migrations in the persistent storage.

7.10 Conclusion We have presented PIE: a DSL, API, and runtime for developing interactive software development pipelines. PIE provides a straightforward programming model that enables direct and concise expression of pipelines with minimal boilerplate, reducing the development and maintenance effort of pipelines. Compared to the state of the art, PIE reduces the code required to express an interactive pipeline by a factor of 6 in a case study on syntax-aware editors. Furthermore, we have evaluated PIE on two complex interactive software development pipelines, showing that the domain-speciﬁc integration of features in PIE enable concise expression of pipelines, which are normally cumbersome to express with a combination of traditional build systems and general-purpose languages. Acknowledgements This research was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Speciﬁc Languages) and NWO VICI Project (639.023.206) (Language Designer’s Workbench).

108 Scalable Incremental Building with Dynamic Task Dependencies

Abstract Incremental build systems are essential for fast, reproducible software builds. 8 Incremental build systems enable short feedback cycles when they capture dependencies precisely and selectively execute build tasks efficiently. A much overlooked feature of build systems is the expressiveness of the scripting language, which directly influences the maintainability of build scripts. In this chapter, we present a new incremental build algorithm that allows build engineers to use a full-fledged programming language with explicit task invocation, value and file inspection facilities, and conditional and iterative language constructs. In contrast to prior work on incrementality for such programmable builds, our algorithm scales with the number of tasks affected by a change and is independent of the size of the software project being built. Specifically, our algorithm accepts a set of changed files, transitively detects and re-executes affected build tasks, but also accounts for new task dependencies discovered during building. We have evaluated the performance of our algorithm in a real-world case study and confirm its scalability.

8.1 Introduction Virtually every large software project employs a build system to resolve dependencies, compile source code, and package binaries. One great feature of build systems besides build automation is incrementality: After a change to a source or configuration file, only part of a build script needs re-execution while other parts can be reused from a previous run. Indeed, incremental build systems are a key enabler for short feedback cycles. The reliable and long-term maintainable usage of incremental build systems requires the following three properties: Efficiency. The most obvious requirement is that rebuilds must be efficient. That is, the amount of time required for a rebuild must be proportional to how many build tasks are affected by a change. Specifically, a small change affecting few tasks should only incur a short rebuild time. Precision. An incremental rebuild is only useful if it yields the exact same result as a clean build. To this end, incremental build systems must capture precise dependency information about file usage and task invocations. Make-like build systems do not offer means for capturing precise dependencies. Instead, over-approximation (*.h) leads to inefficiency because of considering too many files, and under-approximation (mylib.h) leads to incorrect rebuilds because of missing dependencies (e.g., other.h). Precise dependency information is

109 required for efﬁcient and correct rebuilds. Expressiveness. Like all software artifacts, build scripts grow during a project’s lifetime [100] and require increasing maintenance [93]. Therefore, build scripts should be written in expressive languages, avoiding accidental complexity. That is, build scripting languages should not require build engineers to apply complicated design patterns (e.g., recursive [106] or generated Makeﬁles) for expressing common scenarios.

Current incremental build systems put a clear focus on efficiency and precision, but fall short in terms of expressiveness. In particular, in order to support incremental rebuilds, current systems impose a strict separation of configuration and build stages. All variability of the build process needs to be fixed in the configuration stage, whereas the build stage merely executes a pre-configured build plan. This model contradicts reality, where how to build an artifact depends on the execution of other build tasks. We have observed two sources of variability in building. First, based on the result of other tasks, conditional building selects one of multiple build tasks to process a certain input. Second, based on the result of other tasks, iterative building invokes build tasks multiple times on different inputs. In both cases, dependencies on task invocations only emerge during the build; build engineers cannot describe these dynamic dependencies in the configuration phase. We illustrate a concrete example in section 8.2. A solution to the expressiveness problem is to provide build engineers with a full-fledged programming language. In such a system, build tasks are procedures that can invoke other build tasks in their body. Build tasks can inspect the output of invoked tasks and use that to conditionally and iteratively invoke further tasks. The problem of such a programmable build system is that it is difficult to achieve incrementality. We are only aware of a single build system that is both programmable and incremental: Pluto [36]. Unfortunately, the incremental build algorithm of Pluto has an important limitation: To check which tasks need re-execution, Pluto needs to traverse the entire dependency graph of the previous build and has to touch every file that was read or written in the previous build. This contradicts our first requirement, efficiency, because the rebuild time of Pluto depends on the size of the software project more than it depends on the size of the change. In particular, even when no file was changed, Pluto’s algorithm requires seconds to determine that indeed no task requires re-execution. We illustrate Pluto’s algorithm using an example in section 8.2. In this paper, we design, implement, and evaluate a new incremental build algorithm for build systems with dynamic task dependencies. While Pluto’s algorithm only takes the old dependency graph as input and traverses it top- down, our algorithm also takes a set of changed files and primarily traverses the dependency graph bottom-up. We can collect changed files, for example, from IDEs that manage their workspace or by using a file system watchdog. Our algorithm uses the changed files to drive rebuilding of tasks, only loading and executing those tasks that are (transitively) affected by a change. However,

110 due to dynamic task dependencies, the dependency graph can change from one build to the next one. Our build algorithm accounts for newly discovered and deleted task dependencies by mixing bottom-up and top-down traversals. Our new incremental build algorithm provides significant performance improvements when changes are small. We have conducted a real-world case study on the Spoofax language workbench, a tool built for developing domain-specific languages (DSLs). The build script of Spoofax processes DSL specification files and generates interpreters, compilers, and IDE plug-ins for them. We found that our algorithm successfully eliminates the overhead of large dependency graphs and provides efficient rebuilding that is proportional to the change size. In summary, we make the following contributions. We review programmatic build scripts, incremental building with Pluto, and why this does not scale (section 8.2). We describe our key idea of bottom-up incremental building, and what is needed to make it work (section 8.3) We present our hybrid incremental build algorithm that mixes bottom-up and top-down building (section 8.4), and briefly discuss its implementation (section 8.5). We evaluate the performance of the hybrid algorithm against Pluto’s algorithm with a case study on the Spoofax language workbench (section 8.6).

8.2 Background and Problem Statement Most build systems provide a declarative scripting language. Declarative languages are great as they let developers focus on what to compute rather than how to compute it. However, we argue that declarativity is misdirected when it comes to describing sophisticated build processes that involve conditional and iterative task application. For example, consider the build script in listing 8.1. We wrote this build script in the PIE build script language [91], which mostly provides standard programming language concepts. That is, the build script performs iterative building by defining and calling functions (tasks) like main and parseYaml, stores results of tasks in local variables such as config and src which can be immediately used by subsequent tasks, and involves conditional building with control structures like if and for. By and large, our build script is a normal program that happens to handle file paths and invoke external processes to generate and run tests. But how can we execute such a programmatic build script incrementally? Most build systems require declarative specifications of build tasks for this reason: to support efficient incremental rebuilds. However, Erdweg et al. demonstrated that it is also possible to incrementally execute programmatic build scripts, with Pluto [36], a build system that incrementally executes build scripts written in Java. The PIE language we used in our example is an alternative front-end to Pluto [91]. The build algorithm of Pluto constructs a dependency graph of a build while the build script runs. For example, consider the dependency graph of our example script in fig. 8.1. The dependency graph contains a node for each invoked task and for each read or written file. Edges between nodes encode

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 111 func main() -> string{ val config= parseYaml(./config.yaml) val src= config.srcDir if(config.checkStyle){ val styleOk= checkStyle(src) if(config.failOnStyle&&!styleOk) return "style error" } val userTests= ./test/** val genTests= genTests(src, ./test-gen) var failed=0 for(test <- userTests ++ genTests){ val testOk= runTest(test) if(!testOk) failed +=1 } return "Failed tests: "+ failed } func parseYaml(p: path) -> Config {...} func checkStyle(src: path) -> bool {...} func genTests(src: path, trg: path) -> path* {...} func runTest(test: path) -> bool {...}

Listing 8.1: Build script that invokes tasks conditionally and iteratively at build time. dependencies. A task depends on the tasks it invokes and on the files it reads or writes. Moreover, when a task reads a file that was generated by another task, the reading task depends on the generating task such that the generating task is executed first. While not shown in our graph, both task-task edges and task-file edges are labeled with stamps (e.g., timestamp, hashsum) that determine if the task output respectively file content is up-to-date. The incremental build algorithm of Pluto takes the dependency graph of the previous run and selectively reruns tasks to ensure consistency of the build. The dependency graph of a build is consistent if for each invoked task (i) all read and written files are up-to-date and (ii) the outputs of all called tasks are up-to-date. An incremental build algorithm is correct if it always restores consistency [36]. Or intuitively: a correct incremental build algorithm yields the same result as a clean build. The challenge is to restore consistency with as little computational effort as possible. For example, let us assume the initial dependency graph (top-left in fig. 8.1) is consistent to begin with. We discuss three different changes C1–C3: C1. If we change file ./config.yaml to turn off style checking, task main becomes inconsistent since the stamp of its file dependency changes (e.g., newer timestamp, changed hashsum). We can restore consistency by rerunning tasks parseYaml and main only; all other tasks remain consistent since they neither invoke main nor read files written by main. The incremental build yields a new dependency graph (top-right in fig. 8.1), where the new invocation of main does not depend on checkStyle anymore. C2. If instead, we change the content of user test ./test/X, task runTest(./test/X) becomes inconsistent and needs rerunning. Since task main

112 Initial dependency graph parse ./conﬁg parse ./conﬁg Change C1 Yaml(..) .yaml Yaml(..) .yaml

check ./src/ check ./src/ main ./src/A ./src/B main ./src/A ./src/B Style(..) B_aux Style(..) B_aux

gen gen Tests(..) Tests(..)

./test- ./test- ./test- ./test- ./test/ ./test/ gen/ gen/ gen/ gen/ X X A B A B runTest runTest

reads file writes file requires task affected by change

Change C2 Change C3 parse ./conﬁg parse ./conﬁg Yaml(..) .yaml Yaml(..) .yaml

check ./src/ check ./src/ main ./src/A ./src/B main ./src/A ./src/B ./src/C Style(..) B_aux Style(..) B_aux

gen gen Tests(..) Tests(..)

./test- ./test- ./test- ./test- ./test- ./test/ ./test/ gen/ gen/ gen/ gen/ gen/ X X A B A B C runTest runTest runTest

Figure 8.1: The dependency graph of a build captures task and ﬁle dependencies and is the basis for incremental building.

calls runTest(./test/X), it needs rerunning if the boolean value returned by runTest flips. Either way, the dependency graph remains unaffected (bottom- left in fig. 8.1). C3. Finally, if we add a source file ./src/C, tasks checkStyle and genTests are affected because they depend on the directory ./src. If the new file has a style error, this affects main and yields a new dependency graph where no testing occurs (not shown). Otherwise, let us assume genTests produces a new test ./test-gen/C for the added file, which affects main’s scan of directory test-gen. During the subsequent rerun of main, we discover a new task invocation runTest(./test-gen/C) (bottom-right in fig. 8.1). The incremental build algorithm of Pluto can handle these and any other changes correctly. Moreover, the algorithm is optimally incremental in the sense that it only executes a task if absolutely necessary. The basic idea of the algorithm is to start at the root node(s) of the dependency graph, to traverse it depth-first, and to interleave consistency checking and rerunning. In particular, while rerunning a task that invokes another task, if the invoked task exists in the dependency graph, continue with consistency checking of the invoked task and only rerun it if necessary. This interleaving of consistency

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 113 checking and rerunning is what enables support for conditional and iterative task invocations. Problem Statement. The incremental build algorithm of Pluto has an important limitation. Irrespective of the changed files, it has to traverse the entire dependency graph to check consistency and to discover tasks that need rerunning. While that may be fine for larger changes that affect large parts of the graph like C3, the overhead for small-impact changes like C2 is significant. Especially in interactive settings, where developers routinely trigger sequences of small- impact changes, this overhead can quickly render the system unresponsive. The problem is that the algorithm does not scale down to small changes when scaling up to large dependency graphs. Our goal is to design, realize, and evaluate a new incremental build algorithm for programmatic build scripts that scales in the size of a change. That is, rebuild times should be proportional to the impact a change has on the overall build. In particular, rebuild times should be independent of the size of the dependency graph. These requirements preclude a full traversal of the dependency graph to discover affected tasks as done by Pluto. Instead, our new algorithm takes the set of changed files as input and only ever visits affected tasks.

8.3 Key Idea and Challenges The key idea to increasing the scalability of the Pluto algorithm is to execute tasks bottom-up. In this section, we motivate this approach, and we discuss the corner cases that require adjustments to a pure bottom-up algorithm. 8.3.1 Bottom-Up Traversal The key problem of the Pluto build algorithm is that it visits and checks tasks that are ultimately unaffected. For example, in change C2 in section 8.2, only a single task (runTest) is affected by the change to file ./test/X. However, Pluto will visit and check all reachable tasks in a top-down depth-first traversal, including the tasks that are not affected by the change (parseYaml, checkStyle, and genTests). Establishing that these tasks are unaffected is expensive, as our benchmarks demonstrate (section 8.6). To make the algorithm scale, it should only visit the nodes of the dependency graph that are actually affected by a change. The changes that trigger a re-build are to files, which are at the leaves of the dependency graph. Tasks that need to be recomputed depend directly or indirectly on such file changes. Instead of looking for tasks that may indirectly depend on a change and gradually getting closer to the actual change, as Pluto does, why not start with those changes and the tasks that depend on them? The key idea of our algorithm is to traverse the dependency graph bottom- up, driven by file changes, only visiting and checking affected tasks. The algorithm first executes the tasks that are directly affected by changed files. For example, in change C2, file ./test/X changes, which directly affects task runTest(./test/X), which must therefore be re-executed. Tasks can also be

114 indirectly affected by a file change, namely when it reads a file produced by an affected task or when it reads the output value of an affected task. For example, in change C3, file ./src/C is added, which triggers re-execution of genTests, which yields a new output value to main, which thus is indirectly affected, re-executes, and creates a new runTest task. A subsequent edit of file ./src/C triggers genTests again, which produces the same value as before but updates the generated file ./test-gen/C, which affects the corresponding runTest task (the main task is not affected this time). Thus, a bottom-up traversal executes tasks that are affected by changed files or by other affected tasks, following a path from the changed leaves of the dependency graph to the root(s). However, a pure bottom-up traversal is not adequate to support programmatic build scripts with dynamic dependencies. We discuss the adjustments that are necessary to realize an adequate algorithm. 8.3.2 Top-Down Initialization In order to perform a bottom-up traversal over the dependency graph, we need a dependency graph to start with. Therefore, we start with Pluto’s top-down algorithm to obtain the initial dependency graph. This is efficient, since every task is affected in the initial build. 8.3.3 Early Cut-Off By default, a bottom-up traversal takes the transitive closure of dependencies, re-executing all tasks on the path from a changed file to the root(s) of the dependency graph. However, re-execution of a task does not always lead to a new result. If the result was the same as before, the path to the root can be cut off early. For example, in C2 main depends on runTest(./test/X), which depends on the changed ./test/X file. So, do we need to re-execute main? That depends on the output of task runTest(./test/X). If the result is a different (integer) value than before, the number of tests that fail changes, and main should be re-executed Otherwise, main is not affected and we can cut off the build early1, as shown in the bottom-left part of fig. 8.1. 8.3.4 Order of Recomputation Another potential problem of naive bottom-up evaluation is that tasks may be executed multiple times. For example, in change C3, main depends on two existing affected tasks: checkStyle and genTests. A possible execution trace when checkStyle does affect main (not shown in the figure), is to execute checkStyle, then main which is affected by checkStyle, then execute genTests, and then execute main again because it is affected by genTests. Executing a task multiple times is not only inefficient, but also causes glitches: inconsistent results that are exposed to users. To avoid such re-executions, we should ensure that all affected dependencies of a task are executed before the task itself. Instead of eagerly executing tasks

1In a real-world build script, runTest would output a report of which tests fail and why, and main would be re-executed whenever this changes. We support this, but chose to keep the example from section 8.2 simple for demonstration purposes.

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 115 when encountered during a bottom-up traversal, we schedule tasks in a priority queue, which is topologically sorted according to the dependency graph. Until the queue is empty, scheduled tasks are removed from the front of the queue and executed. The topological ordering of the priority queue ensures that task dependencies are executed before the task itself.

8.3.5 Dynamic Dependencies

The ﬁnal challenge is to support dynamic dependencies during a bottom-up traversal. Consider change C3 again, where a new task runTests(./test-gen/C) is discovered by main. A bottom-up traversal can never detect such a dynamic dependency, since it only has access to the dependency graph of the previous run. To remedy this, we temporarily switch to top-down depth-ﬁrst building when executing a task, so that the task can discover dependencies to new tasks, discover dependencies to existing tasks, or remove existing dependencies.

When discovering a dependency to a new task, top-down depth-ﬁrst building continues recursively by (eagerly) executing the new task. For example, in change C3, task main is (indirectly) affected and thus is built in a top-down manner (after its dependencies checkStyle and genTests have been built), which recursively calls task runTests(./test-gen/C) and registers a dependency to it. Furthermore, when a dependency is discovered to a task t that exists in the old dependency graph, it might be affected already. That is, t and dependencies of t may have been scheduled in the queue. We cannot execute t before executing its dependencies. Therefore, we temporarily switch back to bottom-up building, executing dependencies of t that are scheduled in the queue, until t itself is executed or found unaffected. Then we switch back to top-down building and continue executing the caller. Finally, when a dependency is removed (i.e., dependency to a task that was made in the previous run, but not in this run), the dependency graph is updated, but no further action is taken.

8.3.6 Dependency Graph Validation

As in the Pluto algorithm, we also need to enforce validity of the dependency graph by detecting overlapping generated files, hidden task dependencies, and cyclic tasks. An overlapping generated file occurs when more than one task generates (creates or writes to) the same file. This makes it unclear in which order those tasks must be executed to bring the file into a consistent state, and is therefore disallowed. Furthermore, a hidden dependency occurs when a task requires (reads) a file that was generated by another task, without the requiring task depending on the generator task. Such a dependency must be made explicit, so that the generated file is updated by the generator task before being read by the requiring task. Finally, a task is cyclic when it (indirectly) calls itself. We disallow cyclic tasks to ensure termination of the build algorithm. We check these invariants on-the-fly while constructing the new dependency graph for subsequent incremental builds.

116 var Tq function execAndSchedule(t, DGold) var Te val r := exec(t) var Oc schedAffByFiles(r.genFiles, DGold) var DGnew schedAffCallersOf(t, r.output, DGold) return r.output function buildNewTask(t, DGold) Te := ∅ function schedAffByFiles(F, DGold) Oc := ∅ for f ← F do DGnew := DGold for (stamp, t) ← DGold.requireesOf( f) do exec(t) if ¬stamp.isConsistent( f) then Tq := Tq ∪ t function buildWithChangedFiles(F, DGold) if (stamp, t) ← DG .generatorOf( f) then Te := ∅ old if ¬stamp.isConsistent( f) then Oc := ∅ Tq := Tq ∪ t DGnew := DGold Tq := PriorityQueue(DGold.depOrder()) function schedAffCallersOf(t, o, DGold) schedAffByFiles(F, DGold) for (stamp, tcall ) ← DGold.callersOf(t) do while Tq 6= ∅ do if ¬stamp.isConsistent(o) then execAndSchedule(Tq.poll(), DGold) Tq := Tq ∪ tcall

(a) Variables and main build functions. (b) Change-driven, bottom-up building.

function exec(t) if t ∈ Te then abort Te := Te ∪ t; val r := t.run(); Te := Te \ t DGnew := DGnew ∪ r; validate(t, r); observe(t, r.output) Oc[t] := r.output; return r.output

function require(t, DGold) if o ← Oc[t] then return o else if t ∈ DGold then return requireNow(t, DGold) else return exec(t)

function requireNow(t, DGold) while val tmin := Tq.leastDepFromOrEq(t) do Tq := Tq \ tmin val o := execAndSchedule(tmin, DGold) if t = tmin then return o val o := DGold.outputOf(t) observe(t, o); Oc[t] := o; return o function validate(t, r) for f ← r.genFiles do for (_, tgen) ← DGnew.generatorOf( f) do if t 6= tgen then abort for f ← r.reqFiles do for (_, tgen) ← DGnew.generatorOf( f) do if ¬DGnew.callsTaskTr(t, tgen) then abort

Listing 8.2: Change-driven incremental build algorithm.

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 117 8.4 Change-Driven Incremental Building

In this section, we present our hybrid algorithm that mixes bottom-up and top-down incremental building based on the observations and ideas from the previous section. We present the algorithm in three parts: main functions (listing 8.2a), bottom-up building (listing 8.2b), and execution (listing 8.2c). All functions share four global variables defined at the top of listing 8.2a. Variable Tq is a topologically ordered priority queue of affected tasks that still need to be executed. Variable Te is a set of currently executing tasks, used to detect cyclic tasks. Variable Oc is a cache of output values for tasks that have already been executed. Finally, variable DGnew is the new dependency graph that is constructed from the old dependency graph and dynamic dependencies on-the-fly. We provide two entry points to incremental building in listing 8.2a, both of which first clear the set of executing tasks Te, clear the cache Oc, and copy the old dependency graph DGold to DGnew. Function buildNewTask is the entry point for an initial build. Function buildNewTask then simply invokes function exec (listing 8.2c) to execute the task. We describe exec below. The second entry point buildWithChangedFiles is more interesting as it initiates bottom-up building. It takes as input a set of changed file paths F, represented as filesystem path strings such as ./config.yaml, and the old dependency graph DGold. The basic idea is to schedule and run affected tasks using priority queue Tq until all affected tasks are up-to-date. To this end, we create a new priority queue using the task dependencies in DGold as a topological ordering. We call function schedAffByFiles (described below) with the old dependency graph to find all tasks directly affected by the changed file paths F, and add those tasks to the queue Tq. The main loop of bottom-up building is the following while-loop: As long as there are affected tasks in the queue, poll a scheduled task (retrieve the task at the front and remove it) from the queue, execute it, and add all tasks affected by it to the queue. Since the queue is topologically ordered, dependencies of tasks are executed before the task itself. Unless a task itself does not terminate (for example by recursively calling new tasks ad infinitum), the queue becomes empty at some point since cyclic tasks are disallowed, terminating the algorithm.

8.4.1 Bottom-Up Building

Whenever a task occurs in the priority queue Tq, it is definitely affected (directly or indirectly) by changed files. Hence, no further consistency check is necessary. Function execAndSchedule in listing 8.2b accepts an affected task, runs it unconditionally using exec, and schedules new tasks based on the generated files and output value of the executed task. If a task does not change or create new generated files, nor produce a new output value, no new tasks will be scheduled and building may be cut off early. Function schedAffByFiles schedules tasks based on changed file paths F. If task t requires a changed file and the stamp stamp has changed (is inconsistent) then t is affected by the change to f and is scheduled by adding it to Tq.

118 Analogously, if a task generates a file that has changed, it is affected and thus scheduled. A stamp contains a summary of a file’s content, such as the last modification date or a hash, and is used to efficiently check whether a file has changed with isConsistent. For example, when using the file’s modification date as a stamp, we compare the modification date in the stamp, with the current modification date of the file on the local filesystem, and consider the file changed if the modification date is different. We use the old dependency graph DGold (computed in a previous run of the algorithm) to find tasks that require a file (requireesOf), and to find the task that generates a file (generatorOf), along with the stamp that was produced at the time the dependency was created. Likewise, the schedAffCallersOf function schedules callers of task t based on changes to its output value o. If tcall has a dependency to task t, and that dependency is inconsistent with relation to the new output value o of t, then tcall is affected by the new output value o and is scheduled. Similarly, we use a stamp of the output value, which could be the full output value, such as an integer representing the number of failing tests, or a summary of the value such as a hash, and compare the stamp with the new output value with isConsistent. Finally, the old dependency graph DGold is used to find callers of a task with callersOf.

8.4.2 Execution, Requirement, and Validation Function exec (listing 8.2c) executes the body of t. During task execution, a task may require (call) other tasks with the require function. Therefore, we first need to check if we are already executing task t, and abort when a cycle is detected. Then, we add t to the set of executing tasks Te, run the body of the task, and remove t from Te. Once execution completes, we update the new dependency graph DGnew with the result r of executing t. A result r contains the dynamic dependencies the task made during execution: a set reqFiles of read files, genFiles of created or written to files, and a set reqTasks of other tasks that were called by t; and the output value output that the task produced. A dependency graph DG is a set of those results, where each task has a single result. We then validate the new dependency graph, call any external observers of the task’s output with observe, cache the output, and finally return the output. We use function exec to execute tasks both during bottom-up and top-down traversals. While exec is agnostic to the traversal order, function require must take care to handle tasks required bottom-up and top-down correctly. We distinguish three cases. If t was already executed (visited) this run, we return its cached output value Oc[t]. Otherwise, we check if t was in the old dependency graph DGold. If task t is new and does not occur in DGold, then we execute it unconditionally. Note that no existing task in DGold can depend or be affected by the new task t. If task t existed before in DGold, we only execute it if it is actually affected. Since the caller of t awaits the output of t, we use function requireNow to force its checking and possible execution now. Task t is affected if it

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 119 occurs in queue Tq or if any of its dependencies occurring in Tq will affect it later. Function requireNow repeatedly finds dependency tmin of t that is lowest in the dependency graph (closes to the leaves). Since the queue only contains affected tasks, we execute tmin and schedule tasks affected by it. We continue until either we have executed the required task t, or until no more dependencies of task t are affected and we can reuse t’s output value from the old dependency graph with DGold.outputOf(t). Note that this latter case always triggers for tasks scheduled bottom-up by buildWithChangedFiles, because their dependencies cannot occur in Tq anymore. The validate function incrementally validates the correctness of the new dependency graph after executing a task t. For a dependency graph to be correct, it may not have overlapping generated files, nor any hidden dependencies. If another task tgen generates the same file f as t does, there is an overlapping generated file and execution is aborted. Furthermore, if t requires a file f without a (transitive) task dependency on tgen that generates f, there is a hidden dependency and execution is aborted. In both cases, this signals that there is an error in the build script. 8.4.3 Properties An incremental build algorithm is correct if it produces the exact same result as a clean build. Therefore, all affected and new tasks must be executed. Our algorithm is correct for tasks in the old dependency graph: if a task is affected, it will be scheduled. A task is affected directly by depending on a changed file, or indirectly (transitively) by depending on a changed file that an affected task generates, or by depending on the changed output of an affected task. All indirectly affected tasks are always found by traversing the dependency graph bottom-up, through polling the queue and scheduling affected tasks. Finally, all scheduled tasks are executed. Our algorithm is also correct for new tasks that are executed top-down like the Pluto algorithm, which is correct [36]. The only difference is the reqireNow function which first executes the dependencies of the task, but does eventually execute the task itself. Therefore, the hybrid algorithm is correct. For optimality, we only consider and execute affected tasks. For existing tasks, this is true because only affected tasks are scheduled. New tasks are affected and always executed. However, we only want to execute needed tasks. The hybrid algorithm considers all task in the old dependency graph as needed. This is an overapproximation, because it can happen that an affected task is not needed any more after top-down execution, since a task may remove its dependency to an affected task. Therefore, theoretically, the hybrid algorithm is only partially optimal. However, this is a rare case, as shown in the evaluation in section 8.6.

8.5 Implementation We have implemented the hybrid algorithm as an alternative execution algorithm for PIE [91], a system for developing interactive software development

120 pipelines, consisting of a DSL and API for implementing interactive pipelines, and a runtime for incrementally executing them. Interactive software development pipelines are similar to incremental build systems: they are used to incrementally build software artifacts, and also require fast feedback for usage in interactive environments with many low-impact changes such as IDEs and code editors. PIE builds forth on Pluto by reusing its model and algorithm, but provides a concise and expressive DSL for developing interactive pipelines and build scripts, minimizing boilerplate in contrast to Pluto’s Java API. Our algorithm is implemented as a separate executor in the PIE runtime, fully conforming to its API. That is, we can run existing PIE build scripts without changes to our algorithm. Furthermore, since PIE implements the Pluto build algorithm, we can compare our algorithm against Pluto’s, for the exact same build scripts. PIE, including our hybrid algorithm, is open source software that can be found online [116].

8.6 Evaluation In this section, we evaluate the performance of the hybrid algorithm, compared to Pluto’s pure top-down algorithm. We describe our experimental setup, show the results, interpret them, and discuss threats to validity. 8.6.1 Experimental Setup We compare the performance of the Pluto incremental build algorithm, as implemented in the PIE runtime, against our hybrid incremental build algorithm, which we have implemented in PIE runtime. Build Script. As a build script, we reuse the Spoofax-PIE pipeline, a reimplementation of a large part of the Spoofax pipeline, which was used as a case study of PIE [91] (section 7.6, listing 7.5). Spoofax [75] is a language workbench [38] (a set of tools for developing languages) in which languages are specified in terms of meta-languages, such as SDF [153] for syntax specification, and NaBL [5, 109, 89] for name and type analysis. The Spoofax pipeline derives artifacts from a language specification, such as a parse table for parsing, and a constraint generator and solver for solving name and type analysis. Furthermore, Spoofax supports interactive language development in an IDE setting, enabling a language developer to modify a language specification, resulting in immediate feedback in example programs of that language, and also supports developing multiple languages side-by-side. The Spoofax-PIE reimplementation supports these features. The build script is open-source and can be found online [132]. As input, the Spoofax build script takes a workspace directory consisting of language specifications, where each language specification has a configuration file describing how to build the language specification, a specification of the syntax, styling, and name and type analysis in meta-languages, and example programs. A configuration file at the root of the workspace lists the locations of all language specifications, and locations of the Spoofax meta-languages. As a concrete workspace, we use a directory with three Spoofax language

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 121 specifications for the Tiger, Calc, and MiniJava languages. Describing the Spoofax build script is outside of the scope of this paper. However, we do argue why Spoofax requires a programmatic build script with dynamic dependencies. The Spoofax build script frequently makes use of conditional building, where the result of executing a task influences a condition for another task. For example, when a program fails to parse, the program cannot be analyzed, since analysis requires an AST. Therefore, a condition that checks whether the parsing task succeeds guards the analysis task. Furthermore, Spoofax also makes frequent use of iterative building, where tasks are invoked multiples on different inputs which are outputs of previous tasks. For example, there is a single task description for parsing a file, which is dispatched based on the result of parsing the workspace configuration file, parsing the language specification configuration files, the concrete files that are in the workspace, and the extension of each file. Without a programmatic build script, all these forms of variability would have to be encoded in the configuration step of a declarative build script, which is not possible because many values only become evident during build script execution. Changes. To measure incremental performance, we have synthesized a chain of 60 realistic changes with varying impacts. First, a from-scratch build is performed that builds all language specifications. Then, we make changes in the form of opening or changing a text editor, requiring execution of a task that provides feedback for that editor, or of modifying and saving a file, which requires execution of tasks that keep the workspace up-to-date. Changes include: editing and saving example programs, styling specifications, syntax specifications, and name and type analysis specifications; adding a new language specification; undoing changes; and two extreme cases where we run the build with no or all files changed. These changes have varying impacts, where the impact is determined by how many tasks are affected by a change, and the run time of those tasks. For example, changing a syntax definition file requires recompilation of the parse table and reparsing of all example programs. Changing the name and types specification has a larger impact because it requires regeneration of a constraint generator, compilation of the constraint generator, application of the generator against all example programs, and finally application of the constraint solver to solve all generated constraints. A small impact change is editing an example program in an editor, which just requires parsing, styling, and analysis for that program. We run the exact same changes against the Pluto and our hybrid algorithm, with the only difference that we pass the changed files to our hybrid algorithm, whereas Pluto does not require this. We run the chain of changes against one algorithm in one go, to simulate a full editing session. Technicalities. We run the benchmark using the JMH [69] benchmarking framework, which runs the benchmark for an algorithm in a separate forked JVM, letting the JVM JIT fully specialize to that algorithm. Furthermore, it runs the benchmark multiple times before starting measurements, to ensure that the JVM is warmed up. Finally, it ensures that the garbage collector is executed

122 enwso h ecmrigrslsaditrrtte.I sntpossible not is It them. interpret and results benchmarking the show now We 1 -xsrpeettedfeetcags(ecie eo) h -xsrpeet the represents y-axis the below), (described changes different the represent x-axis ohoeignweiosadeio etcags o xml programs example for changes, text editor and editors new opening top-down both identically. changes. use perform we Editor algorithm hybrid B) graph, and dependency Pluto the initial both the Therefore, building. obtain To specifications. build. go Initial change. now We A) of scale. kind logarithmic each with for Pluto chart results the column the both over a for in change, Figure algorithm, aggregated instead. hybrid each those and for show measurements and time changes the of shows kinds different for taken all time for measurements time discuss to macOS i Core does run previous a in one. produced new garbage the influence that not so benchmark, a running before all = L changes, no = = K refactor, G changed. analysis specification files change, = analysis I small = refactor, all J specification specification = changes, syntax syntax specification = B = H build, change, F cascading initial changes, specification specification, specification = syntax styling language A = adding D time = algorithm. changes, the program hybrid E show example our we = change, and C changes, each algorithm editor For Pluto scale. the logarithmic for in taken The seconds in measurements. taken time time benchmark aggregate with chart Column 8.2: Figure ade ytehbi loih.I sipratt ucl rcs editor process quickly to important Chapter is It efficiently are algorithm. therefore and hybrid impact, the small hybrid by a handled the have changes combined, these changes because high editor all For is algorithm specifications. language and 8 .

. aggregate runtime in seconds 8 6 100s ehv xctdtebnhako aBo r iha with Pro MacBook a on benchmark the executed have We 10s . . 1s 0 2 b eut n Interpretation and Results 144 68.1 7 10 A processor, 8 68.7 with , . . 12 clbeIceetlBidn ihDnmcTs Dependencies Task Dynamic with Building Incremental Scalable 12.1 11 . 6 B h ecmr a xctdwt a with executed was benchmark The . 1.1 16 eod atr rvdn peu of speedup a providing faster, seconds is,w efr niiilbid uligallanguage all building build, initial an perform we First, Bo tc eoy and memory, stack of MB 9.1 16 C eageaeternigtm o l dtrchanges: editor all for time running the aggregate We 0.005 0.0 Bof GB 8.8 pluto D 0.7 0.7 1600 30.4 E 27.5 H DDR MHz 8.5 60 F 3.9 hne.Teeoew grgt the aggregate we Therefore changes. 8.4 G 4 21.5 3 Bo epmemory. heap of GB eoy n S,running SSD, a and memory, 17.6 H hybrid 3.0 28.6 1016 64 I 18.8 btJEo version of JRE -bit .Tesedpis speedup The %. 50.9 J 49.1 2 . 7 3.3 H Intel GHz K 0.00001 0.0 69.8 123 L 8 70.1 . 2 changes in IDEs, as programmers make many changes to editors and require fast feedback cycles. C) Example program file changes. We modify the files of several example program, and add a new example program file. For these changes combined, the hybrid algorithm takes 0.005 seconds, providing a 9 second (182133%) speedup. Again, these changes have a small impact, and are therefore efficiently handled by the hybrid algorithm, whereas the Pluto algorithm still requires checking of the entire dependency graph. D) Styling specification change. We modify the styling specification of the Calc language, and add a styling specification to the Tiger language. For these changes, the hybrid algorithm is 8 seconds faster, providing a 1245% speedup. The impact of these changes are slightly larger: changes to the styling specification require re-styling of open editors, but are still relatively smaller in impact and thus efficiently handled. E) Adding language specification. We add the MiniJava language to the workspace, requiring it to be built, and its example programs to be processed. Since this change causes many new tasks to be executed, its impact is large. The hybrid algorithm performs roughly the same as the Pluto algorithm, providing only a 2.9 second (11%) speedup because of reduced dependency graph checking. F) Syntax specification small change. We modify lexical syntax definition of the Calc language to parse numbers incorrectly, and undo the change afterwards. This requires the parse table to be rebuilt, and requires processing of Calc’s example programs. The hybrid algorithm provides a 4.5 second (118%) speedup, because a smaller part of the dependency graph is checked. G) Syntax specification cascading change. We modify the Calc syntax definition in such a way that the resulting parser will fail to parse all example programs, and also in such a way that new AST signatures need to be generated. From the syntax specification, Spoofax generates AST signature files that the name and type analysis uses. These signature files have changed, therefore requiring the name and type analysis specification to be recompiled. Finally, all example programs must be reparsed and reanalyzed. However, because example programs cannot be parsed any more, they also cannot be analyzed any more, since name and type analysis requires an AST. Therefore, the dependency from the process example file task, to the task that analyzes the AST of an example program, disappears. The Pluto algorithm first visits the process example file task, which removes its dependency to the analysis task, and therefore never recompiles the name and type analysis specification. However, the hybrid algorithm goes bottom-up to first recompile the name and type analysis specification, and only then executes the process example file task, therefore executing a task that was not required to be executed. In this case, the hybrid algorithm was 13 seconds slower, causing a 61% slowdown. This is a tradeoff of the hybrid algorithm: if a dependency to a task disap-

124 pears, the hybrid algorithm will still visit it. However, these cases are very rare, only a single change triggers this kind of behavior. For example, if at least one example program could be parsed into an AST (possibly through error recovery), the analysis specification has to be recompiled. We undo the change afterwards to make example programs parse again. H) Syntax specification refactor. We refactor a part of the MiniJava syntax definition into another file, which results in a semantically equivalent parser. The hybrid algorithm provides a 14.5 second (483%) speedup, because it first rebuilds the parse table, detects that it did not change, and then cuts off the build early. Contrary to the previous change, a bottom-up traversal here helps in cutting down the incremental build time, by not even traversing the unaffected part of the dependency graph. I) Analysis specification change. We modify the name and type analysis specification of the Calc language, such that it scopes bindings differently, and undo the change afterwards. Because changing these specifications has a moderate impact, the hybrid algorithm provides a moderate 9.7 second speedup (52%). J) Analysis specification refactor. We refactor a part of the Tiger name and type analysis specification into another file. Even though this results in a semantically equivalent analyzer, the change detection of the Spoofax-PIE build script is not smart enough to detect this. Because compiling the name and type analysis specification, and then performing constraint solving for all Tiger example programs, is expensive, this change has a large impact. Therefore, the Pluto and hybrid algorithm perform nearly identically. K) No changes. When there are no changes, the hybrid algorithm essentially performs no work, completing in sub-millisecond time, while the Pluto algorithm still needs to check the entire dependency graph, costing 3.3 seconds of time. This is the constant overhead that even small-impact changes suffer from with the Pluto algorithm, which the hybrid algorithm saves. L) All files changed. Finally, we change all source files by appending a space to the end of each file. Realistically, this kind of change can happen when checking out a different branch in a source control management system such as Git. When all source files change, using a bottom-up approach makes no sense, since (almost all) tasks will be affected, while incurring overhead because of scheduling. Therefore, we detect when more than 50% of source files (all required files, for which there is no generator task) change, and run a top-down build with the Pluto algorithm instead, therefore running as fast as the Pluto algorithm does. This heuristic seems to work well, but may require further tweaking. Conclusion. We can conclude that, for this build script and workspace directory, our algorithm scales better with the impact of a change than the Pluto algorithm, for many kinds of changes. The only exceptions being when all files are changed, for which a full rebuild could be triggered, or when a dependency to an expensive task is removed, which rarely happens.

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 125 8.6.3 Threats to Validity A possible threat to validity is that we have benchmarked the algorithms against a single build script. However, it is a complex build script that represents the realistic scenario of interactive language development in a language workbench. For example, the build script requires dynamic file dependencies in order to track precise dependencies which only become evident during a build. Furthermore, it also requires dynamic task dependencies, in order to dispatch the correct tasks based on the configuration of the workspace and each language specification. Another possible threat is that we have synthesized a chain of changes, instead of using existing change scenarios. However, we have constructed 60 changes to different kinds of aspects; such as changing an example program, and changing a file of the syntax specification; and with varying levels of impact, ranging from changing the text in a single editor, to changing a file of the name and type specification, which transitively affects many other tasks.

8.7 Related Work We now discuss related work, starting with the large body of work on incremental build systems with static dependencies. Static Dependencies. Make [133] is an incremental build system with declarative build rules based on files. It has limited support for dynamic file dependencies, and no proper support for dynamic task dependencies. Because of these limitations, Make scales well for simple build scripts, since it can first topologically sort dependencies, iterate over the dependencies, and incrementally execute affected tasks. While it is possible to emulate dynamic task dependencies, this requires tedious Makefile generation, encoding of all dependencies as files, and recursive Make execution. Therefore, Make is not sufficient for more complicated build scripts. Many build systems follow a similar approach to Make, first building a task DAG, and then executing it. For example, Gradle [67], Bazel [47], Buck [40], PROM [79], Fabricate [64], Tup [127], and Ninja [99] follow this approach, with slight variations. Gradle is a build automation tool, programmable in the Groovy language, that, like our hybrid algorithm, also supports values as inputs and outputs of tasks. PROM replaces declarative make rules with logical programming, while keeping the same incremental build algorithm. Fabricate uses system tracing to automatically infer file dependencies, but is only supported on Linux. Tup, like our hybrid build algorithm, requires a list of changed files as input, instead of scanning all files, to more efficiently build the task DAG. Ninja, unlike Make, detects changes to the commands of a rule, resulting in a rebuild if the rule is changed. None of the above systems supports dynamic file or task dependencies. Dynamic Dependencies. Some build systems intertwine incremental execution with the discovery of file and task dependencies. Pluto [36] is a Java library for developing incremental build scripts with dynamic dependencies. As

126 discussed throughout this paper, Pluto uses a top-down algorithm that does not scale to small changes over large dependency graphs. OMake [62] is a build system with Make-like syntax, but with a richer dependency tracking mechanism and a more complicated algorithm. It has limited support for dynamic file dependencies through scanner rules that scan depfiles and register their dependencies during execution. However, it does not support dynamic task dependencies; all tasks dependencies are specified statically in the build rules. Shake [107, 108] is a Haskell library for implementing build scripts with incremental execution. It has limited support for dynamic file dependencies, allowing needed files to be discovered dynamically, but generated file dependencies must be specified statically as the build target. It also has limited support for dynamic task dependencies: Tasks are named by keys, and those tasks can be required like files through their keys. However, these tasks are not parameterized, nor can they return values, making their use as dynamic task dependencies tedious. Incremental Computing. Our work is also related to approaches on incremental computing. Datalog [22] is a logic programming language with incremental solvers [50]. The are several differences between our hybrid algorithm and incremental Datalog solvers. Datalog solvers can deal with cycles, eagerly compute all facts, and use static dependencies from the Datalog program, whereas the hybrid algorithm (and build systems in general) disallow cycles, only compute demanded facts, and use dynamic dependencies. Adapton [52, 51] is a library for on-demand (lazy) incremental computation. Like the Pluto and our hybrid algorithm, Adapton supports a form of dynamic task dependencies: dynamic computation dependencies which form a computation (thunk) graph. Initially, Adapton builds a full computation graph. When the computation graph is affected by a change, edges of thunks are marked as dirty. Then, when a dirtied computation is demanded, it transitively "cleans" the edges of thunks during change propagation, re-executing out-of-date computations. Nominal Adapton’s [51] formal development relaxes the ordering of dirtying and cleaning, allowing dirtying while change propagation is running. However, the algorithm or implementation with this relaxed ordering is not described.

8.8 Conclusion We have shown the need for an efﬁcient, precise, and expressive build system. Many build systems are efﬁcient and precise, but not expressive, making complex build script development tedious. Pluto, a recent incremental build system that supports programmable build scripts with dynamic dependencies, is expressive, but does not scale with the impact of a change, because it requires a top-down traversal over the entire dependency graph for each change. To overcome this scalability problem, we have realized a hybrid algorithm that mixes bottom-up building for scalability, and top-down building for expressiveness through dynamic dependencies. We have evaluated the

Chapter 8. Scalable Incremental Building with Dynamic Task Dependencies 127 performance of our hybrid algorithm against Pluto’s algorithm, with a case study on the Spoofax language workbench. The evaluation demonstrates that the hybrid algorithm, with the exception of one kind of change, indeed scales better with the impact of a change, and is therefore faster than the Pluto algorithm, in particular for low-impact changes. Acknowledgments This research was supported by NWO/EW Free Competition Project 612.001.114 (Deep Integration of Domain-Speciﬁc Languages) and NWO VICI Project (639.023.206) (Language Designer’s Workbench).

128 Conclusion

We now conclude by summarizing interactive programming systems, our vision of language-parametric methods, and our five core contributions. We discuss our thesis, and how our core contributions relate to it. Finally, we end by discussing directions for future work. 9 9.1 Interactive Programming Systems A programming system for a programming language supports the development of programs through a batch compiler that validates programs and by transforming those programs into executable forms. An interactive programming system additionally supports the development of programs by providing automatic, continuous, responsive, and inline feedback to the programmer. However, a programming system is only truly interactive when it is correct and responsive. A method to achieve responsiveness is incrementality, where the response time is proportional to the impact of a change. Furthermore, responsiveness is only achieved when incrementality is scalable, where the incremental system can scale down to many low-impact changes, and scale up to large inputs. This is especially important in interactive programming systems, since the majority of changes are small (e.g., typing a character into a source code editor), and programs are large. Finally, it is important that correctness is still guaranteed in the presence of scalable incrementality. However, manually implementing an incremental system is a challenge as it requires the application of cross-cutting techniques such as dependency tracking, caching, cache invalidation, change detection, and persistence, which are complicated and error-prone to implement. Furthermore, scalable incrementality increases the challenge, as incrementality must scale up to large dependency graphs, cache large amounts of data, do cache invalidation through these large graphs, and detect low-impact changes. Finally, the scalable and incremental implementation must be correct, which is unlikely when manually implementing complicated and error-prone techniques. Therefore, our vision is to use language-parametric methods to develop responsive and correct interactive programming systems. Such a language-parametric method takes as input an implementation or description of a programming language, and automatically produces (parts of) an interactive programming system, without the language developer having to manually implement a correct, incremental, and scalable interactive programming system. Our thesis is that these language-parametric methods for developing interactive programming systems are feasible and useful. We have shown feasibility of language-parametric methods in the five core contributions of this dissertation by developing three language-parametric methods: declarative specification of

129 incremental name and type analysis, bootstrapping of language workbench meta-DSLs, and pipelining of interactive programming systems.

9.2 Language-Parametric Methods

We now summarize the language-language parametric methods developed in this dissertation and discuss their usefulness.

9.2.1 Incremental Name and Type Analysis

We have developed a language-parametric method for incremental name and type analysis consisting of the NaBL (chapter 2), and the incremental task engine (chapter 3). NaBL is a meta-DSL for declarative specification of name binding and scope rules of programming languages in terms of definitions and use sites, properties of names, namespaces for separating categories of names, scopes in which definitions are visible, and imports between scopes. From such a specification, the NaBL compiler derives a name analysis and editor services for inline error checking, reference resolution, and code completion, freeing the language developer from having to manually implement these parts. Therefore, NaBL is a language-parametric method for developing correct name analysis and corresponding interactive editor services. We extend NaBL with incrementality and type checking, using a language independent task engine for incremental name and type analysis. In this approach, we specify name and scope rules in NaBL, typing rules in TS – a meta-DSL for simple type system specification – from which we automatically derive a traversal that collects naming and typing tasks when given a program. These tasks are sent to the task engine, which then executes changed tasks to incrementally execute name and type analysis, updating data structures required for editor services, and responsively updating inline error messages. Therefore, NaBL, TS, and the task engine are a language-parametric method for developing correct and responsive name and type analysis with corresponding editor services. We have evaluated NaBL by specifying the name binding of a subset of C# (sections 2.2 and 2.3), and have evaluated the task engine approach by specifying the name and type rules of the WebDSL language and by confirming responsiveness through benchmarking (section 3.6). Furthermore, as discussed in chapter 4, the task engine approach is used to specify and run the incremental name and type analysis of the Green-Marl [63, 94] DSL, and the NaBL, TS, and SDF [153] meta-DSLs of the Spoofax [75] language workbench. Finally, NaBL, TS, and the task engine were used to teach students about incremental name and type analysis as part of the 2015-2016 edition of the Compiler Construction lab [145] where they develop a full version of the MiniJava [9] language.

130 9.2.2 Bootstrapping meta-DSLs of Language Workbenches A bootstrapped compiler can compile its own source code, because the compiler is written in the compiled language itself, providing several benefits such as a high-level implementation of the compiler, a large-scale test case for the compiler, and improvement dissemination. However, DSLs have limited expressiveness (by design) and are therefore ill-suited for bootstrapping. There- fore, language workbenches provide high level meta-languages for developing DSLs and their compilers, freeing language developers from having to bootstrap their DSLs. What we desire instead, is bootstrapping the meta-language compilers of language workbenches, to inherit the benefits of bootstrapping stated above. However, bootstrapping a language workbench is complicated by the fact that most provide multiple separate domain-specific meta-languages for describing different language aspects such as syntax, name and type analysis, code generation, and so forth. Thus, in order to build a meta-language compiler, we need to apply multiple meta-language compilers, entailing intricate dependencies that sound language workbench bootstrapping needs to handle. Furthermore, meta-languages often generate generators, which may in turn generate more generators, requiring an unknown number of build iterations to apply all generators and possibly find defects in them. Our solution to these problems is to do versioning and dependency tracking between meta-languages, and perform fixpoint bootstrapping, where we iteratively self-apply meta-language compilers to derive new versions until no change occurs, or until we find a defect (listing 5.1). Bootstrapping operations can be started, cancelled (when diverging), and rolled back (when defect) interactively, supporting the interactive programming system of the language workbench. In conclusion, our bootstrapping approach provides a (meta)language-parametric method for correctly and interactively bootstrapping the meta-languages of language workbenches, in an interactive programming environment. We have evaluated our bootstrapping approach by bootstrapping the eight meta-DSLs of the Spoofax language workbench (section 8.6). We make seven realistic changes to one or more meta-languages and perform fixpoint bootstrapping operations. We were able to create new baselines after successful bootstrapping attempts; make breaking changes by decomposing changes into multiple compatible ones; find defects in changes, roll back to the existing baseline, fix the defect, and reattempt bootstrapping; and perform these operations in the interactive programming system of Spoofax. To this day, we are still bootstrapping the meta-languages of Spoofax as part of its build. We are still versioning, creating explicit dependencies, and releasing new baselines of Spoofax’s meta-languages, even with the addition of two extra meta-DSLs: FlowSpec [129] and Statix [6]. 9.2.3 Pipelining of Interactive Programming Systems We have developed PIE, which provides a language-parametric method for developing interactive software development pipelines, a superset of correct and

Chapter 9. Conclusion 131 responsive interactive programming environments (chapter 7); and developed a change-driven algorithm for PIE which makes it scalable (chapter 8). An interactive software development pipeline automates parts of the software engineering process, such as building software via build scripts, but also reacts immediately to changes in input, and provides timely feedback to the user. An interactive programming system is an instance of such a pipeline, where changes to programs are immediately processed to provide timely feedback to programmers. However, interactivity complicates the development of pipelines, if responsiveness and correctness become the responsibility of the pipeline programmer, rather than being supported by the underlying system. PIE consists of a DSL, API, and runtime for developing correct and responsive interactive software development pipelines, where ease of development is a focus. The PIE DSL serves as a front-end for developing pipelines with minimal boilerplate in a functional language. The PIE API is a lower-level front-end for developing foreign pipeline functions which cannot be modeled in the DSL. Finally, the runtime incrementally executes pipelines implemented in the API using Pluto’s incremental build algorithm. However, the incremental build algorithm that we used did not scale, because it needs to traverse the entire dependency graph (produced in a previous build) from top to bottom, making the run-time of the algorithm dependent on the size of the dependency graph, not the impact of the change. This quickly becane a problem in interactive programming systems, where there are many changes and those changes have a low-impact (e.g., programmer typing characters into an editor), while the program and its induced dependency graph is large. To solve this scalability problem, we have developed a new incremental build algorithm that performs change-driven rebuilding (listings 8.2a to 8.2c). Our algorithm scales with the impact of a change, and is independent from the size of the dependency graph, because it only ever visits affected tasks. Therefore, PIE with our change-driven incremental build algorithm provides a language-parametric method for developing correct and responsive interactive programming systems. Furthermore, PIE can more generally be applied to interactive software development pipelines, such as build scripts, continuous integration pipelines, benchmarking pipelines. We evaluate PIE with a case study by reimplementing a signiﬁcant part of the interactive programming system of the Spoofax language workbench (listing 7.5). The existing pipeline of Spoofax’s interactive programming system was scattered across four different formalisms, decreasing ease of development; overapproximates dependencies, causing loss of incrementality; and underapproximates dependencies, causing loss of correctness. However, with PIE, we can easily integrate the different components of Spoofax; such as its parser, analyzers, transformations, build scripts, editor services, meta-languages, and dynamic language loading; into a single formalism. PIE ensures that the pipeline is correct and responsive, without the pipeline programmer having to implement techniques such as incrementality, or without having to reason about correctness. We also experimentally evaluate the performance of our change-driven

132 bottom-up algorithm with the Spoofax pipeline (fig. 8.2). To measure incremental performance an scalability, we synthesized a chain of 60 realistic changes of varying types and impacts, ranging from changing an example program, to changing the syntax specification of a language, to adding a new language specification. Results show that for low-impact changes (i.e., changes that only cause a small number of tasks to be actually affected), our change-driven algorithm is several orders of magnitude faster than the previous algorithm we used, while not slower for high-impact changes. Finally, to show that PIE can be used for other interactive software development pipelines, we have also performed a case study with the benchmarking suite from the accompanying artifact [136] of Criterion [137], which measures performance of immutable data structures on the JVM. Criterion uses a bash script to orchestrate benchmarking tasks, requiring to re-run all benchmarks after changes. We converted this script into a PIE pipeline (listing 7.6), which runs each benchmark and subject pair in isolation, enabling incrementality where PIE only executes a benchmark against all subjects if a benchmark changes, and only executes a subject against all benchmarks when a subject changes.

9.3 Future Work We now discuss future work on incremental name and type analysis, bootstrapping of meta-DSLs, and pipelining of interactive programming systems.

9.3.1 Incremental Name and Type Analysis Expressiveness. As discussed in chapter 4, future work for NaBL, TS and the task engine is to increase the expressiveness of the approach to support more kinds of names, scopes, and type systems. Much of this future work has already been done in the context of Scope Graphs [109], an extension of scope graphs into a full analysis framework based on constraint solving [5], and further improvements to expressiveness [6]. However, making this constraint- based analysis framework feasible for interactive programming systems is still a challenge, as it is not yet incremental nor scalable, and does not provide editor services such as good inline error messages, code completion, semantic code styling, and occurrence highlighting. Therefore, future work should focus on the combination of high expressiveness of name and type systems, while properly supporting correct and responsive interactive programming systems. Type-Directed Transformations. Furthermore, transformations are frequently name or type-directed instead of syntax-directed (e.g., compile a Java class into a class ﬁle, for every public Java class in the program), or need to query (properties of) names or types. NaBL supports this by building an index of names and providing an API to it, and the task engine supports this by building tasks which represent queries, for which the value is available through an API when performing a transformation. While it is possible to do name and type-directed transformations, these APIs feel ad-hoc, and do not support incremental transformations. One interesting direction of future work is to ﬁg-

Chapter 9. Conclusion 133 ure out a high-quality API for name and type-directed transformations, name and type queries, and how to (automatically) incrementalize transformations with such an API. Updating Analysis Data after Transformations. Finally, a frequently occurring pattern in compilation is to perform multiple small optimization transformations to the program, where each one transforms the program into a new (semantically equivalent) program. However, after such a transformation, the name and type information can be invalidated, because names or types may have been renamed, created, removed, or moved into a different scope. Since optimizations need access to name and type information, we must re-execute name and type analysis to make that information up-to-date again. Even when name and type analysis is incremental, it is costly to do so because of change detection and other overhead, which dominates when hundreds or even thousands of small optimization transformations occur. Therefore, another interesting direction of future work is to ﬁgure out how to incrementally update name and type information after (small) transformations.

9.3.2 Bootstrapping of Meta-DSLs

Our fixpoint bootstrapping approach either produces a new baseline or finds a defect after a number of fixpoint iterations, or diverges when the compilers of the meta-DSLs diverge. Therefore, our approach is a dynamic one: we can only figure this out after executing the compilers of the meta-DSLs. Future work could be to find out a way to do this statically, possibly by exploiting the fact that compilers typically converge. Finally, bootstrapping can also be seen as a sort of pipeline. It is currently not possible to perform fixpoint bootstrapping with PIE, as PIE does not have a fixpoint operations. If we want to integrate fixpoint bootstrapping in PIE, we need to add an incremental fixpoint operation to PIE.

9.3.3 Pipelining of Interactive Programming Systems

There is a lot of future work for PIE in the space of pipelining of interactive programming systems, and software development pipelines in general. Observability. PIE currently does not track if (the effect of) a task is observable to the outside world. For example, if we create a task that provides feedback for a code editor (e.g., code styling and inline error messages), but that code editor is currently not visible (e.g., it is hidden behind another window or was closed by the programmer), then we do not need to execute that task when the code changes, because the effect is not observable. We need to track which tasks are observable to the outside world, provide operations for marking tasks as (un)observed, and never execute tasks that are directly or transitively unobserved, to increase efficiency. The challenge is to find an efficient way to maintain this information, while keeping the incremental build algorithm correct. Furthermore, observability information could be used to perform garbage collection of tasks that are no longer required.

134 Concurrency. PIE currently does not support concurrency or parallelism. Inter- active programming systems are concurrent systems where multiple things can happen concurrently, such as running name and type analysis on one program, while parsing another program, while the user is requesting code completion. Similarly, processors have many cores which require parallelism to exploit. Concurrent and parallelism support is a challenge in the presence of dynamic dependencies, as multiple tasks could read/write to files concurrently without prior knowledge, causing conflicts. Furthermore, concurrently running tasks require a consistency model such as eventual consistency, as tasks may use the values produced by other tasks. Extending the PIE model and algorithm with concurrency and parallelism while overcoming these problems is a challenge. It is also a technical challenge, as efficient concurrent and parallel execution is hard to implement right, especially in the presence of the mutable filesystem, incrementality, and persistence.

Deferred Tasks. PIE currently does not support deferring a task while it is executing: a task either fully executes, or fails. Deferring execution is useful in the case where a task currently does not have enough information yet to execute, and that information cannot be retrieved by executing another task, because it is unknown which task provides this information. For example, in name analysis, use sites in a module frequently refer to deﬁnition sites in other modules through imports. However, when the other module has not been analyzed yet, and it is unknown where this module resides, it is not possible to complete name analysis for the current module, and the task must be deferred until the other module has been analyzed. The challenge here is to extend the PIE model and algorithm with support for deferring tasks in an efﬁcient way, possibly through coroutines or other asynchronous programming models.

Partial Evaluation. Partial evaluation could be used to automate deployment in PIE pipelines. Tasks in a pipeline depend on other data by depending on the files or output values from other tasks. Sometimes, this data is completely dynamic. For example, in a live language development pipeline for a DSL, the task that compiles programs of the DSL depends on another task that builds the compiler, which in turn depends on the compiler specification source files of the DSL. Whenever this compiler specification changes, a new compiler is built, and all example programs are recompiled with this new compiler. On the other hand, when we want to deploy the DSL to a customer’s computer, the compiler does not change any more and becomes completely static. Instead of deploying the compiler specification source files to the customer, we would rather only deploy the compiler, to avoid the customer having to build the compiler, having to store source files which never change, and possibly to prevent reverse engineering of the compiler via source code. Currently, to achieve this, we would need to manually adapt the pipeline to accept both compiler specification source files and a built compiler, which is tedious. With partial evaluation, we can automate this process by specifying which input data is static, execute the tasks of the pipeline, and replace tasks which (transitively) depend on completely static data with a task that just

Chapter 9. Conclusion 135 returns the static data. Applications. Finally, we want to keep evaluating PIE by application to more subdomains of interactive software development pipelines.

136 Bibliography

[1] Wil M. P. van der Aalst and Arthur H. M. ter Hofstede. “YAWL: yet another workflow language”. In: Inf. Syst. 30.4 (2005), pp. 245–275. doi: 10.1016/j.is.2004.02.002. [2] A. V. Aho, M. S. Lam, R. Sethi, and J. D. Ullman. Compilers: Principles, Techniques, and Tools (2nd Edition). Addison Wesley, Aug. 2006. [3] Johan Åkesson, Torbjörn Ekman, and Görel Hedin. “Implementation of a Modelica compiler using JastAdd attribute grammars”. In: Science of Computer Programming 75.1-2 (2010), pp. 21–38. doi: 10.1016/j.scico. 2009.07.003. [4] Peter Amstutz, Michael R. Crusoe, Nebojša Tijanić,Brad Chapman, John Chilton, Michael Heuer, Andrey Kartashov, Dan Leehr, Hervé Ménager, Maya Nedeljkovich, Matt Scales, Stian Soiland-Reyes, and Luka Stojanovic. “Common Workflow Language, v1.0”. In: (2016). doi: 10.6084/m9.figshare.3115156.v2. [5] Hendrik van Antwerpen, Pierre Néron, Andrew P. Tolmach, Eelco Vis- ser, and Guido Wachsmuth. “A constraint language for static semantic analysis based on scope graphs”. In: Proceedings of the 2016 ACM SIG- PLAN Workshop on Partial Evaluation and Program Manipulation, PEPM 2016, St. Petersburg, FL, USA, January 20 - 22, 2016. Ed. by Martin Erwig and Tiark Rompf. ACM, 2016, pp. 49–60. isbn: 978-1-4503-4097-7. doi: 10.1145/2847538.2847543. [6] Hendrik van Antwerpen, Casper Bach Poulsen, Arjen Rouvoet, and Eelco Visser. “Scopes as types”. In: Proceedings of the ACM on Program- ming Languages 2.OOPSLA (2018). doi: 10.1145/3276484. [7] Apache. Spark. https://spark.apache.org/. (Visited on 10/04/2019). [8] Andrew W. Appel. “Axiomatic Bootstrapping: A Guide for Compiler Hackers”. In: ACM Transactions on Programming Languages and Systems 16.6 (1994), pp. 1699–1718. doi: 10.1145/197320.197336. [9] Andrew W. Appel. Modern Compiler Implementation in Java, 2nd edition. Cambridge University Press, 2002. isbn: 0-521-82060-X. [10] John Warner Backus. “The syntax and semantics of the proposed international algebraic language of the Zurich ACM-GAMM Conference”. In: IFIP Congress. 1959, pp. 125–131. [11] Jon Bentley. “Programming pearls: little languages”. In: Commun. ACM 29 (1986). doi: 10.1145/6424.315691. [12] Lorenzo Bettini. Implementing Domain-Specific Languages with Xtext and Xtend. 2nd. Packt Publishing, 2016.

137 [13] Mark G. J. van den Brand. “PREGMATIC - a generator for incremental programming environments”. PhD thesis. University Nijmegen, 1992. [14] Mark G. J. van den Brand, Arie van Deursen, Jan Heering, H. A. de Jong, Merijn de Jonge, Tobias Kuipers, Paul Klint, Leon Moonen, Pieter A. Olivier, Jeroen Scheerder, Jurgen J. Vinju, Eelco Visser, and Joost Visser. “The ASF+SDF Meta-environment: A Component-Based Language De- velopment Environment”. In: Compiler Construction, 10th International Conference, CC 2001 Held as Part of the Joint European Conferences on The- ory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings. Ed. by Reinhard Wilhelm. Vol. 2027. Lecture Notes in Com- puter Science. Springer, 2001, pp. 365–370. isbn: 3-540-41861-X. doi: 10.1016/S1571-0661(04)80917-4. [15] Mark G. J. van den Brand, Jeroen Scheerder, Jurgen J. Vinju, and Eelco Visser. “Disambiguation Filters for Scannerless Generalized LR Parsers”. In: Compiler Construction, 11th International Conference, CC 2002, Held as Part of the Joint European Conferences on Theory and Practice of Soft- ware, ETAPS 2002, Grenoble, France, April 8-12, 2002, Proceedings. Ed. by R. Nigel Horspool. Vol. 2304. Lecture Notes in Computer Science. Springer, 2002, pp. 143–158. isbn: 3-540-43369-4. doi: 10.1007/3-540- 45937-5_12. [16] Harvey Bratman. “A alternate form of the "UNCOL diagram"”. In: Communications of the ACM 4.3 (1961), p. 142. doi: 10.1145/366199. 366249. [17] Martin Bravenboer, Arthur van Dam, Karina Olmos, and Eelco Visser. “Program Transformation with Scoped Dynamic Rewrite Rules”. In: Fundamenta Informaticae 69.1-2 (2006). https://content.iospress. com/articles/fundamenta-informaticae/fi69-1-2-06, pp. 123–178. [18] Martin Bravenboer, Karl Trygve Kalleberg, Rob Vermaas, and Eelco Visser. “Stratego/XT 0.17. A language and toolset for program transformation”. In: Science of Computer Programming 72.1-2 (2008), pp. 52–70. doi: 10.1016/j.scico.2007.11.003. [19] Doug Brown, John Levine, and Tony Mason. Lex & Yacc. 2nd. O’Reilly Series. O’Reilly Media, 1992. isbn: 9781565920002. [20] Christoff Bürger, Sven Karol, Christian Wende, and Uwe Aßmann. “Reference Attribute Grammars for Metamodel Semantics”. In: Software Language Engineering - Third International Conference, SLE 2010, Eindhoven, The Netherlands, October 12-13, 2010, Revised Selected Papers. Ed. by Brian A. Malloy, Steffen Staab, and Mark van den Brand. Vol. 6563. Lecture Notes in Computer Science. Springer, 2010, pp. 22–41. isbn: 978-3-642- 19439-9. doi: 10.1007/978-3-642-19440-5_3.

138 [21] Cristiano Calcagno, Walid Taha, Liwen Huang, and Xavier Leroy. “Im- plementing Multi-stage Languages Using ASTs, Gensym, and Reflec- tion”. In: Generative Programming and Component Engineering, Second In- ternational Conference, GPCE 2003, Erfurt, Germany, September 22-25, 2003, Proceedings. Ed. by Frank Pfenning and Yannis Smaragdakis. Vol. 2830. Lecture Notes in Computer Science. Springer, 2003, pp. 57–76. isbn: 3-540-20102-5. doi: 10.1007/978-3-540-39815-8_4. [22] Stefano Ceri, Georg Gottlob, and Letizia Tanca. “What you Always Wanted to Know About Datalog (And Never Dared to Ask)”. In: IEEE Trans. Knowl. Data Eng. 1.1 (1989), pp. 146–166. doi: 10.1109/69.43410. [23] Donald D. Chamberlin and Raymond F. Boyce. “SEQUEL: A Structured English Query Language”. In: Proceedings of 1974 ACM-SIGMOD Work- shop on Data Description, Access and Control, Ann Arbor, Michigan, May 1-3, 1974, 2 Volumes. Ed. by Randall Rustin. ACM, 1974, pp. 249–264. doi: 10.1145/800296.811515. [24] Philippe Charles, Robert M. Fuhrer, and Stanley M. Sutton Jr. “IMP: a meta-tooling platform for creating language-specific ides in eclipse”. In: 22nd IEEE/ACM International Conference on Automated Software En- gineering (ASE 2007), November 5-9, 2007, Atlanta, Georgia, USA. Ed. by R. E. Kurt Stirewalt, Alexander Egyed, and Bernd Fischer. ACM, 2007, pp. 485–488. isbn: 978-1-59593-882-4. doi: 10.1145/1321631.1321715. [25] Alan J. Demers, Thomas W. Reps, and Tim Teitelbaum. “Incremental Evaluation for Attribute Grammars with Application to Syntax-Directed Editors”. In: POPL. 1981, pp. 105–116. doi: 10.1145/567532.567544. [26] Arie van Deursen and Paul Klint. “Little languages: little maintenance?” In: Journal of Software Maintenance 10.2 (1998). http://citeseerx.ist. psu.edu/viewdoc/summary?doi=10.1.1.50.4726, pp. 75–92. [27] Arie van Deursen, Paul Klint, and Joost Visser. “Domain-Specific Lan- guages: An Annotated Bibliography”. In: SIGPLAN Notices 35.6 (2000), pp. 26–36. doi: 10.1145/352029.352035. [28] Eelco Dolstra, Merijn de Jonge, and Eelco Visser. “Nix: A Safe and Policy-Free System for Software Deployment”. In: Proceedings of the 18th Conference on Systems Administration (LISA 2004), Atlanta, USA, No- vember 14-19, 2004. http://www.usenix.org/publications/library/ proceedings/lisa04/tech/dolstra.html. USENIX, 2004, pp. 79–92. [29] Eelco Dolstra and Andres Löh. “NixOS: a purely functional Linux distribution”. In: Proceeding of the 13th ACM SIGPLAN international conference on Functional programming, ICFP 2008, Victoria, BC, Canada, September 20-28, 2008. Ed. by James Hook and Peter Thiemann. ACM, 2008, pp. 367–378. isbn: 978-1-59593-919-7. doi: 10 . 1145 / 1411204 . 1411255.

BIBLIOGRAPHY 139 [30] Eelco Dolstra, Eelco Visser, and Merijn de Jonge. “Imposing a Memory Management Discipline on Software Deployment”. In: 26th International Conference on Software Engineering (ICSE 2004), 23-28 May 2004, Edin- burgh, United Kingdom. IEEE Computer Society, 2004, pp. 583–592. isbn: 0-7695-2163-0. doi: 10.1109/ICSE.2004.1317480. [31] Jay Earley and Howard E. Sturgis. “A formalism for translator interactions”. In: Communications of the ACM 13.10 (1970), pp. 607–617. doi: 10.1145/355598.362740. [32] Torbjörn Ekman and Görel Hedin. “The JastAdd extensible Java compiler”. In: Proceedings of the 22nd Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOP- SLA 2007, October 21-25, 2007, Montreal, Quebec, Canada. Ed. by Richard P. Gabriel, David F. Bacon, Cristina Videira Lopes, and Guy L. Steele Jr. ACM, 2007, pp. 1–18. isbn: 978-1-59593-786-5. doi: 10.1145/1297027. 1297029. [33] Torbjörn Ekman and Görel Hedin. “The JastAdd system - modular extensible compiler construction”. In: Science of Computer Programming 69.1-3 (2007), pp. 14–26. doi: 10.1016/j.scico.2007.02.003. [34] EPFL. The Scala Programming Language. https://www.scala-lang.org/. (Visited on 10/04/2019). [35] Sebastian Erdweg. “Extensible Languages for Flexible and Principled Domain Abstraction”. PhD thesis. Philipps-Universität Marburg, Mar. 2013. [36] Sebastian Erdweg, Moritz Lichter, and Manuel Weiel. “A sound and optimal incremental build system with dynamic dependencies”. In: Proceedings of the 2015 ACM SIGPLAN International Conference on Object- Oriented Programming, Systems, Languages, and Applications, OOPSLA 2015, part of SPLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015. Ed. by Jonathan Aldrich and Patrick Eugster. ACM, 2015, pp. 89–106. isbn: 978-1-4503-3689-5. doi: 10.1145/2814270.2814316. [37] Sebastian Erdweg, Tillmann Rendel, Christian Kästner, and Klaus Oster- mann. “SugarJ: library-based syntactic language extensibility”. In: Pro- ceedings of the 26th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2011, part of SPLASH 2011, Portland, OR, USA, October 22 - 27, 2011. Ed. by Cristina Videira Lopes and Kathleen Fisher. ACM, 2011, pp. 391–406. isbn: 978-1-4503-0940-0. doi: 10.1145/2048066.2048099. [38] Sebastian Erdweg, Tijs van der Storm, Markus Völter, Meinte Boersma, Remi Bosman, William R. Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex Loh, Gabriël Konat, Pedro J. Molina, Martin Palatnik, Risto Pohjonen, Eugen Schindler, Klemens Schindler, Riccardo Solmi, Vlad A. Vergu, Eelco Visser, Kevin van der Vlist, Guido Wachsmuth, and Jimi van der Woning. “The State of the Art in Language Workbenches - Conclusions from the Language Workbench Challenge”. In: Software

140 Language Engineering - 6th International Conference, SLE 2013, Indianapolis, IN, USA, October 26-28, 2013. Proceedings. Ed. by Martin Erwig, Richard F. Paige, and Eric Van Wyk. Vol. 8225. Lecture Notes in Computer Science. Springer, 2013, pp. 197–217. isbn: 978-3-319-02653-4. doi: 10.1007/978- 3-319-02654-1_11. [39] Sebastian Erdweg, Tijs van der Storm, Markus Völter, Laurence Tratt, Remi Bosman, William R. Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex Loh, Gabriël Konat, Pedro J. Molina, Martin Palatnik, Risto Pohjonen, Eugen Schindler, Klemens Schindler, Riccardo Solmi, Vlad A. Vergu, Eelco Visser, Kevin van der Vlist, Guido Wachsmuth, and Jimi van der Woning. “Evaluating and comparing language workbenches: Existing results and benchmarks for the future”. In: Computer Languages, Systems & Structures 44 (2015), pp. 24–47. doi: 10.1016/j. cl.2015.08.007. [40] Facebook. Buck: a fast build tool. https://buckbuild.com/. (Visited on 10/04/2019). [41] Robby Findler, John Clements, Cormac Flanagan, Matthew Flatt, Shri- ram Krishnamurthi, Paul Steckler, and Matthias Felleisen. “DrScheme: a programming environment for Scheme”. In: Journal of Functional Pro- gramming 12.2 (2002), pp. 159–182. [42] Robby Findler and PLT. DrRacket: Programming Environment. Tech. rep. PLT-TR-2010-2. http://racket-lang.org/tr2/. PLT Design Inc., 2010. [43] Bent Flyvbjerg. “Five Misunderstandings about Case-Study Research”. In: Qualitative Inquiry 12.2 (Apr. 2006). [44] Apache Software Foundation. Ant. https://ant.apache.org/. (Visited on 10/04/2019). [45] Apache Software Foundation. Maven. https://maven.apache.org/. (Visited on 10/04/2019). [46] Martin Fowler. Language Workbenches: The Killer-App for Domain Spe- ciﬁc Languages? http : / / www . martinfowler . com / articles / languageWorkbench.html. 2005. [47] Google. Bazel - a fast, scalable, multi-language and extensible build system. https://bazel.build/. (Visited on 10/04/2019). [48] Google. Guice. https : / / github . com / google / guice. (Visited on 10/04/2019). [49] Danny M. Groenewegen, Zef Hemel, Lennart C. L. Kats, and Eelco Visser. “WebDSL: a domain-speciﬁc language for dynamic web applications”. In: Companion to the 23rd Annual ACM SIGPLAN Confer- ence on Object-Oriented Programming, Systems, Languages, and Applica- tions, OOPSLA 2008, October 19-13, 2007, Nashville, TN, USA. Ed. by Gail E. Harris. ACM, 2008, pp. 779–780. isbn: 978-1-60558-220-7. doi: 10.1145/1449814.1449858.

BIBLIOGRAPHY 141 [50] Ashish Gupta and Inderpal Singh Mumick. “Maintenance of Materi- alized Views: Problems, Techniques, and Applications”. In: IEEE Data Eng. Bull. 18.2 (1995). db/journals/debu/GuptaM95.html, pp. 3–18. [51] Matthew A. Hammer, Joshua Dunﬁeld, Kyle Headley, Nicholas Labich, Jeffrey S. Foster, Michael W. Hicks, and David Van Horn. “Incremental computation with names”. In: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Lan- guages, and Applications, OOPSLA 2015, part of SPLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015. Ed. by Jonathan Aldrich and Patrick Eu- gster. ACM, 2015, pp. 748–766. isbn: 978-1-4503-3689-5. doi: 10.1145/ 2814270.2814305. [52] Matthew A. Hammer, Yit Phang Khoo, Michael Hicks, and Jeffrey S. Foster. “Adapton: composable, demand-driven incremental computation”. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014. Ed. by Michael F. P. O’Boyle and Keshav Pingali. ACM, 2014, p. 18. isbn: 978-1-4503-2784-8. doi: 10.1145/2594291.2594324. [53] Görel Hedin. “An Introductory Tutorial on JastAdd Attribute Gram- mars”. In: Generative and Transformational Techniques in Software Engineer- ing III - International Summer School, GTTSE 2009, Braga, Portugal, July 6-11, 2009. Revised Papers. Ed. by Joao M. Fernandes, Ralf Lämmel, Joost Visser, and João Saraiva. Vol. 6491. Lecture Notes in Computer Science. Springer, 2009, pp. 166–200. isbn: 978-3-642-18022-4. doi: 10.1007/978- 3-642-18023-1_4. [54] Görel Hedin. “Incremental Semantic Analysis”. PhD thesis. 1992. [55] Görel Hedin. “Incremental Static-Semantic Analysis for Object-Oriented Languages Using Door Attribute Grammars”. In: Attribute Grammars, Applications and Systems, International Summer School SAGA, Prague, Czechoslovakia, June 4-13, 1991, Proceedings. Ed. by Henk Alblas and Borivoj Melichar. Vol. 545. Lecture Notes in Computer Science. Springer, 1991, pp. 374–379. isbn: 3-540-54572-7. [56] Görel Hedin. “Reference Attributed Grammars”. In: Informatica (Slove- nia) 24.3 (2000). [57] Florian Heidenreich, Jendrik Johannes, Sven Karol, Mirko Seifert, and Christian Wende. “Derivation and Reﬁnement of Textual Syntax for Models”. In: Model Driven Architecture - Foundations and Applications, 5th European Conference, ECMDA-FA 2009, Enschede, The Netherlands, June 23-26, 2009. Proceedings. Ed. by Richard F. Paige, Alan Hartman, and Arend Rensink. Vol. 5562. Lecture Notes in Computer Science. Springer, 2009, pp. 114–129. isbn: 978-3-642-02673-7. doi: 10.1007/978-3-642- 02674-4_9.

142 [58] Florian Heidenreich, Jendrik Johannes, Jan Reimann, Mirko Seifert, Christian Wende, Christian Werner, Claas Wilke, and Uwe Aßmann. “Model-driven Modernisation of Java Programs with JaMoPP”. In: Joint Proceedings of the First International Workshop on Model-Driven Software Migration (MDSM 2011) and the Fifth International Workshop on System Quality and Maintainability (SQM 2011), March 1, 2011 in Oldenburg, Germany. CEUR Workshop Proceedings, Mar. 2011, pp. 8–11. [59] Zef Hemel, Danny M. Groenewegen, Lennart C. L. Kats, and Eelco Visser. “Static consistency checking of web applications with WebDSL”. In: Journal of Symbolic Computation 46.2 (2011), pp. 150–182. doi: 10. 1016/j.jsc.2010.08.006. [60] Zef Hemel, Lennart C. L. Kats, Danny M. Groenewegen, and Eelco Visser. “Code generation by model transformation: a case study in transformation modularity”. In: Software and Systems Modeling 9.3 (2010), pp. 375–402. doi: 10.1007/s10270-009-0136-1. [61] Zef Hemel and Eelco Visser. “Declaratively programming the mobile web with Mobl”. In: Proceedings of the 26th Annual ACM SIGPLAN Confer- ence on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2011, part of SPLASH 2011, Portland, OR, USA, October 22 - 27, 2011. Ed. by Cristina Videira Lopes and Kathleen Fisher. ACM, 2011, pp. 695–712. isbn: 978-1-4503-0940-0. doi: 10.1145/2048066.2048121. [62] Jason Hickey and Aleksey Nogin. “OMake: Designing a Scalable Build Process”. In: Fundamental Approaches to Software Engineering, 9th Interna- tional Conference, FASE 2006, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2006, Vienna, Austria, March 27- 28, 2006, Proceedings. Ed. by Luciano Baresi and Reiko Heckel. Vol. 3922. Lecture Notes in Computer Science. Springer, 2006, pp. 63–78. isbn: 3-540-33093-3. doi: 10.1007/11693017_7. [63] Sungpack Hong, Hassan Chafi, Eric Sedlar, and Kunle Olukotun. “Green-Marl: a DSL for easy and efficient graph analysis”. In: Pro- ceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2012, London, UK, March 3-7, 2012. Ed. by Tim Harris and Michael L. Scott. ACM, 2012, pp. 349–362. isbn: 978-1-4503-0759-8. doi: 10.1145/2150976.2151013. [64] B. Hoyts and Simon Alford. fabricate. https : / / github . com / SimonAlfie/fabricate. 2009. (Visited on 10/04/2019). [65] Human Usable Textual Notation Specification. Object Management Group. 2004. [66] Harry D. Huskey, M. H. Halstead, and R. McArthur. “NELIAC - Dialect of ALGOL”. In: Communications of the ACM 3.8 (Aug. 1960), pp. 463–468. doi: 10.1145/367368.367373. [67] Gradle Inc. Gradle Build Tool. https : / / gradle . org/. (Visited on 10/04/2019).

BIBLIOGRAPHY 143 [68] interactive. In: Cambridge Dictionary. 2018. url: https://dictionary. cambridge . org / dictionary / english / interactive (visited on 10/04/2019). [69] Java Microbenchmarking Harness (JMH). http://openjdk.java.net/ projects/code-tools/jmh/. (Visited on 10/04/2019). [70] Jenkins Pipeline syntax. https://jenkins.io/doc/book/pipeline/ syntax/. (Visited on 10/04/2019). [71] JetBrains. Kotlin Programming Language. https://kotlinlang.org/. (Visited on 10/04/2019). [72] Gregory F. Johnson and Charles N. Fischer. “A Meta-Language and System for Nonlocal Incremental Attribute Evaluation in Language- Based Editors”. In: POPL. 1985, pp. 141–151. [73] Uwe Kastens. “Ordered Attributed Grammars”. In: Acta Informatica 13 (1980), pp. 229–256. [74] Uwe Kastens and William M. Waite. “Modularity and Reusability in Attribute Grammars”. In: Acta Informatica 31.7 (1994). http://portal. acm.org/citation.cfm?id=191491, pp. 601–627. [75] Lennart C. L. Kats and Eelco Visser. “The Spoofax language workbench: rules for declarative specification of languages and IDEs”. In: Proceedings of the 25th Annual ACM SIGPLAN Conference on Object- Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010. Ed. by William R. Cook, Siobhán Clarke, and Martin C. Rinard. Reno/Tahoe, Nevada: ACM, 2010, pp. 444–463. isbn: 978-1-4503-0203-6. doi: 10.1145/1869459.1869497. [76] Lennart C. L. Kats, Eelco Visser, and Guido Wachsmuth. “Pure and declarative syntax definition: paradise lost and regained”. In: Proceed- ings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010. Ed. by William R. Cook, Siobhán Clarke, and Martin C. Rinard. Reno/Tahoe, Nevada: ACM, 2010, pp. 918–932. isbn: 978-1-4503-0203-6. doi: 10 . 1145/1869459.1869535. [77] Sven Keidel, Wulf Pfeiffer, and Sebastian Erdweg. “The IDE portability problem and its solution in Monto”. In: Proceedings of the 2016 ACM SIGPLAN International Conference on Software Language Engineering, Ams- terdam, The Netherlands, October 31 - November 1, 2016. Ed. by Tijs van der Storm, Emilie Balland, and Dániel Varró. http://dl.acm.org/ citation.cfm?id=2997368. ACM, 2016, pp. 152–162. isbn: 978-1-4503- 4447-0. [78] Steven Kelly, Kalle Lyytinen, and Matti Rossi. “MetaEdit+ A Fully Con- figurable Multi-User and Multi-Tool CASE and CAME Environment”. In: Seminal Contributions to Information Systems Engineering, 25 Years of CAiSE. Ed. by Janis A. Bubenko Jr., John Krogstie, Oscar Pastor, Barbara Pernici, Colette Rolland, and Arne Sølvberg. Springer, 2013, pp. 109– 129. isbn: 978-3-642-36926-1. doi: 10.1007/978-3-642-36926-1_9.

144 [79] Thilo Kielmann. PROM: A flexible, PROLOG-based make tool. Technical Report Report TI-4/91. Darmstadt, Germany: Institute of Theoretical Computer Science, Darmstadt University of Technology, 1991. [80] Paul Klint. “A Meta-Environment for Generating Programming Envi- ronments”. In: ACM Transactions on Software Engineering Methodology 2.2 (1993), pp. 176–201. doi: 10.1145/151257.151260. [81] Paul Klint, Tijs van der Storm, and Jurgen J. Vinju. “RASCAL: A Domain Specific Language for Source Code Analysis and Manipulation”. In: Ninth IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2009, Edmonton, Alberta, Canada, September 20-21, 2009. IEEE Computer Society, 2009, pp. 168–177. isbn: 978-0-7695-3793-1. doi: 10.1109/SCAM.2009.28. [82] Donald E. Knuth. “backus normal form vs. Backus Naur form”. In: Communications of the ACM 7.12 (1964), pp. 735–736. doi: 10.1145/ 355588.365140. [83] Donald E. Knuth. “Semantics of Context-Free Languages”. In: Theory Comput. Syst. 2.2 (1968). http://www.springerlink.com/content/ m2501m07m4666813/, pp. 127–145. [84] Gabriel Konat. PIE implementation for 2018. Mar. 2018. doi: 10.5281/zenodo.1199192. [85] Gabriel Konat. Spoofax-PIE implementation for 2018. Mar. 2018. doi: 10.5281/zenodo.1199194. [86] Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Bootstrapping Domain-Specific Meta-Languages in Language Workbenches”. In: Pro- ceedings of the 2016 ACM SIGPLAN International Conference on Gener- ative Programming: Concepts and Experiences, GPCE 2016, Amsterdam, The Netherlands, October 31 - November 1, 2016. Ed. by Bernd Fischer and Ina Schaefer. ACM, 2016, pp. 47–58. isbn: 978-1-4503-4446-3. doi: 10.1145/2993236.2993242. [87] Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Scalable incremental building with dynamic task dependencies”. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineer- ing, ASE 2018, Montpellier, France, September 3-7, 2018. Ed. by Marianne Huchard, Christian Kästner, and Gordon Fraser. ACM, 2018, pp. 76–86. doi: 10.1145/3238147.3238196. [88] Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Towards Live Language Development”. In: Workshop on Live Programming Systems (LIVE). 2016. [89] Gabriël Konat, Lennart C. L. Kats, Guido Wachsmuth, and Eelco Visser. “Declarative Name Binding and Scope Rules”. In: Software Language Engineering, 5th International Conference, SLE 2012, Dresden, Germany, September 26-28, 2012, Revised Selected Papers. Ed. by Krzysztof Czar- necki and Görel Hedin. Vol. 7745. Lecture Notes in Computer Science.

BIBLIOGRAPHY 145 Springer, 2012, pp. 311–331. isbn: 978-3-642-36089-3. doi: 10.1007/978- 3-642-36089-3_18. [90] Gabriël Konat, Luís Eduardo de Souza Amorim, Sebastian Erdweg, and Eelco Visser. Bootstrapping, Default Formatting, and Skeleton Editing in the Spoofax Language Workbench. Language Workbench Challenge (LWC@SLE). https://2016.splashcon.org/details/lwc2016/4/ Bootstrapping- Default- Formatting- and- Skeleton- Editing- in- the-Spoofax-Language-Workb. 2016. [91] Gabriël Konat, Michael J. Steindorfer, Sebastian Erdweg, and Eelco Visser. “PIE: A Domain-Specific Language for Interactive Software Development Pipelines”. In: Programming Journal 2.3 (2018), p. 9. doi: 10.22152/programming-journal.org/2018/2/9. [92] Shriram Krishnamurthi. Programming Languages: Application and Inter- pretation. http : / / www . cs . brown . edu / ~sk / Publications / Books / ProgLangs/2007-04-26/. 2007. [93] Epperly T Kumfert G. Software in the DOE: The Hidden Overhead of "The Build". Tech. rep. Lawrence Livermore National Laboratory, 2002. [94] Oracle Labs. Parallel Graph AnalytiX. https : / / www . oracle . com / technetwork/oracle- labs/parallel- graph- analytix/overview/ index.html. (Visited on 10/04/2019). [95] Olivier Lecarme, Mireille Pellissier, and Marie-Claude Thomas. “Computer-aided Production of Language Implementation Systems: A Review and Classification”. In: Software: Practice and Experience 12.9 (1982), pp. 785–824. [96] John Levine. Flex & Bison. O’Reilly Series. O’Reilly Media, 2009. isbn: 9780596155971. [97] Alex Loh, Tijs van der Storm, and William R. Cook. “Managed data: modular strategies for data abstraction”. In: ACM Symposium on New Ideas in Programming and Reflections on Software, Onward! 2012, part of SPLASH ’12, Tucson, AZ, USA, October 21-26, 2012. Ed. by Gary T. Leavens and Jonathan Edwards. ACM, 2012, pp. 179–194. isbn: 978-1- 4503-1562-3. doi: 10.1145/2384592.2384609. [98] David Mackenzie, Tom Tromey, Alexandre Duret-Lutz, Ralf Wildenhues, and Stefano Lattarini. GNU Automake. Free Software Foundation, Feb. 2018. [99] Evan Martin. The Ninja build system. https : / / ninja - build . org / manual.html. (Visited on 10/04/2019). [100] Shane McIntosh, Bram Adams, and Ahmed E. Hassan. “The evolution of ANT build systems”. In: Proceedings of the 7th International Working Conference on Mining Software Repositories, MSR 2010 (Co-located with ICSE), Cape Town, South Africa, May 2-3, 2010, Proceedings. Ed. by Jim Whitehead and Thomas Zimmermann. IEEE, 2010, pp. 42–51. isbn: 978-1-4244-6803-4. doi: 10.1109/MSR.2010.5463341.

146 [101] Erik Meijer. “Reactive extensions (Rx): curing your asynchronous programming blues”. In: ACM SIGPLAN Commercial Users of Functional Programming. ACM. 2010, p. 11. doi: 10.1145/1900160.1900173. [102] Marjan Mernik, Jan Heering, and Anthony M. Sloane. “When and how to develop domain-speciﬁc languages”. In: ACM Computing Surveys 37.4 (2005), pp. 316–344. doi: 10.1145/1118890.1118892. [103] Emma Anna Van Der Meulen. “Incremental Rewriting”. PhD thesis. University of Amsterdam, 1994. [104] Jon Meyer and Troy Downing. Java Virtual Machine. O Reilly, 1997. isbn: 1-56592-194-1. [105] Heather Miller, Philipp Haller, and Martin Odersky. “Spores: A Type- Based Foundation for Closures in the Age of Concurrency and Distri- bution”. In: ECOOP 2014 - Object-Oriented Programming - 28th European Conference, Uppsala, Sweden, July 28 - August 1, 2014. Proceedings. Ed. by Richard Jones. Vol. 8586. Lecture Notes in Computer Science. Springer, 2014, pp. 308–333. isbn: 978-3-662-44201-2. doi: 10.1007/978-3-662- 44202-9_13. [106] Peter Miller. Recursive make considered harmful. http : / / aegis . sourceforge.net/auug97.pdf. (Visited on 10/04/2019). [107] Neil Mitchell. “Shake before building: replacing make with haskell”. In: ACM SIGPLAN International Conference on Functional Programming, ICFP’12, Copenhagen, Denmark, September 9-15, 2012. Ed. by Peter Thie- mann and Robby Bruce Findler. ACM, 2012, pp. 55–66. isbn: 978-1-4503- 1054-3. doi: 10.1145/2364527.2364538. [108] Andrey Mokhov, Neil Mitchell, Simon L. Peyton Jones, and Simon Marlow. “Non-recursive make considered harmful: build systems at scale”. In: Proceedings of the 9th International Symposium on Haskell, Haskell 2016, Nara, Japan, September 22-23, 2016. Ed. by Geoffrey Mainland. ACM, 2016, pp. 170–181. isbn: 978-1-4503-4434-0. doi: 10 . 1145 / 2976002 . 2976011. [109] Pierre Néron, Andrew P. Tolmach, Eelco Visser, and Guido Wachsmuth. “A Theory of Name Resolution”. In: Programming Languages and Systems - 24th European Symposium on Programming, ESOP 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, 2015. Proceedings. Ed. by Jan Vitek. Vol. 9032. Lecture Notes in Computer Science. Springer, 2015, pp. 205–231. isbn: 978-3-662-46668-1. doi: 10.1007/978-3-662-46669-8_9. [110] K. V. Nori, U. Ammann, K. Jensen, and H. H. Nägeli. The PASCAL

Compiler: Implementation Notes. Tech. rep. ETH Zürich, 1974. [111] Object Constraint Language. 2.3.1. Object Management Group. 2012. [112] Martin Odersky. “Deﬁning Context-Dependent Syntax Without Using Contexts”. In: ACM Transactions on Programming Languages and Systems 15.3 (1993), pp. 535–562. doi: 10.1145/169683.174159.

BIBLIOGRAPHY 147 [113] Hubert Österle, Jörg Becker, Ulrich Frank, Thomas Hess, Dimitris Kara- giannis, Helmut Krcmar, Peter Loos, Peter Mertens, Andreas Oberweis, and Elmar J. Sinz. “Memorandum on design-oriented information systems research”. In: EJIS 20.1 (2011), pp. 7–10. doi: 10.1057/ejis.2010. 55. [114] Daniël Pelsmaeker. “Portable Editor Services”. MA thesis. 2018. [115] Maarten C. Pennings. “Generating incremental attribute evaluators”. Ph.D. Thesis. Computer Science, Utrecht University, Nov. 1994. [116] PIE implementation. https://github.com/metaborg/pie. (Visited on 10/04/2019). [117] Benjamin C. Pierce. Types and Programming Languages. Cambridge, Mass- achusetts: MIT Press, 2002. [118] Arnd Poetzsch-Heffter. “Implementing High-Level Identification Speci- fications”. In: Compiler Construction, 4th International Conference on Com- piler Construction, CC 92, Paderborn, Germany, October 5-7, 1992, Proceed- ings. Ed. by Uwe Kastens and Peter Pfahler. Vol. 641. Lecture Notes in Computer Science. Springer, 1992, pp. 59–65. isbn: 3-540-55984-1. [119] Arnd Poetzsch-Heffter. “Logic-Based Specification of Visibility Rules”. In: PLILP. 1991, pp. 63–74. [120] Arnd Poetzsch-Heffter. “Programming Language Specification and Prototyping Using the MAX System”. In: Programming Language Im- plementation and Logic Programming, 5th International Symposium, PLILP 93, Tallinn, Estonia, August 25-27, 1993, Proceedings. Ed. by Maurice Bruynooghe and Jaan Penjam. Vol. 714. Lecture Notes in Computer Science. Springer, 1993, pp. 137–150. isbn: 3-540-57186-8. [121] Thomas W. Reps. Generating language-based environments. Cambridge, MA, USA: Massachusetts Institute of Technology, 1984. isbn: 0-262- 18115-0. [122] Thomas W. Reps. “Optimal-Time Incremental Semantic Analysis for Syntax-Directed Editors”. In: POPL. 1982, pp. 169–176. doi: 10.1145/ 582153.582172. [123] Thomas W. Reps and Tim Teitelbaum. “The Synthesizer Generator”. In: Proceedings of the first ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments. New York, USA: ACM, 1984, pp. 42–48. doi: 10.1145/800020.808247. [124] Thomas W. Reps, Tim Teitelbaum, and Alan J. Demers. “Incremental Context-Dependent Analysis for Language-Based Editors”. In: ACM Transactions on Programming Languages and Systems 5.3 (1983), pp. 449– 477. doi: 10.1145/2166.357218.

148 [125] Tiark Rompf and Martin Odersky. “Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs”. In: Generative Programming And Component Engineering, Proceedings of the Ninth International Conference on Generative Programming and Component Engineering, GPCE 2010, Eindhoven, The Netherlands, October 10-13, 2010. Ed. by Eelco Visser and Jaakko Järvi. ACM, 2010, pp. 127–136. isbn: 978-1-4503-0154-1. doi: 10.1145/1868294.1868314. [126] Guido Salvaneschi, Gerold Hintz, and Mira Mezini. “REScala: bridging between object-oriented and functional style in reactive applications”. In: 13th International Conference on Modularity, MODULARITY ’14, Lugano, Switzerland, April 22-26, 2014. Ed. by Walter Binder, Erik Ernst, Achille Peternier, and Robert Hirschfeld. ACM, 2014, pp. 25–36. isbn: 978-1- 4503-2772-5. doi: 10.1145/2577080.2577083. [127] Mike Shal. Build System Rules and Algorithms. http://gittup.org/ tup/build_system_rules_and_algorithms.pdf. 2009. (Visited on 10/04/2019). [128] Mary Shaw. “Writing Good Software Engineering Research Papers”. In: Proceedings of the 25th International Conference on Software Engineer- ing, May 3-10, 2003, Portland, Oregon, USA. http://computer.org/ proceedings/icse/1877/18770726abs.htm. IEEE Computer Society, 2003, pp. 726–737. [129] Jeff Smits and Eelco Visser. “FlowSpec: declarative dataflow analysis specification”. In: Proceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering, SLE 2017, Vancouver, BC, Canada, October 23-24, 2017. Ed. by Benoît Combemale, Marjan Mernik, and Bernhard Rumpe. ACM, 2017, pp. 221–231. isbn: 978-1-4503-5525-4. doi: 10.1145/3136014.3136029. [130] Emma Söderberg. “Contributions to the Construction of Extensible Semantic Editors”. PhD thesis. 2012. [131] Emma Söderberg and Görel Hedin. “A Comparative Study of Incre- mental Attribute Grammar Solutions to Name Resolution”. In: 2012. [132] Spoofax-PIE implementation. https://github.com/metaborg/spoofax- pie. (Visited on 10/04/2019). [133] Richard M. Stallman, Roland McGrath, and Paul D. Smith. GNU Make. Free Software Foundation, May 2016. [134] Standard ECMA-334 C# Language Specification, 5th edition. 2017. [135] Dave Steinberg, Frank Budinsky, Marcelo Paternostro, and Ed Merks. Eclipse Modeling Framework. 2nd ed. Addison-Wesley, 2009. [136] Michael J. Steindorfer and Jurgen J. Vinju. Artifact for ’Optimizing Hash- Array Mapped Tries for Fast and Lean Immutable JVM Collections’. https: //github.com/msteindorfer/oopsla15-artifact. 2015. (Visited on 10/04/2019).

BIBLIOGRAPHY 149 [137] Michael J. Steindorfer and Jurgen J. Vinju. “Optimizing hash-array mapped tries for fast and lean immutable JVM collections”. In: Proceed- ings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2015, part of SPLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015. Ed. by Jonathan Aldrich and Patrick Eugster. ACM, 2015, pp. 783–800. isbn: 978-1-4503- 3689-5. doi: 10.1145/2814270.2814312. [138] Tijs van der Storm, William R. Cook, and Alex Loh. “Object Grammars”. In: Software Language Engineering, 5th International Conference, SLE 2012, Dresden, Germany, September 26-28, 2012, Revised Selected Papers. Ed. by Krzysztof Czarnecki and Görel Hedin. Vol. 7745. Lecture Notes in Computer Science. Springer, 2012, pp. 4–23. isbn: 978-3-642-36089-3. doi: 10.1007/978-3-642-36089-3_2. [139] S. Doaitse Swierstra and Harald Vogt. “Higher Order Attribute Gram- mars”. In: Attribute Grammars, Applications and Systems, International Summer School SAGA, Prague, Czechoslovakia, June 4-13, 1991, Proceedings. Ed. by Henk Alblas and Borivoj Melichar. Vol. 545. Lecture Notes in Computer Science. Springer, 1991, pp. 256–296. isbn: 3-540-54572-7. [140] Symas. Lightning Memory-mapped Database. https://symas.com/lmdb/. (Visited on 10/04/2019). [141] Tamás Szabó, Sebastian Erdweg, and Markus Völter. “IncA: a DSL for the deﬁnition of incremental program analyses”. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016, Singapore, September 3-7, 2016. Ed. by David Lo, Sven Apel, and Sarfraz Khurshid. ACM, 2016, pp. 320–331. isbn: 978-1-4503-3845-5. doi: 10.1145/2970276.2970298. [142] M. Levin T. Hart. AI Memo 39 - The New Compiler. Tech. rep. MIT, 1962. [143] Walid Taha and Tim Sheard. “MetaML and multi-stage programming with explicit annotations”. In: Theoretical Computer Science 248.1-2 (2000), pp. 211–242. doi: 10.1016/S0304-3975(00)00053-0. [144] Task Engine Benchmarking Results. https : / / bitbucket . org / slde / opendata-experiments. (Visited on 10/04/2019). [145] Delft University of Technology. TUDelft Compiler Construction, Lab 6: Name Analysis. http://tudelft- in4303.github.io/assignments/ ms2/lab6.html. (Visited on 10/04/2019). [146] Sam Tobin-Hochstadt, Vincent St-Amour, Ryan Culpepper, Matthew Flatt, and Matthias Felleisen. “Languages as libraries”. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, San Jose, CA, USA, June 4-8, 2011. Ed. by Mary W. Hall and David A. Padua. ACM, 2011, pp. 132–141. isbn: 978-1-4503-0663-8. doi: 10.1145/1993498.1993514.

150 [147] Vlad A. Vergu, Pierre Néron, and Eelco Visser. “DynSem: A DSL for Dynamic Semantics Specification”. In: 26th International Conference on Rewriting Techniques and Applications, RTA 2015, June 29 to July 1, 2015, Warsaw, Poland. Ed. by Maribel Fernández. Vol. 36. LIPIcs. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2015, pp. 365–378. isbn: 978-3-939897-85-9. doi: 10.4230/LIPIcs.RTA.2015.365. [148] Sander Vermolen, Chris Melman, Elmer van Chastelet, Danny Groe- newegen, and Eelco Visser. YellowGrass source code. https://github. com/webdsl/yellowgrass. (Visited on 10/04/2019). [149] Sander Vermolen, Chris Melman, Elmer van Chastelet, Danny Groe- newegen, and Eelco Visser. YellowGrass: a tag-based issue tracker powered by WebDSL. https://yellowgrass.org/. (Visited on 10/04/2019). [150] Eelco Visser. “A Bootstrapped Compiler for Strategies (Extended Ab- stract)”. In: Strategies in Automated Deduction (STRATEGIES’99). Trento, Italy, July 1999, pp. 73–83. [151] Eelco Visser. “Program Transformation with Stratego/XT: Rules, Strate- gies, Tools, and Systems in Stratego/XT 0.9”. In: Domain-Specific Pro- gram Generation, International Seminar, Dagstuhl Castle, Germany, March 23-28, 2003, Revised Papers. Ed. by Christian Lengauer, Don S. Batory, Charles Consel, and Martin Odersky. Vol. 3016. Lecture Notes in Com- puter Science. Springer, 2003, pp. 216–238. isbn: 3-540-22119-0. doi: 10.1007/978-3-540-25935-0_13. [152] Eelco Visser. Scannerless Generalized-LR Parsing. Tech. rep. P9707. Pro- gramming Research Group, University of Amsterdam, July 1997. [153] Eelco Visser. “Syntax Definition for Language Prototyping”. PhD thesis. University of Amsterdam, Sept. 1997. [154] Eelco Visser. WebDSL Blog source code. https://github.com/webdsl/ blog. (Visited on 10/04/2019). [155] Eelco Visser, Guido Wachsmuth, Andrew P. Tolmach, Pierre Néron, Vlad A. Vergu, Augusto Passalaqua, and Gabriël Konat. “A Language Designer’s Workbench: A One-Stop-Shop for Implementation and Veri- fication of Language Designs”. In: Onward! 2014, Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflec- tions on Programming & Software, part of SPLASH ’14, Portland, OR, USA, October 20-24, 2014. Ed. by Andrew P. Black, Shriram Krishnamurthi, Bernd Bruegge, and Joseph N. Ruskiewicz. ACM, 2014, pp. 95–111. isbn: 978-1-4503-3210-1. doi: 10.1145/2661136.2661149. [156] Harald Vogt, S. Doaitse Swierstra, and Matthijs F. Kuiper. “Efficient Incremental Evaluation of Higher order Attribute Grammars”. In: PLILP. 1991, pp. 231–242. doi: 10.1007/3-540-54444-5_102. [157] Harald Vogt, S. Doaitse Swierstra, and Matthijs F. Kuiper. “Higher- Order Attribute Grammars”. In: PLDI. 1989, pp. 131–145.

BIBLIOGRAPHY 151 [158] Tobi Vollebregt, Lennart C. L. Kats, and Eelco Visser. “Declarative speci- ﬁcation of template-based textual editors”. In: International Workshop on Language Descriptions, Tools, and Applications, LDTA ’12, Tallinn, Estonia, March 31 - April 1, 2012. Ed. by Anthony Sloane and Suzana Andova. ACM, 2012, pp. 1–7. isbn: 978-1-4503-1536-4. doi: 10.1145/2427048. 2427056. [159] Markus Völter and Konstantin Solomatov. “Language Modularization and Composition with Projectional Language Workbenches illustrated with MPS”. In: Software Language Engineering, Third International Con- ference, SLE 2010. Ed. by Mark G. J. van den Brand, Brian Malloy, and Steffen Staab. Lecture Notes in Computer Science. Springer, 2010. [160] Guido Wachsmuth, Gabriël Konat, Vlad A. Vergu, Danny M. Groe- newegen, and Eelco Visser. “A Language Independent Task Engine for Incremental Name and Type Analysis”. In: Software Language Engineer- ing - 6th International Conference, SLE 2013, Indianapolis, IN, USA, October 26-28, 2013. Proceedings. Ed. by Martin Erwig, Richard F. Paige, and Eric Van Wyk. Vol. 8225. Lecture Notes in Computer Science. Springer, 2013, pp. 260–280. isbn: 978-3-319-02653-4. doi: 10.1007/978-3-319- 02654-1_15. [161] Guido Wachsmuth, Gabriël Konat, and Eelco Visser. “Language Design with the Spoofax Language Workbench”. In: IEEE Software 31.5 (2014), pp. 35–43. doi: 10.1109/MS.2014.100. [162] Edwin Westbrook, Mathias Ricken, Jun Inoue, Yilong Yao, Tamer Ab- delatif, and Walid Taha. “Mint: Java multi-stage programming using weak separability”. In: Proceedings of the 2010 ACM SIGPLAN Confer- ence on Programming Language Design and Implementation, PLDI 2010, Toronto, Ontario, Canada, June 5-10, 2010. Ed. by Benjamin G. Zorn and Alexander Aiken. ACM, 2010, pp. 400–411. isbn: 978-1-4503-0019-3. doi: 10.1145/1806596.1806642. [163] Eric Van Wyk, Derek Bodin, Jimin Gao, and Lijesh Krishnan. “Silver: An extensible attribute grammar system”. In: Science of Computer Program- ming 75.1-2 (2010), pp. 39–54. doi: 10.1016/j.scico.2009.07.004. [164] Dashing Yeh. “On Incremental Evaluation of Ordered Attribute Gram- mars”. In: BIT 23.3 (1983), pp. 308–320. doi: 10.1007/BF01934460. [165] Dashing Yeh and Uwe Kastens. “Improvements of an incremental evaluation algorithm for ordered attribute grammars”. In: SIGPLAN Notices 23.12 (1988), pp. 45–50. doi: 10.1145/57669.57672.

152 Curriculum Vitae

Gabriël Ditmar Primo Konat Born August 13th 1988 in The Hague, the Netherlands.

2018 - present Postdoctoral Research Delft University of Technology Department of Software Technology, Programming Languages group

2012 - 2018 Ph.D. in Computer Science Delft University of Technology Department of Software Technology, Programming Languages group

06/2013 - 08/2013, 07/2014 - 09/2014, 07/2015 - 09/2015 Research Assistant Oracle Labs in Redwood Shores, California, United States of America

2009 - 2012 M.Sc. in Computer Science (cum laude) Delft University of Technology Specialization: Software Engineering

2005 - 2009 B.Sc. in Computer Science Institute of Applied Sciences, Rijswijk Specialization: Software Development

2000 - 2005 HAVO diploma Segbroek College in The Hague Specialization: Nature & Technology

153 154 List of Publications

• Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Scalable incremental building with dynamic task dependencies”. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, September 3-7, 2018. Ed. by Marianne Huchard, Christian Kästner, and Gordon Fraser. ACM, 2018, pp. 76–86. doi: 10.1145/ 3238147.3238196 • Gabriël Konat, Michael J. Steindorfer, Sebastian Erdweg, and Eelco Vis- ser. “PIE: A Domain-Specific Language for Interactive Software Develop- ment Pipelines”. In: Programming Journal 2.3 (2018), p. 9. doi: 10.22152/ programming-journal.org/2018/2/9 • Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Bootstrapping Domain- Specific Meta-Languages in Language Workbenches”. In: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences, GPCE 2016, Amsterdam, The Netherlands, October 31 - November 1, 2016. Ed. by Bernd Fischer and Ina Schaefer. ACM, 2016, pp. 47–58. isbn: 978-1-4503-4446-3. doi: 10.1145/2993236.2993242 • Gabriël Konat, Luís Eduardo de Souza Amorim, Sebastian Erdweg, and Eelco Visser. Bootstrapping, Default Formatting, and Skeleton Editing in the Spoofax Language Workbench. Language Workbench Challenge (LWC@SLE). https : / / 2016 . splashcon . org / details / lwc2016 / 4 / Bootstrapping - Default - Formatting - and - Skeleton - Editing - in - the - Spoofax - Language-Workb. 2016 • Gabriël Konat, Sebastian Erdweg, and Eelco Visser. “Towards Live Language Development”. In: Workshop on Live Programming Systems (LIVE). 2016 • Sebastian Erdweg, Tijs van der Storm, Markus Völter, Laurence Tratt, Remi Bosman, William R. Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex Loh, Gabriël Konat, Pedro J. Molina, Martin Palatnik, Risto Pohjonen, Eugen Schindler, Klemens Schindler, Riccardo Solmi, Vlad A. Vergu, Eelco Visser, Kevin van der Vlist, Guido Wachsmuth, and Jimi van der Woning. “Evaluating and comparing language workbenches: Existing results and benchmarks for the future”. In: Computer Languages, Systems & Structures 44 (2015), pp. 24–47. doi: 10.1016/j.cl.2015.08.007 • Eelco Visser, Guido Wachsmuth, Andrew P. Tolmach, Pierre Néron, Vlad A. Vergu, Augusto Passalaqua, and Gabriël Konat. “A Language Designer’s Workbench: A One-Stop-Shop for Implementation and Verification of Lan- guage Designs”. In: Onward! 2014, Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, part of SPLASH ’14, Portland, OR, USA, October 20-24, 2014. Ed. by Andrew P. Black, Shriram Krishnamurthi, Bernd Bruegge, and Joseph

155 N. Ruskiewicz. ACM, 2014, pp. 95–111. isbn: 978-1-4503-3210-1. doi: 10.1145/2661136.2661149 • Guido Wachsmuth, Gabriël Konat, and Eelco Visser. “Language Design with the Spoofax Language Workbench”. In: IEEE Software 31.5 (2014), pp. 35–43. doi: 10.1109/MS.2014.100 • Sebastian Erdweg, Tijs van der Storm, Markus Völter, Meinte Boersma, Remi Bosman, William R. Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex Loh, Gabriël Konat, Pedro J. Molina, Martin Palatnik, Risto Pohjonen, Eugen Schindler, Klemens Schindler, Riccardo Solmi, Vlad A. Vergu, Eelco Visser, Kevin van der Vlist, Guido Wachsmuth, and Jimi van der Woning. “The State of the Art in Language Workbenches - Conclusions from the Language Workbench Challenge”. In: Software Language Engineering - 6th International Conference, SLE 2013, Indianapolis, IN, USA, October 26-28, 2013. Proceedings. Ed. by Martin Erwig, Richard F. Paige, and Eric Van Wyk. Vol. 8225. Lecture Notes in Computer Science. Springer, 2013, pp. 197–217. isbn: 978-3-319-02653-4. doi: 10.1007/978-3-319-02654-1_11 • Guido Wachsmuth, Gabriël Konat, Vlad A. Vergu, Danny M. Groenewegen, and Eelco Visser. “A Language Independent Task Engine for Incremental Name and Type Analysis”. In: Software Language Engineering - 6th Inter- national Conference, SLE 2013, Indianapolis, IN, USA, October 26-28, 2013. Proceedings. Ed. by Martin Erwig, Richard F. Paige, and Eric Van Wyk. Vol. 8225. Lecture Notes in Computer Science. Springer, 2013, pp. 260–280. isbn: 978-3-319-02653-4. doi: 10.1007/978-3-319-02654-1_15 • Gabriël Konat, Lennart C. L. Kats, Guido Wachsmuth, and Eelco Visser. “Declarative Name Binding and Scope Rules”. In: Software Language Engineer- ing, 5th International Conference, SLE 2012, Dresden, Germany, September 26-28, 2012, Revised Selected Papers. Ed. by Krzysztof Czarnecki and Görel Hedin. Vol. 7745. Lecture Notes in Computer Science. Springer, 2012, pp. 311–331. isbn: 978-3-642-36089-3. doi: 10.1007/978-3-642-36089-3_18

156