Methods and Techniques for the Design and Implementation of Domain-Specific Languages PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Technische Universiteit Delft, op gezag van de Rector Magnificus prof. ir. K.C.A.M. Luyben, voorzitter van het College voor Promoties, in het openbaar te verdedigen op woensdag 11 januari 2012 om 12:30 uur door Zef HEMEL doctorandus informatica geboren te Groningen Dit proefschrift is goedgekeurd door de promotor: Prof. dr. A. van Deursen Copromotor: Dr. E. Visser Samenstelling promotiecommissie: Rector Magnificus voorzitter Prof. dr. A. van Deursen Delft University of Technology, promotor Dr. E. Visser Delft University of Technology, copromotor Prof. dr. h. c. ir. M. J. Plasmeijer Radboud University Nijmegen Prof. dr. M. G. van den Brand Eindhoven University of Technology Prof. dr. C. M. Jonker Delft University of Technology Prof. dr. ir. G. J. P. M. Houben Delft University of Technology Dr. W. R. Cook University of Texas at Austin The work in this thesis has been carried out at the Delft University of Tech- nology, under the auspices of the research school IPA (Institute for Program- ming research and Algorithmics). The research was financially supported by the Netherlands Organisation for Scientific Research (NWO)/Jacquard project 638.001.610, MoDSE: Model-Driven Software Evolution. Copyright c 2012 Zef Hemel Printed and bound in The Netherlands by CPI Wohrmann¨ Print Service. ISBN 978-90-8570-794-3 Preface I vividly remember my application interview for this Ph.D. position, early 2007. A time when buses still stopped almost in front of “the tallest building on campus” in Delft where the Software Engineering Research Group is situ- ated. My Ph.D. supervisor to be, Eelco Visser, decided to try out a new style of interviewing he had picked up during his postdoc in Portland, Oregon. This style entailed about four hours of interviews with him, Arie and other Ph.D. students in the group. The talks with Eelco were toughest, he asked me challenging questions, including “so, are you any good?” Arie spent his interviewing time laying out his vision for the group and its history. One Ph.D. student I talked to, Ali Mesbah, was happy that there was somebody applying with a web background; another, Martin Bravenboer, said I would really enjoy working with Eelco Visser. After hours of interviews they offered me the job with only one concern: “will you be able to focus on a single topic for four years?” Well, dear reader, you have the answer in your hands. It takes you on a journey of four years of research, Zef-style. I worked on topics ranging from workflow, intermediate languages, compiler implementation techniques to design concerns related to languages for web applications and mobile ap- plications. Yet, the questions I asked myself over the past four years have remained the same: how do we make software development more productive and more fun? How do we stop spending our programming time spelling out the nitty gritty detail that we don’t care about? Domain-specific languages are one way to accomplish that, and this little book contains my scribblings re- lated to both designing and implementing such languages. Even though the work is done, at least for me, I look forward to collaborat- ing with the people in Delft in the future. I’m happy to work at Cloud9 IDE, Inc. where I will apply much of what I have learned the past four years and I look forward to do so in collaboration with the people in Delft. Industry has a lot to learn from research — but research also has a lot to learn from industry. ACKNOWLEDGEMENTS As support during this process I would like to thank a couple of people. First and foremost, my supervisor and co-promotor Eelco Visser. He provided guidance and taught me everything I needed to know about how to conduct research and has been an inspiration throughout the process. Second, I would like to thank my promotor, Arie van Deursen, for his help, feedback and guid- ance along the way — whenever I felt insecure, he was reassuring. I would also like to thank my reading committee for their valuable feedback and kind words: William Cook, Rinus Plasmeijer, Mark van den Brand, Catholijn Jonker and Geert-Jan Houben. iii Then, I would like to thank all my other colleagues at SERG, and especially our coffee/tea/lunch group: Danny Groenewegen, Sander Vermolen, Sander van der Burg, Maartje de Jonge, Rob Vermaas, Eelco Dolstra and Lennart Kats (my bro’, who people always confused me with, for some unknown reason). I would also like to thank Esther van Seters, who helped me with the administrative side of finishing up this Ph.D. I would like to thank Martin Bravenboer and Eelco Dolstra for the source files that formed the basis for this thesis’ lay-out, and Alberto Gonzalez´ Sanchez´ for the inspiration for its cover. The photograph used for the cover was taken by Trey Ratcliff and is entitled “Morning. Coffee. Yellowstone. Fog.”1 The interpretation of how it relates to the thesis is left as an exercise to the reader. I would like to thank my family, for their support throughout the years — my parents in particular. And last, but surely not least, I would like to thank my wife Justyna — without her I would likely not have done this Ph.D., she supported me always. Zef Hemel December 8, 2011 Poznan,´ Poland 1http://www.flickr.com/photos/stuckincustoms/4885953697/ iv Contents 1 Introduction 1 1.1 Domain-Specific Languages . 1 1.2 DSLs and the Web . 3 1.3 WebDSL . 4 1.4 Problem statement . 4 1.5 Verification . 5 1.6 Coverage and Abstraction . 6 1.7 Compiler Modularity and Separation of Concerns . 7 1.8 Retargetability . 9 1.9 An Integrated DSL for the Mobile Web . 10 1.10 Research Method . 11 1.11 Origin of Chapters . 11 2 Static Consistency Checking of Web Applications with WebDSL 13 2.1 Introduction . 13 2.1.1 Causes of Late Failure . 14 2.1.2 Design for Consistency Checking . 15 2.1.3 Contributions . 15 2.2 Failures in Web Applications . 16 2.2.1 Web Application Aspects . 16 2.2.2 Moment of Failure . 18 2.2.3 Frameworks . 18 2.2.4 Case 1: Consistency of References to the Data Model . 19 2.2.5 Case 2: Consistency of Links to Pages . 20 2.2.6 Summary . 21 2.3 Framework Design and Fault Detection . 21 2.3.1 Reflection and Run-time Code Manipulation . 22 2.3.2 Linguistic Separation . 23 2.3.3 Limited Static Type Checking . 24 2.3.4 Run-time consistency checking . 25 2.3.5 Summary . 25 2.4 Designing for Static Verifiability . 26 2.4.1 Data Model . 27 2.4.2 User interface . 27 2.4.3 Application Logic . 28 2.4.4 Access Control . 28 2.4.5 Verifiability versus Flexibility . 29 2.5 Rule-Based Consistency Checking . 30 2.5.1 Language Definition . 30 2.5.2 Static Consistency Checking . 31 2.5.3 Stratego . 33 v 2.5.4 Name Resolution . 34 2.5.5 Type Analysis . 37 2.5.6 Consistency Checking . 38 2.5.7 Summary . 40 2.6 Discussion and Related Work . 40 2.6.1 Consistency Checking Capabilities Integrated Into Lan- guages and Frameworks . 40 2.6.2 External Consistency Checkers . 43 2.6.3 Finding Faults by Unit Testing . 44 2.6.4 Previous Work . 45 2.6.5 Future work . 45 2.7 Conclusion . 46 Postscript . 47 3 WebWorkFlow 49 3.1 Introduction . 49 3.1.1 Contributions . 51 3.1.2 Outline . 51 3.2 WebWorkFlow by Example . 52 3.3 Transforming Procedures . 56 3.4 Encoding Patterns . 61 3.5 Discussion . 62 3.6 Conclusion . 65 Postscript . 67 4 Code Generation by Model Transformation 69 4.1 Introduction . 69 4.1.1 Contributions . 70 4.1.2 Outline . 71 4.2 WebDSL . 72 4.2.1 Data Model . 72 4.2.2 User Interface . 72 4.2.3 Data Operations . 73 4.3 Implementing WebDSL by Code Generation . 74 4.4 Code Generation by Term Rewriting . 76 4.4.1 Representing Models and Code with Terms . 78 4.4.2 Rewrite Rules . 79 4.4.3 Concrete Object Syntax . 79 4.4.4 Rewriting versus Template Engines . 80 4.4.5 Composing Generation Rules . 81 4.5 Transforming Generated Code . 82 4.5.1 Identifier Composition . 83 4.5.2 Partial Classes and Methods . 83 4.5.3 Interface Extraction . 85 4.5.4 A Revised Pipeline . 85 4.6 Model-to-Model Transformations . 85 vi 4.6.1 Modules . 86 4.6.2 Modular Data Models . 87 4.6.3 Template Definitions . 88 4.6.4 Deriving User Interface Elements from Types . 88 4.6.5 Access Control . 88 4.6.6 Workflow . 90 4.7 Implementing Model-to-Model Transforms . 90 4.7.1 Local-to-local . 91 4.7.2 Global-to-local . 91 4.7.3 Local-to-global . 92 4.8 Transformation Modularity and Extensibility . 93 4.8.1 Staged Normalization . 94 4.8.2 Innermost Normalization . 95 4.8.3 Normalization with Local-to-Global Rules . 96 4.9 Combining Type Analysis and Transformation . 97 4.9.1 Integrating Type Analysis and Transformation . 98 4.9.2 Name Resolution . 100 4.9.3 Type Analysis . 102 4.9.4 Type Constraints . 102 4.9.5 Type Analysis during Transformation . 102 4.10 Discussion . 104 4.10.1 Compilation by Normalization . 104 4.10.2 Rewriting Tools . 105 4.10.3 Model Transformation .