Dsl Engineering 1
Total Page:16
File Type:pdf, Size:1020Kb
dsl engineering 1 DSL Engineering Designing, Implementing and Using Domain-Specific Languages Markus Voelter with Sebastian Benz Christian Dietrich Birgit Engelmann Mats Helander Lennart Kats Eelco Visser Guido Wachsmuth (c) 2010 - 2013 Markus Voelter Feedback, Slides, and other updates at http://dslbook.org This PDF book is donationware: while you can get the PDF for free from dslbook.org, I would highly appreciate if you used the Donate button on that website to send over some amount of money you deem appropriate for what you get from the book. Thank you! A printed version of this book is available as well. Please go to http://dslbook.org to find links to various Amazon stores where you can buy the print version. Contents I Introduction 5 1 About this Book 7 1.1 Thank You! . 7 1.2 This book is Donationware . 8 1.3 Why this Book . 9 1.4 What you will Learn . 10 1.5 Who should Read this Book . 10 1.6 About the Cover . 11 1.7 Feedback, Bugs and Updates . 11 1.8 The Structure of the Book . 11 1.9 How to Read the Book . 12 1.10 Example Tools . 13 1.11 Case Studies and Examples . 14 2 Introduction to DSLs 23 2.1 Very Brief Introduction to the Terminology . 23 2.2 From General Purpose Languages to DSLs . 25 2.3 Modeling and Model-Driven Development . 29 2.4 Modular Languages . 32 2.5 Benefits of using DSLs . 38 2.6 Challenges . 41 2.7 Applications of DSLs . 45 2.8 Differentiation from other Works and Approaches 48 II DSL Design 51 3 Conceptual Foundations 55 3.1 Programs, Languages and Domains . 55 3.2 Model Purpose . 59 3.3 The Structure of Programs and Languages . 61 4 dslbook.org 3.4 Parsing versus Projection . 65 4 Design Dimensions 67 4.1 Expressivity . 68 4.2 Coverage . 78 4.3 Semantics and Execution . 80 4.4 Separation of Concerns . 100 4.5 Completeness . 109 4.6 Language Modularity . 114 4.7 Concrete Syntax . 130 5 Fundamental Paradigms 139 5.1 Structure . 139 5.2 Behavior . 147 5.3 Combinations . 156 6 Process Issues 159 6.1 DSL Development . 159 6.2 Using DSLs . 166 III DSL Implementation 171 7 Concrete and Abstract Syntax 175 7.1 Fundamentals of Free Text Editing and Parsing . 177 7.2 Fundamentals of Projectional Editing . 186 7.3 Comparing Parsing and Projection . 187 7.4 Characteristics of AST Formalisms . 194 7.5 Xtext Example . 198 7.6 Spoofax Example . 205 7.7 MPS Example . 210 8 Scoping and Linking 219 8.1 Scoping in Spoofax . 221 8.2 Scoping in Xtext . 227 8.3 Scoping in MPS . 232 9 Constraints 237 9.1 Constraints in Xtext . 239 9.2 Constraints in MPS . 240 9.3 Constraints in Spoofax . 245 10 Type Systems 251 10.1 Type Systems Basics . 252 10.2 Type Calculation Strategies . 253 dsl engineering 5 10.3 Xtext Example . 258 10.4 MPS Example . 261 10.5 Spoofax Example . 265 11 Transformation and Generation 269 11.1 Overview of the approaches . 270 11.2 Xtext Example . 272 11.3 MPS Example . 279 11.4 Spoofax Example . 288 12 Building Interpreters 295 12.1 Building an Interpreter with Xtext . 297 12.2 An Interpreter in MPS . 304 12.3 An Interpreter in Spoofax . 306 13 IDE Services 311 13.1 Code Completion . 311 13.2 Syntax Coloring . 314 13.3 Go-to-Definition and Find References . 319 13.4 Pretty-Printing . 321 13.5 Quick Fixes . 325 13.6 Refactoring . 327 13.7 Labels and Icons . 331 13.8 Outline . 332 13.9 Code Folding . 334 13.10Tooltips/Hover . 335 13.11Visualizations . 337 13.12Diff and Merge . 339 14 Testing DSLs 341 14.1 Syntax Testing . 342 14.2 Constraints Testing . 344 14.3 Semantics Testing . 347 14.4 Formal Verification . 354 14.5 Testing Editor Services . 361 14.6 Testing for Language Appropriateness . 365 15 Debugging DSLs 367 15.1 Debugging the DSL Definition . 367 15.2 Debugging DSL Programs . 374 16 Modularization, Reuse and Composition 391 16.1 Introduction . 391 16.2 MPS Example . 392 6 dslbook.org 16.3 Xtext Example . 412 16.4 Spoofax Example . 426 IV DSLs in Software Engineering 437 17 DSLs and Requirements 441 17.1 What are Requirements? . 441 17.2 Requirements versus Design versus Implemen- tation . 443 17.3 Using DSLs for Requirements Engineering . 445 17.4 Integration with Plain Text Requirements . 448 18 DSLs and Software Architecture 453 18.1 What is Software Architecture? . 453 18.2 Architecture DSLs . 455 18.3 Component Models . 466 19 DSLs as Programmer Utility 475 19.1 The Context . 476 19.2 Jnario Described . 477 19.3 Implementation . 480 19.4 Summary . 485 20 DSLs in the Implementation 487 20.1 Introduction . 487 20.2 Challenges in Embedded Software . 489 20.3 The mbeddr Approach . 491 20.4 Design and Implementation . 499 20.5 Experiences . 509 20.6 Discussion . 516 21 DSLs and Product Lines 519 21.1 Introduction . 519 21.2 Feature Models . 520 21.3 Connecting Feature Models to Artifacts . 521 21.4 From Feature Models to DSLs . 526 21.5 Conceptual Mapping from PLE to DSLs . 532 22 DSLs for Business Users 537 22.1 Intentional Software . 537 22.2 The Project Challenge . 538 22.3 The DSL-Based Solution . 539 22.4 Wrapping Up . 556 Part I Introduction 1 About this Book This book is about creating domain-specific languages. It covers three main aspects: DSL design, DSL implementa- tion and software engineering with DSLs. The book only looks at external DSLs and focuses mainly on textual syn- tax. The book emphasizes the use of modern language work- benches. It is not a tutorial for any specific tool, but it pro- vides examples and some level of detail for some of them: Xtext, MPS and Spoofax. The goal of the book is to provide a thorough overview of modern DSL engineering. The book is based on my own experience and opinions, but strives to be objective. 1.1 Thank You! Before I do anything else, I want to thank my reviewers. This book has profited tremendously from the feedback you sent me. It is a lot of work to read a book like this with sufficient concentration to give meaningful feedback. All of you did that, so thank you very much! Here is the list, in alphabetical order: Alexander Shatalin, Bernd Kolb, Bran Selic, Christa Schwan- ninger, Dan Ratiu, Domenik Pavletic, Iris Groher, Jean Bezivin, Jos Warmer, Laurence Tratt, Mats Helander, Nora Ludewig, Se- bastian Zarnekow and Vaclav Pech. I also want to thank the contributors to the book. They have added valuable perspectives and insights that I couldn’t have delivered myself. In particular: • Eelco Visser contributed significantly to the DSL design sec- tion. In fact, this was the part we started out with when we still had the plan to write the book together. It was initially 10 dslbook.org his idea to have a section on DSL design that is independent of implementation concerns and particular tools. • Guido Wachsmuth and Lennart Kats contributed all the ex- amples for the Spoofax language workbench and helped a lot with the fundamental discussions on grammars and parsing. • Mats Helander contributed the Business DSL case study with the Intentional Domain Workbench in Part IV of the book. • Birgit Engelmann and Sebastian Benz wrote the chapter on utility DSLs that features the JNario language based on Xtext and Xbase. • Christian Dietrich helped me with the language modular- ization examples for Xtext and Xbase. Also, Moritz Eysholdt has contributed a section on his Xpext testing framework. Finally, some parts of this book are based on papers I wrote with other people. I want to thank these people for letting me use the papers in the book: Bernd Kolb, Domenik Pavletic, Daniel Ratiu and Bernhard Schaetz. A special thank you goes to my girlfriend Nora Ludewig. She didn’t just volunteer to provide feedback on the book, she also had to endure all kinds of other discussions around the topic all the time. Thanks Nora! Alex Chatziparaskewas, Henk Kolk, Magnus Christerson and Klaus Doerfler allowed me to use "their" applications as examples in this book. Thank you very much! I also want to thank itemis, for whom I have worked as an independent consultant for the last couple of years. The expe- rience I gained there, in particular while working with MPS in the LWES research project, benefitted the book greatly! Finally, I want to thank my copyeditor Steve Rickaby. I had worked with Steve on my pattern books and I really wanted to work with him again on this one – even though no publisher is involved this time. Luckily he was willing to work with me directly. Thank you, Steve! 1.2 This book is Donationware This book is available as a print version and as a PDF version. You are currently reading the PDF version. The print version dsl engineering 11 can be acquired Amazon. The specific links are available at http://dslbook.org. This book is not free even though you can download it from Donations are much simpler than http://dslbook.org without paying for it. I ask you to please paying for the book at the time when when you get it: you can check out go to the website and donate some amount of money you deem the contents before you pay, you can appropriate for the value you take from the book. You should pay whatever you think makes sense for you, and we all don’t have to deal also register the book at the website, so I can keep you up-to- with DRM and chasing down "illegal date with new versions.