Generative Programming Principles and Techniques of Software Engineering Based on Automated Configuration and Fragment-Based Component Models by Dipl.-Inf. Krzysztof Czarnecki A dissertation submitted in partial fulfillment of the requirements for the degree of Doktor-Ingenieur Department of Computer Science and Automation Technical University of Ilmenau October 1998 Supervisory Committee: Universitair Hoofd Docent Dr. Mehmet Aksit, University of Twente, The Netherlands Prof. Dr. Ulrich W. Eisenecker, FH Heidelberg, Germany Prof. Dr.-Ing. habil. Dietrich Reschke (Chairperson of Supervisory Committee), TU Ilmenau, Germany Oral defense held on August 9, 1999 iii Abstract Current object-oriented (OO) and component technologies suffer from several problems such as the lack of analysis and design methods for the development for reuse, lack of effective techniques for dealing with many variants of components, loss of design knowledge due to the semantic gap between domain abstractions and programming language features, and runtime performance penalties for clean and flexible design. This thesis proposes Generative Programming (GP) as a comprehensive software development paradigm to achieving high intentionality, reusability, and adaptability without the need to compromise the runtime performance and computing resources of the produced software. In the area of analysis and design for GP, we investigate Domain Engineering (DE) methods and their integration with OO analysis and design (OOA/D) methods. The main difference between DE methods and OOA/D methods is that the first are geared towards developing whole families of systems while the latter focus on developing single systems. We identify feature modeling as the main contribution of DE to OOA/D. Feature models represent the configurability aspect of reusable software at an abstract level, i.e. without committing to any particular implementation technique such as inheritance, aggregation, or parameterized classes. We give a precise and extended formulation of the feature diagram notation and investigate the relationship between feature modeling, OO modeling and Aspect- Oriented Programming. In the area of implementation for GP, we study various metaprogramming technologies. We identify modularly extensible programming environments as the ideal programming platform for GP, which allows implementing domain-specific optimizations, domain-specific displaying and editing, domain-specific debugging and code analysis, new composition mechanisms, etc., in a scalable way. We also propose new implementation techniques such as configuration generators based on mixin models with automatic configuration and configuration repositories and make several contribution to template metaprogramming. Based on the analysis of the areas mentioned above, we propose a new Domain Engineering method for the development of algorithmic reusable libraries (DEMRAL), which integrates various DE, OO, and AOP concepts. We validate the method by applying it to the domain of matrix computations, which results in the development of the Generative Matrix Computation Library (GMCL). We provide two implementation GMCL, one using generative programming techniques in C++ and another one in Intentional Programming (an modularly extendible programming environment). In addition to validating the usefulness of DEMRAL, the GMCL case study provides a concrete comparison of two generative implementation technologies. The C++ implementation of the matrix component (which is a part of C++ GMCL) comprises only 7500 lines of C++ code, but it is capable of generating more than 1840 different kinds of matrices. Despite the large number of provided matrix variants, the performance of the generated code is comparable with the performance of manually coded variants. The application of template metaprogramming allowed a highly intentional library API and a highly efficient library implementation at the same time. The implementation of GMCL within the Intentional Programming system (IP) demonstrates the advantages of IP, particularly in the area of debugging and displaying. iv Acknowledgements First I would like to thank my wife Katja for her love, encouragement, and patient support. I am also grateful to my parents for giving me best of their life and the necessary education. I would like to thank Dietrich Reschke, the chairman of supervisory committee for his invaluable advice, excitement, and generous support for this work. I am very grateful to Ulrich Eisenecker, one of the supervisors of this thesis, for encouraging me to work in this area, for giving me his professional guidance and advice throughout all phases of this work, for sharing many inspiring thoughts during our close research cooperation on Generative Programming, and for carefully reading and commenting hundreds of pages of all my professional writing. I also would like to express special thanks to Mehmet Aksit for his valuable suggestions and encouragement and for kindly accepting to be one of the supervisors of this thesis. Many thanks also go to Reinhold Schönefeld, who supervised the initial phase of my doctorial research. During my research on this thesis I had the pleasure to supervise two brilliant students Tobias Neubert and Johannes Knaupp. Tobias implemented the full version of the generative C++ matrix component and is also responsible for many improvements of the configuration generator concept. Johannes contributed several improvements of the template metaprogramming techniques, extended the C++ matrix component, and developed a configuration generator for the family of LU matrix factorization algorithms (work directly building on this thesis). This thesis would not have been finished without the generous support and encouragement of my friends and colleagues at Daimler-Benz AG Research and Technology in Ulm, especially my managers Wolfgang Hanika and Roland Trauter and my colleagues from the OSVA project team Wolfgang Köpf and Michael Jungmann. Special thanks go to Charles Simonyi, Greg Shaw, and the entire Intentional Programming (IP) team at Microsoft Research for patiently teaching me in the ways of IP and helping me with the IP implementation of the matrix case study during my research visit in January 1998. I would also like to thank Gregor Kiczales, Crista Lopes, John Lamping, and other members of the Aspect-Oriented Programming (AOP) team at Xerox PARC for explaining to me the AOP ideas, providing me with an early version of AspectJ, and sharing many stimulating thoughts that lead to the development of Dynamic Cool during my research visit in October/November 1997. I am grateful to Mark Simos who shared with me both his invaluable insights into the area of Domain Engineering and his excitement for this area. I also would like to express my thanks to Don Batory for the advice and positive feedback he gave me. Attending his tutorial on generators at ICSR’96 provided me with important motivation for doing this research. Many thanks also go to Karl Lieberherr for his positive feedback and the many stimulating discussions we had. During the Dagstuhl seminar on “Generic Programming” in March 1998, Ulrich Eisenecker, Robert Glück, Daveed Vandevoorde, and Todd Veldhuizen and I started an informal working group on Active Libraries. Our numerous discussions at Dagstuhl and later per e-mail greatly influenced the last steps of this work. I would like to thank all of the group members. During several conferences, I had the pleasure to meet many interesting people from the reuse community and the object-oriented community who gave me useful feedback and shared with me their ideas. My thanks go to Ted Biggerstaff, Ulrich Breymann, Mike VanHilst, Sholom Cohen and the participants of the WISR’97 working group on Domain Engineering and Object Technology, Andrew Lumsdaine, Satoshi Matsuoka, David Quinlan, Jeremy Siek, Yannis Smaragdakis, Douglas Smith, Partick Steyart, Bruce Weide, and many others. I also would like to thank many other researchers, including Ira Baxter, John Favaro, Martin Griss, and Jim Neighbors, who played important roles in helping me form my ideas. v In addition to the supervisors, several other researchers, including William Harrison, Gregor Kiczales, Karl Lieberherr, Harold Ossher, Greg Shaw, Mark Simos, and Peri Tarr gave me invaluable comments on various chapters of earlier drafts of this thesis. I also would like to thank Bogdan Franczyk and Witold Wendrowski for their professional support and friendship. Funding for this work has been provided by Daimler-Benz AG and German Federal Ministry of Education, Science, Research and Technology (BMBF) through the OSVA project. Knowing that this list is certainly not exhaustive, I hope that those who I have forgotten to mention will forgive me. vi Overview Part I Setting the Stage_________________________________________ 1 Chapter 1 What Is this Thesis About?____________________________ 3 Chapter 2 Conceptual Modeling _______________________________ 17 Part II Analysis and Design Methods and Techniques________________ 31 Chapter 3 Domain Engineering ________________________________ 33 Chapter 4 Domain Engineering and Object-Oriented Analysis and Design66 Chapter 5 Feature Modeling __________________________________ 83 Part III Implementation Technologies __________________________ 117 Chapter 6 Generators _______________________________________ 119 Chapter 7 Aspect-Oriented Decomposition and Composition _______ 181 Chapter 8 Static Metaprogramming in C++______________________ 241 Part IV Putting