Framework-Specific Modeling Languages by MichalAntkiewicz A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Electrical and Computer Engineering Waterloo, Ontario, Canada, 2008 c Micha lAntkiewicz 2008 ISBN: 978-0-494-43232-7 I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. Micha lAntkiewicz ii Abstract Framework-specific modeling languages (FSMLs) help developers build applications based on object-oriented frameworks. FSMLs formalize abstractions and rules of the framework's application programming interfaces (APIs) and can express models of how applications use an API. Such models, referred to as framework-specific models, aid developers in understanding, creating, and evolving application code. We present the concept of FSMLs, propose a way of specifying their abstract syntax and semantics, and show how such language specifications can be interpreted to provide reverse, forward, and round-trip engineering of framework-specific mod- els and framework-based application code. We present a method for engineering FSMLs that was extracted post-mortem from the experience of building four such languages. The method is driven by the use cases that the FSMLs under development are to support. We present the use cases, the overall process, and its instantiation for each language. The presenta- tion focuses on providing concrete examples for engineering steps, outcomes, and challenges. It also provides strategies for making engineering decisions. The presented method and experience are aimed at framework developers and tool builders who are interested in engineering new FSMLs. Furthermore, the method represents a necessary step in the maturation of the FSML concept. Finally, the presented work offers a concrete example of software language engineering. FSML engineering formalizes existing domain knowledge that is not present in language form and makes a strong case for the benefits of such formalization. We evaluated the method and the exemplar languages. The evaluation is both empirical and analytical. The empirical evaluation involved measuring the precision and recall of reverse engineering and verifying the correctness or forward and round- trip engineering. The analytical evaluation focused on the generality of the method. iii Acknowledgements I would like to thank the people who have contributed to this thesis. First and foremost, I would like to thank my supervisor Professor Krzysztof Czarnecki who enormously helped to shape, reinvent, and crystallize the material presented in this thesis in countless discussions. I am deeply grateful that despite his busy schedule, Professor Czarnecki always found the time to delve into the details of my research and that he was never satisfied with \easy" answers. His emphasis on the quality of research is a valuable lesson I am striving to follow. I would also like to thank Professor Ladan Tahvildari, Professor Kostas Kon- togiannis, and Professor Michael Godfrey for serving on my thesis committee. I thank Professor Ralf L¨ammelfor being my external examiner. I am also deeply grateful to my wife Agata, who's continuous support, trust, and understanding were crucial to finishing this thesis. I would like to thank my colleagues from the Generative Software Development Lab at the University of Waterloo who contributed to this research and its imple- mentation: Thiago Tonelli Bartolomei, Herman Lee, and Matthew Stephan. I also thank Henry Lau for his work on the implementation. iv Acknowledgment of contributions Most of the material presented in this thesis was taken from existing and submitted papers. The structure of the thesis is based on [12], to which my personal contri- bution was approximately 70%. Chapter 4 is based on [14], to which my personal contribution was approximately 70%. Section 2.5 is based on [10], to which my personal contribution was approximately 90%. Micha lAntkiewicz As one of the authors of [10, 12, 14], I fully acknowledge the thesis author's statement above and give full permission to use any part of the papers we co- authored mentioned above. I also fully acknowledge that the thesis represents original research conducted by the thesis author. Krzysztof Czarnecki As one of the authors of [14], I fully acknowledge the thesis author's statement above and give full permission to use any part of the papers we co-authored men- tioned above. I also fully acknowledge that the thesis represents original research conducted by the thesis author. Thiago Tonelli Bartolomei As one of the authors of [12], I fully acknowledge the thesis author's statement above and give full permission to use any part of the papers we co-authored men- tioned above. I also fully acknowledge that the thesis represents original research conducted by the thesis author. Matthew Stephan v Dedication I would like to dedicate this thesis to my mother Ma lgorzataand my wife Agata. vi Contents List of Tables xii List of Figures xiv 1 Introduction 1 1.1 Contributions . 5 2 Framework-Specific Modeling Languages 6 2.1 Overview . 6 2.2 Feature Models of Abstract Syntax . 10 2.3 Mapping Definitions . 15 2.4 Code Queries and Transformations . 17 2.5 Metamodel Interpretation Algorithms . 19 2.5.1 Forward engineering . 20 2.5.2 Reverse engineering . 21 2.5.3 Round-trip engineering . 24 2.6 The Generic FSML Infrastructure . 28 2.7 Summary . 29 3 Method for Engineering FSMLs 30 3.1 Overview . 30 3.2 Inception . 32 3.3 Elaboration . 39 3.4 Construction . 42 3.5 Transition . 49 3.6 Summary . 51 vii 4 Evaluation of Reverse Engineering 52 4.1 Introduction . 52 4.2 Challenges of Statically Analyzing Completion Code . 53 4.3 Setup of the Study . 54 4.3.1 Setup of phase 1 . 54 4.3.2 Setup of phase 2 . 55 4.3.3 Setup of phase 3 . 56 4.3.4 Data collection process . 56 4.4 Results of Phase 1: Code Patterns & Code Queries . 58 4.5 Results of Phase 2: Evaluation of the Simple Code Queries . 64 4.5.1 Precision & recall . 64 4.5.2 The Refined Code Queries . 65 4.5.3 Interpretation of the data . 66 4.5.4 Conclusion for phases 1 and 2 . 70 4.6 Results of Phase 3: the Refined Code Queries . 70 4.6.1 Static analysis services used by the code queries . 70 4.6.2 Implementation of the refined code queries . 75 4.6.3 Precision & recall data and interpretation . 77 4.6.4 Conclusion for phase 3 . 80 4.7 Discussion . 82 4.7.1 Threats to validity . 82 4.7.2 Empirical approach to code query refinement . 84 4.7.3 Difficulties of analyzing and understanding framework-based code . 84 4.8 Conclusion . 86 5 Evaluation of Forward Engineering 87 5.1 Introduction . 87 5.2 Code Transformations for Java . 88 5.3 Setup of the Test . 90 5.4 Results and Discussion . 93 5.4.1 Threats to validity . 94 5.5 Conclusion . 95 viii 6 Evaluation of Round-Trip Engineering 96 6.1 Introduction . 96 6.2 Setup of the Tests . 97 6.3 Results and Discussion . 98 6.3.1 Results of test 1 . 98 6.3.2 Results of test 2 . 99 6.3.3 Threats to validity . 99 6.4 Conclusion . 101 7 Evaluation of the Method 102 7.1 Evaluation of the Exemplar Languages . 102 7.1.1 Framework API understanding . 102 7.1.2 Completion code understanding and analysis . 103 7.1.3 Completion code generation and evolution . 105 7.2 Threats to Validity . 106 8 Related Work 109 8.1 Framework Instantiation . 109 8.2 Reverse Engineering . 111 8.3 Forward Engineering . 112 8.4 Round-Trip Engineering . 114 8.5 Reengineering . 114 8.6 FSML Engineering Method . 115 9 Conclusion 118 9.1 Summary of Contributions . 118 9.2 Limitations and Future Work . 119 APPENDICES 121 A Mapping types, Constraints, and Forward parameters 122 B Applications Used in the Third Phase of the Evaluation of Reverse Engineering 124 B.1 Eclipse . 124 B.2 Struts . 124 B.3 Applets . 124 ix C Applications Used in the Computation of the CDC and CDO Met- rics 126 C.1 Eclipse . 126 C.2 Struts . 126 C.3 Applets . 126 D Models Used in the Evaluation of Forward and Round-trip Engi- neering 128 E Traces of the Execution of the Tests in the Evaluation of Round- trip Engineering 135 F Complete Metamodels of the Exemplar FSMLs 138 F.1 Metamodel of WPI FSML . 139 F.2 Metamodel of Struts FSML . 141 F.3 Metamodel of Applet FSML . 142 F.4 Metamodel of EJB FSML . 144 References 147 x List of Tables 2.1 Examples of languages, their artifacts, and language definition for- malisms . 7 2.2 Selected mapping types for structural code patterns for Java . 15 2.3 Selected Mapping types for behavioural code patterns for Java . 17 2.4 Selected code transformations . 21 2.5 Selected code queries . 23 2.6 Synchronization States Decision Table . 27 3.1 Inception of the exemplar FSMLs . 33 3.2 Overview of the Use Cases . 34 3.3 Distribution of features over sources of knowledge . 37 3.4 Legend of Features' Sources . 37 4.1 Mapping types for structural code patterns for Java . 58 4.2 Mapping types for behavioural code patterns for Java . 59 4.3 Code queries for the callsTo mapping type . 62 4.4 Code queries for the callsRec mapping type . 62 4.5 Code queries for the argVal mapping type . 62 4.6 Code queries for the argSameObj mapping type . 62 4.7 Code queries for the before mapping type .