. . . . . Reviewer: Fabrice BOUQUET Reviewer: Yves LE TRAON Day of the defence: the 18th of April, 2013 ii Abstract The development of tests for avionics systems involves a multiplicity of in- house test languages, with no standard emerging. Test solution providers have to accommodate the habits of different clients, while the exchange of tests between aircraft manufacturers and their equipment/system providers is hindered. We propose a model-driven approach to tackle these prob- lems: test models would be developed and maintained in place of code, with model-to-code transformations towards target test languages. This thesis presents three contributions in this direction. The first one consists in the analysis of four proprietary test languages currently deployed. It al- lowed us to identify the domain-specific concepts, best practices, as well as pitfalls to avoid. The second contribution is the definition of a meta-model in EMF Ecore that integrates all identified concepts and their relations. The meta-model is the basis for building test model editors and code generation templates. Our third contribution is a demonstrator of how these technolo- gies are used for test development. It includes customizable graphical and textual editors for test models, together with template-based transforma- tions towards a test language executable on top of a real test platform. iv I dedicate my dissertation work to my family, my friends and to Calopsitta. Dedic aceast˘atez˘afamiliei mele, prietenilor ¸siCalopsittei. Acknowledgements It is my pleasure to acknowledge here those people whose presence in my life contributed to this dissertation : my academic and industrial advisors, colleagues, friends and family. I am truly indebted to H´el`eneWaeselynck, my academic advisor, for her mentorship. You taught me not only the rigours and intricacies of scientific research, but also how to be a better person. Your humane openness, sup- port and encouragement during my times of doubt were invaluable. This thesis would not have been possible without your wisdom and guidance. Thank you! I also owe sincere thankfulness to Virginie Wiels, my academic co-advisor. Thank you for your implication in my doctorate, your important suggestions and remarks, as well as for the confidence you have shown in me and for your kindness. I would like to express my gratitude to Yann Fusero, my industrial advisor, for his sustained support, as well as for having taught me the rigours of the industrial approach. I would like to express my sincere appreciation to Michel Schieber, my industrial co-advisor, for his interest and participation in my doctorate, as well as for having introduced me to a wide range of fields and activities. Special thanks go to Guy Durrieu for our valuable discussions and his caring and concern regarding this project. I would like to thank Yves le Traon and Fabrice Bouquet for having accepted to review my thesis, as well as Beno^ıtCombemale for having accepted to be a member of my defence committee. I would like to express my gratitude to the management of Cassidian Test & Services: Philippe Lasman and Philippe Quichaud, as well as to the management of the LAAS-CNRS (Laboratory for the Analysis and Archi- tecture of Systems): Karama Kanoun, Mohamed Kaaniche, Raja Chatila, Jean-Louis Sanchez and Jean Arlat, for having welcomed me. It has been a pleasure to work with my colleagues at Cassidian Test & Services. Special thanks go to Florent Paitrault for our rich discussions and techni- cal support, as well as to Gilles Ballanger, Guilhem Bonnafous, Mathieu Garcia and Etienne Allogo, for our rich interactions and especially for their implication in the development of our prototype. I am thankful to Val´erie Aouba for her help, especially towards the end of my doctorate, as well as for having taken the time to come to my thesis defence. I am highly indebted to Caroline Plana-Robert for her kindness and support, as well as for giving me the right advice at the right time. I would also like to thank my colleagues at the LAAS-CNRS, particularly Pierre Andr´e,Maxime Lastera and Fernand Lone-Sang. I am also grateful to Emmanuelle Beatse and Fran¸coiseDucor at Cassidian Test & Services, and Sonia de Sousa at the LAAS-CNRS, for all their help. I would like to express my deep appreciation to the people I worked with within the context of the VISTAS (Virtually Integrated System Tests be- tween Airframers and Suppliers) project. I would like to express my grat- itude to G´erardMontariol for his tutorship. Special thanks go to Olivier Fourcade, No¨elle Marchal and Fran¸coisRuiz, Dominique Dubost and Jean- Michel Lecuna, Mossad Khimoun and Christophe Ginestet, for our fruitful collaboration. Special thanks go to Albert Lahana for having accepted me as his student and for having taught me so much about myself through music, as well as to C´elineRomero for having coached me with her kind and wise words. I am highly indebted and thoroughly grateful to all my friends. My dis- sertation would not have been possible without your constant motivation, support and love. I would like to thank Roxana Albu, Ioana-Mihaela Geantˇa and Miruna Stoicescu, whom while pursuing their own doctorate have been extremely close to me, emotionally and professionally. I would also like to thank Julien Eballard, Alexandra-Mihaela Niculae, Iuniana Oprescu, Alexandra Pencea and Diana Stoica for our enduring friendship. You are all like a second family to me! I would also like to express my gratitude to J´eromeHaquin. Special thanks go to Beno^ıtBaccot, David Brown, Vi- vian Madesclaire and Fran¸cois-HenryRouet, as well as to Mihaela-Elena B^aclea,Daniel Ionescu, George Ionescu and Silvia-Maria Mihalea, for their friendship and for the quality time spent together. I would like to thank Calopsitta for her companionship. I am grateful to Paula Begoun for having given me great skin for the day of the defence. Last, but not the least, I would like to thank my family. Without the love and support of my parents, Tant¸a and Qwo-Wadis Guduvan, and of my grandparents, Tinca and Marin Maxim, and Elisabeta Kedves and Alexandru Guduvan, I would not have come this far. I would also like to thank Areta-Liliana Guduvan for her constant support. Special thanks go to Emilia and Dumitru Gudovan for their kindness. Contents List of Figures ix List of Tables xi Glossary xv 1 Introduction 1 2 State of the Art - Testing 5 2.1 Test Selection . .6 2.1.1 Structural Testing . .7 2.1.2 Functional Testing . .8 2.2 Test Oracle . 11 2.3 Test Development Methodologies . 11 2.4 Test Development Formalisms and Associated Tools . 12 2.5 Conclusion . 16 3 Industrial Context 19 3.1 Avionics Embedded System . 19 3.1.1 Life-Cycle and Testing Activities . 19 3.1.2 Interface Control Document (ICD) . 21 3.1.3 Reactive Behaviour . 23 3.2 Test Platform . 24 3.3 Stakeholders and Evolving Needs . 26 3.4 Conclusion . 27 4 Test Languages Analysis 29 4.1 Test Languages Sample . 30 4.2 Generic Features . 32 4.3 Test Organization . 34 4.3.1 Semantic Organization of Instructions . 35 4.3.2 Thread of Control Organization . 36 v CONTENTS 4.4 Link with System under Test Interfaces . 41 4.4.1 Targeted ICD Hierarchical Levels and Interface Abstraction . 41 4.4.2 SUT Interface Element Identifiers and Access . 42 4.5 Test Language Instructions . 45 4.5.1 System under Test Interaction . 46 4.5.2 Test Verdict Management . 51 4.6 Time Management . 51 4.7 Meta-Modelling Guiding Principles . 54 4.8 Conclusion . 56 5 Test Meta-Model 59 5.1 Eclipse Modeling Framework (EMF) Ecore . 60 5.2 ProviderData and UserData . 61 5.3 TestContext . 65 5.4 High-Level Structural Concepts . 66 5.4.1 SystemUnderTest . 67 5.4.2 TestCase . 69 5.4.3 TestComponent . 69 5.4.4 Verdict Management . 73 5.4.5 TestArchitecture . 73 5.4.6 TestGroup and TestSuite . 74 5.5 Low-Level Structural Concepts . 75 5.6 Behavioural Concepts . 77 5.6.1 ExecutionFlowStatement . 81 5.6.2 BasicStatement . 81 5.6.3 SpecificStatement . 82 5.7 Test Model Development Environment . 85 5.7.1 Graphical Editor . 85 5.7.2 Textual Editor . 86 5.8 Conclusion . 91 6 Test Model Implementation 93 6.1 Acceleo Model-to-Text Tool . 94 6.2 Target Python-Based Test Language (PL5)................ 99 6.2.1 Test Case . 100 6.2.2 Access to SUT Interfaces and Associated Test Actions . 101 6.2.2.1 Simple Test Actions . 101 6.2.2.2 Time-Dependent Test Actions . 102 6.2.2.3 Fault Injection Test Actions . 102 6.2.3 Test Component . 103 vi CONTENTS 6.2.4 Verdict Management . 104 6.2.5 Test Suite . 104 6.3 Architecture of Acceleo Modules/Templates . 105 6.3.1 ProviderData Implementation . 106 6.3.2 SystemUnderTest Implementation . 109 6.3.3 TestCase Implementation . 112 6.3.4 TestComponent Implementation . 114 6.3.5 TestArchitecture Implementation . 117 6.4 Conclusion . 119 7 Case Studies 123 7.1 FWS - Engine Fire Alarm Synthesis . 123 7.2 ADIRS - Consolidated Aircraft Speed Value . 125 7.3 Conclusion . 130 8 Conclusion and Perspectives 135 References 141 vii CONTENTS viii List of Figures 3.1 System under Test Life-Cycle and Testing Activities . 20 3.2 Interface Control Document Sample . 22 3.3 Test Platform Architecture Overview . 25 4.1 Test Exchange Language . 31 4.2 Tests Organization . 35 4.3 Thread of Control Organization . 36 4.4 Link with the System under Test Interfaces . 41 4.5 Access to the System under Test Interfaces .