Improving Input-Output Conformance Testing Theories
Total Page：16
File Type：pdf, Size：1020Kb
Improving input-output conformance testing theories Citation for published version (APA): Noroozi, N. (2014). Improving input-output conformance testing theories. Technische Universiteit Eindhoven. https://doi.org/10.6100/IR780940 DOI: 10.6100/IR780940 Document status and date: Published: 01/01/2014 Document Version: Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement: www.tue.nl/taverne Take down policy If you believe that this document breaches copyright please contact us at: [email protected] providing details and we will investigate your claim. Download date: 28. Sep. 2021 Software quality has become an increasingly important concern in software development, as a consequence of the ever-ascending trend of using software system in critical systems. Testing as a technique for establishing a certain level of software quality has received much attention in the past decades. Model-based testing is a structured approach to testing. Using model- based testing the process of generating test- cases and predicting the correct outcome of test- cases can be mechanized. By using rigorous models for system behavior, model-based testing is formalized in terms of a mathematical notion of conformance. Input-output conformance (ioco) is a widely-studied and commonly-used conformance relation. The ioco relation has some limitation in testing concurrent systems. Such systems are composed of interacting components which often communicate asynchronously with their environment. However, the ioco testing theory was developed based on the assumption that a tester can always communicate with an implementation under test synchronously. It is also well-known that the ioco relation does not have the compositionality property. In this thesis theoretical foundations are developed to provide solutions to make the ioco relation suited for testing concurrent systems. Improving Input-Output Conformance Testing Theories PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven, op gezag van de rector magnificus prof.dr.ir. C.J. van Duijn, voor een commissie aangewezen door het College voor Promoties, in het openbaar te verdedigen op maandag 27 oktober 2014 om 16:00 uur door Neda Noroozi geboren te Mashhad, Iran Dit proefschrift is goedgekeurd door de promotoren en de samenstelling van de promotiecommissie is als volgt: voorzitter: prof.dr. B. Koren 1e promotor: prof.dr. M.R. Mousavi 2e promotor: prof.dr.ir. J.F. Groote copromotor(en): dr.ir. T.A.C. Willemse leden: prof.dr. M. van den Brand prof.dr. H. Brinksma (Universiteit Twente) dr. ir. G.J. Tretmans (Radboud University Nijmegen) prof.dr. J. Peleska (Universität Bremen) Improving Input-Output Conformance Testing Theories Neda Noroozi Copyright c 2014 by Neda Noroozi All rights reserved. Reproduction in whole or in part is prohibited without the written consent of the copyright owner. A catalogue record is available from Eindhoven University of Technology Library ISBN: 978-90-386-3703-7 IPA Dissertation Series 2014-13 Typeset with LATEX (TEXLive 2012) Cover design by Zahra Haririan Printed by the print service of Eindhoven University of Technology, The Netherlands The work in this thesis has been carried out under the auspices of the research school IPA (Institute for Programming research and Algorithmics). The author was working at Eindhoven University of Technology. Contents Preface iii 1 Introduction1 1.1 Validation and Verification..............................3 1.2 Testing..........................................4 1.3 Model-Based Testing.................................5 1.4 Formal Conformance Testing............................7 1.5 Research Questions..................................8 1.6 Outline and Origin of Chapters...........................9 2 Preliminaries 13 2.1 Labeled Transition Systems............................. 13 2.2 Test Cases........................................ 24 2.3 Input-Output Conformance Testing for IOLTSs................. 27 3 Implementing Input-Output Conformance Testing: A Case Study 31 3.1 EFT Switch Functionality.............................. 33 3.2 Testing the EFT Switch................................ 34 3.3 Test Results....................................... 43 3.4 Discussion....................................... 44 3.5 Closing Remarks.................................... 45 4 Implementation Relations 47 4.1 Input-Output Conformance Relations with Quiescence............ 48 4.2 The Effect of Models on the Power of Testing.................. 49 4.3 The Effect of Observations on the Power of Testing.............. 68 4.4 Test Case Generation................................. 73 4.5 Closing Remarks.................................... 80 5 On the Complexity of Input-Output Conformance Testing 83 5.1 ioco Checking for Nondeterministic Models.................. 85 5.2 A Coinductive Definition of ioco .......................... 89 5.3 ioco Checking of Deterministic Specifications................. 91 5.4 Conformance Checking for Internal-Choice Traces............... 98 5.5 Closing Remarks.................................... 102 i CONTENTS 6 Asynchronous Input-Output Conformance Testing 103 6.1 Adapting the ioco Relation to Asynchronous Setting.............. 105 6.2 Restricting Implementation Models........................ 114 6.3 Restricting Specification Models.......................... 119 6.4 Closing Remarks.................................... 128 7 Decomposability in Input Output Conformance Testing 131 7.1 Preliminaries...................................... 133 7.2 Decomposability.................................... 139 7.3 Sufficient Conditions for Decomposability.................... 141 7.4 A Condition for Strong Decomposability..................... 150 7.5 Closing Remarks.................................... 153 8 Conclusions 155 A Proofs of Formal Results in Chapter 6 159 B Proofs of Formal Results in Chapter 7 165 CUPPAAL Models 175 Bibliography 185 Summary 195 Curriculum Vitae 197 ii Preface I moved to the Netherlands four years ago to pursue my doctoral studies. As I worked on the research that has resulted in this dissertation, I have benefited greatly from the support and guidance of a number of people and I would like to take this opportunity to express my gratitude to those who helped me begin, carry out, and finish this research. First of all, I would like to express my special thanks to Mohammad Reza Mousavi, my first promoter. He introduced me to the joys of model-based testing, helped me start my PhD research in this field and has continued to advise and guide me through these four years. His support has not been limited to my work and I have learned a great deal from him in both my research and my personal life. I would like to thank my second promoter Jan Friso Groote, head of the MDSE group, for giving me the opportunity to work as a PhD candidate in his group. I am very grateful for all his support. I enjoyed being part of his group, and learned many things from the discussions I had with him. I would like to express my deepest gratitude to my daily supervisor, Tim Willemse, for his help and supervision. He contributed greatly to the details of my work, presented in this thesis. He always provided invaluable, precise and insightful comments to improve my work; his comments were crucial to all steps. I could always walk into his office to discuss any problems I had and I really appreciated it. I learned from him many valuable things about logical reasoning, problem analysis, and structurally presenting results. I would like to thank my co-authors for their collaboration on the papers that form the basis of this thesis. My special thanks goes to Hamid Reza Asadi and Ramtin Khosravi for their friendly cooperation and inspiring discussions we had on the project of applying model-based testing on an EFT switch. The results of that experience have motivated my PhD research. I thank Tim Willemse and Mohammad Reza Mousavi for their insightful comments and inspiring discussion on the topic of our papers. I would like to thank the committee for reviewing my thesis: