Faculty of Technology and Society Master thesis Proof-of-concept of Model-based testing based on an UML-model of a water-level measurement system Zoubida Alshekhly Namra Jamshaid Gill Exam: Computer Science, Master’s Programme (One Year) Subject Area: Computer Science Date: 08/11-2019 Abstract Software testing is a very important phase in software development as it minimize risks in a software system, however, it consumes time and can be very expensive. With automatic test case generation time consumption and cost can be reduced. Model-based testing is a method to test a software system with a model of the systems behaviour. Automatic test case generation is often considered a favorable support in model-based testing. In this work, the concept of model-based testing is explored along with testing the embedded part of a water-level measurement system (WLM) to investigate the efficiency of model-based testing on a software system. As a result of this, a model-based testing tool, MoMut::UML is used to generate the test-cases on the UML model of WLM system that is built in a UML modeling environment, Eclipse-Papyrus. However, MoMut::UML implements a special type of model-based testing technique, model-based mutation testing; that injects faults in the UML model, and generates test-data on the fault-based model. By this, the behaviour of system-under-test, only the UML model of water-level measurement system, is tested. Keywords: Model-based testing, Model-based testing tool, software system, UML, auto- matic test case generation. Glossary LORAWAN: The LoRaWAN specification is a Low Power, Wide Area (LPWA) network- ing protocol designed to wirelessly connect battery operated ‘things’ to the internet in regional, national or global networks, and targets key Internet of Things (IoT) require- ments such as bi-directional communication, end-to-end security, mobility and localization services. System under test (SUT):System under test refers to a system that is being tested for correct operation. Test oracles:Test oracle, is a mechanism for determining whether a test has passed or failed. The Object Constraint Language (OCL): is a declarative language describing rules applying to Unified Modeling Language (UML) models. 3 Contents 1 Introduction1 1.1 Background....................................1 1.2 Vision of this work................................2 1.3 Contribution and novelty of this work.....................2 1.4 Motivation of using water-level measurement system.............2 1.5 Research aim and research questions......................3 1.5.1 Research aim...............................3 1.5.2 Research question (RQ).........................3 1.6 Scope and limitations..............................3 1.6.1 Research limitations...........................3 2 Theoretical background4 2.1 Automatic testing on software systems.....................4 2.2 Model-Based Testing...............................4 2.3 UML (Unified Modeling Language).......................5 2.4 Model-based testing test criteria........................5 2.5 Model-based testing tools............................6 3 Related work7 3.1 Experiences of System-Level Model-Based GUI Testing of an Android Ap- plication......................................7 3.2 Model Based Testing of System Requirements using UML Use Case Models7 3.3 An Evaluation of Model-Based Testing in Embedded Applications......7 3.4 Water-level measurement system (WLM)....................8 4 Method 10 4.1 Nunamaker and Chen’s methodology...................... 10 4.1.1 Construction of a Conceptual Framework............... 10 4.1.2 Development of a System Architecture................. 11 4.1.3 Analyze and Design the System..................... 11 4.1.4 Build the System............................. 11 4.1.5 Observe and Evaluate the System.................... 11 5 Implementation 12 5.1 Problem breakdown............................... 12 5.2 MBT tool literature review........................... 13 5.3 System Architecture............................... 15 5.3.1 System Architecture of whole process.................. 15 5.4 Model-Based Testing............................... 16 5.4.1 UML model................................ 16 5.4.2 Papyrus.................................. 18 5.4.3 UML model in Papyrus......................... 20 5.4.4 Model based testing tool - MoMuT::UML............... 22 4 5.5 Perform Model-Based Testing.......................... 23 5.5.1 Configuration of MoMut - tool..................... 24 5.5.2 Running the MoMut - tool with the AlarmSystem demo....... 26 5.5.3 Running the MoMut - tool with the water-level measurement system 29 5.6 Model-based Test suite generation....................... 33 6 Results & Analysis 36 6.1 Analysis of Model-based testing......................... 36 6.1.1 Challenges with model-based testing tool: MoMut:UML....... 36 6.1.2 Analysis of UML model......................... 37 6.1.3 Analysis of MBMT Test-suite generation................ 37 6.1.4 Drawbacks of Model-based testing................... 38 6.2 Analyse of the chosen command-line option for MoMut............ 38 6.3 Benefits of MoMut::UML for Model-based testing............... 39 7 Threats to Validity 39 8 Discussion & Conclusion 40 8.1 Awnsering the research questions........................ 40 8.2 Future work.................................... 41 8.3 Conclusion..................................... 41 References 42 9 Appendix A 45 10 Appendix B 46 5 6 1 Introduction This section presents the background of this work followed by the vision, the research aim and the research questions. The vision includes the contribution, and the novelty of the work. Moreover, the scope and limitations are also presented. 1.1 Background Testing is the evaluation process to improve the quality of a product [1]. During the execution of testing a software system, a failure can be detected. The failures are caused by faults in the software system. The faults are usually caused by human-error in design, code etc. When a failure is detected, the fault can be investigated. There are different types of software testing methods, techniques, and test data genera- tion. The different types of testing differ depending on their scale, characteristics and the information needed to design the tests. Examples of testing are; component testing, inte- gration testing, system testing, functional testing, robustness testing, performance testing, usability testing, black-box, white-box testing, and model-based testing which is a form of black-box testing [1]. The model-based testing generates functional tests and can be used for robustness testing. Figure1, illustrates a graph of how the different types of testing are divided into three axis. Every axis represent an aspect in testing, the first axis represents the scale of the system under test, SUT, the second one represents which characteristics to test, and the third axis shows the type of information to test. The figure also shows how the model-based testing is related to these different axis. Figure 1: Different types of testing One of the fundamental factors in software testing is the automatic test case generation. Automatic Test case generation for software testing indicates a process that generates test cases automatically. The automatic test case generation technique is convincing as it re- duces the cost and time, and increase the system efficiency, when it is compared to other 1 test generation processes [2]. The model-based testing (MBT) technique, is one of the popular searched phrases in con- nection with automatic test case generation. Model-based testing is built with the model of a system, in order to generate test cases of the model of a system [3]. There are multiple types of models that can be used in MBT. However, the most practical and widely used model for supporting the MBT technique is the unified modeling language, UML[4]. MBT technique is used in tools that are able to generate test cases by analyzing the behavior of the UML model. Model-based testing have different test criteria, and one of these criteria is used in this work and mentioned in section 2.4 [1]. The SUT used in this work is a water level measurement system [5]. The system consists of different technical parts: embedded, dashboard as a frontend, and a network part. The MBT is performed on the embedded part of the system. The motivation for selecting the embedded system part is mentioned in section 1.4. 1.2 Vision of this work The vision of this work is to provide a proof-of-concept of MBT. For this aim MBT is performed on an embedded system of water-level measurement system which is presented in section 3.4. Subsequently, a guide on how to implement and perform MBT is provided based on the experience developed while performing MBT. 1.3 Contribution and novelty of this work The contribution of the work is to provide the IT industry a base to start adopting MBT by creating an easy-to-follow guide on how to implement MBT. The guide indicates a clear explanation on how to test an embedded system using a specific MBT tool. In this way, a proof-of-concept of the MBT is established to illuminate the privileges of MBT approach. 1.4 Motivation of using water-level measurement system The motivation for using the water-level measurement system, WLM, as a test material for this work is because the system is already manually tested [5]. The automatically generated test cases are conducted by using a MBT tool. The reason behind selecting the embedded system part and not the other associated parts; the front-end part is presented as an open source dashboard, and the interface of the dashboard is not reachable for testing. The network part of the water-level measurement i.e. LoRaWAN network is also unreachable for executing test cases through MBT. Although, the embedded part of the system works as a state-machine and therefor is reasonable for MBT to test. 2 1.5 Research aim and research questions 1.5.1 Research aim The aim of this work is to provide a guide that explains how to generate test cases along with a model-based testing tool. To clarify, this work is a proof-of-concept of model-based testing implemented on a embedded system with test cases generated based on the UML model.