System Testing in a Simulated Environment
Total Page:16
File Type:pdf, Size:1020Kb
School of Innovation, Design and Engineering CDT504 – Computer Science, Advanced Level for 2years Master Degree (30 hp) System Testing in a Simulated Environment By Manuel Palmieri [email protected] CrossControl Advisor: Anders Öberg Mälardalen University Advisor: Antonio Cicchetti Mälardalen University Examinator: Paul Pettersson University of L’Aquila Advisor: Henry Muccini April 2013 I dedicate this thesis to my parents Elio and Rossana, and my sister Ilaria for their love, endless support and encouragement. ii Abstract System Testing in a Simulated Environment is becoming one of the bigger challenges moved by companies with the purpose to improve the quality and increase the dependability of artefacts. Moreover, another big challenge faced by them is the achievement of the independent development of software and hardware systems, with the purpose to speed up and reduce costs of development. Besides, software testing remains today a tricky activity that attracts the interest of many researchers and companies with the purpose to refine or invent new techniques and methods to perform the optimal testing on different case studies and conditions. Nowadays, the execution of testing on some real environments is not adequate because of the impossibility to perform particular kinds of tests that could be destructive for hardware and dangerous for testers. One of the areas that are affected by this issue is certainly the one regarding the production of vehicles that work in critical environments. In this respect, companies in such applicative domain trying to move towards the revolutionary approach to replace real environments with simulated environments. This thesis presents a survey of existing state–of–the–art and state–of–the–practice testing techniques and methods, a simulated environment derived from a vehicle, and how and in what way it is possible to perform testing on a simulated environment. Furthermore, with the purpose to provide a better understanding of how a real environment could be represented with a simulated environment, an overview of replacement is presented. Besides, according to CrossControl and customers’ environments, limits and needs, a case study has been realized to demonstrate how is possible to design, implement and test a simulated environment. iii Acknowledgements I would like to thank my advisors Antonio Cicchetti from Mälardalen University and Anders Öberg from CrossControl AB for their continuous help and huge support in enhancing my capability in system testing area and for making this thesis. Furthermore, sincere thanks go to Rikard Land and Daniel Boström from CrossControl AB for their contribution in achieving the desired goal of the proposal thesis. Besides, I would like to acknowledge Henry Muccini from University of L’Aquila and for giving me the opportunity to achieve my goals by taking part in Global Software Engineering European Master (GSEEM) and Magnus Nolen from CrossControl AB for giving me the possibility to carry out the thesis in CrossControl. Finally, I wish to express my gratitude to my colleagues and friends for their moral support during the entire academic experience. iv Nomenclature The Nomenclature includes abbreviations and terms found in the thesis that are common in Software Testing, and are often used or are not defined. NAME DESCRIPTION 3G 3rd Generation 3GPP 3rd Generation Partnership Program ASN.1 Abstract Syntax Notation One ATM Automated Teller Machine AUTOSAR AUTomotive Open System ARchitecture AWD All-Wheel Drive BSD Berkeley Software Distribution BTT BroadBit Test Tool CAN Controller Area Network CCM CORBA Component Model CD Codec or Coding and Decoding CH Component Handling CORBA Common Object Request Broker Architecture CPL Common Public License CPU Central Processing Unit DSL Digital Subscriber Line ECU Electrical Control Unit EJB Enterprise JavaBeans ETSI European Telecommunications Standards Institute FIT Framework for Integrated Test GUI Graphical User Interface GSM Global System for Mobile Communications HIL Hardware–In–The–Loop HW Hardware I/O or IO Input/output ID Identifier IDE Integrated Development Environment IDL Interface Definition Language IMS Internet Protocol Multimedia Subsystem IPv6 Internet Protocol version 6 KMH Kilometres per Hour LIN Local Interconnect Network LTE Long Term Evolution MC/DC Modified Condition/Decision Coverage v MOST Media Oriented Systems Transport MS Microsoft NGN Next Generation Network OMA Open Mobile Alliance OSI Open Systems Interconnection PA Platform Adapter PC Personal Computer PDP Product Development Process PTO Power Take–Off RAM Random Access Memory ROI Return Of Investment RPDE Runtime Plugin Development Environment RPM Revolution Per Minutes SA System Adapter SAP Systems, Applications and Products SEP System Engineering Process SIP Session Initiation Protocol SIGTRAN Signaling Transport SUT System Under Test SWT Standard Widget Toolkit TC Test Case TCI TTCN–3 Control Interfaces TDD Test–Driven Development TE Text Executable TEM Text Executor Manager TETRA Terrestrial Trunked Radio TL Test Logging TM Test Management TRI TTCN–3 Runtime Interface TS Test System TSE Test Script Editor TSU Test System User TTCN Testing and Test Control Notation TTCN–3 Testing and Test Control Notation – version 3 UI User Interface UML Unified Modelling Language VNC Virtual Network Computing WiMAX Worldwide Interoperability for Microwave Access XML Extensible Markup Language vi Table of Contents Abstract .............................................................................................. iii Acknowledgements ............................................................................ iv Nomenclature ..................................................................................... v Table of Contents .............................................................................. vii 1 Introduction .................................................................................. 1 2 Survey of Testing–Related Literature ............................................. 3 2.1 System Engineering Process ............................................................ 4 2.1.1 Process Input.......................................................................................... 5 2.1.2 Process Development ............................................................................ 6 2.1.3 Process Output....................................................................................... 9 2.2 System Development Model ........................................................... 9 2.3 Product Development Process ...................................................... 11 2.3.1 Phases .................................................................................................. 12 2.3.2 Tollgates ............................................................................................... 13 2.3.3 Disciplines ............................................................................................ 13 2.3.4 Artefacts ............................................................................................... 15 2.4 Testing Techniques ....................................................................... 15 2.4.1 Black–Box Testing ................................................................................ 17 2.4.2 White–Box Testing ............................................................................... 18 2.5 Testing Methods ........................................................................... 19 2.5.1 Unit Testing .......................................................................................... 21 2.5.2 Integration Testing ............................................................................... 24 2.5.3 System Testing ..................................................................................... 25 2.5.4 Hardware–In–the–Loop Testing .......................................................... 27 2.5.5 Fault Injection Testing .......................................................................... 30 2.6 Automated Software Testing ........................................................ 34 2.6.1 Code–Driven Testing ............................................................................ 35 2.6.2 Graphical User Interface Testing ......................................................... 35 2.7 Test Coverage ............................................................................... 36 2.7.1 Code Coverage ..................................................................................... 37 3 Survey of Existing Test Notations and Tools ................................. 41 vii 3.1 General Notations ......................................................................... 41 3.1.1 Testing Tools ........................................................................................ 42 3.2 TTCN–3 Notation........................................................................... 44 3.2.1 Application Fields ................................................................................. 45 3.2.2 Architecture ......................................................................................... 46 3.2.3 Core Language ..................................................................................... 51 3.2.4 Testing Tools ........................................................................................ 54 4 CCSimTech Simulated Environment ............................................. 58 4.1 Basic Concepts and Architecture ..................................................