2008 International Conference on Software Testing, Verification, and Validation Model-based Testing of Automotive Systems Eckard Bringmann, Andreas Krämer PikeTec GmbH, Germany [email protected], [email protected] Abstract systems within the next decade. Consequently the impact of software on the customer and hence on In recent years the development of automotive market shares and competition will be enormous. This embedded devices has changed from an electrical and establishes software as a key technology in the mechanical engineering discipline to a combination of automotive domain. software and electrical/mechanical engineering. The During recent years the growth of software in the effects of this change on development processes, automotive industry led to a development process methods, and tools as well as on required engineering founded on model based technologies which have skills were very significant and are still ongoing today. many advantages for automotive developments. Firstly At present there is a new trend in the automotive model-based technologies such as MATLAB/Simulink, industry towards model-based development. Software Statemate, MatrixX, or LabView are domain specific components are no longer handwritten in C or and support powerful mechanisms for handling and Assembler code but modeled with processing continuous signals and events which are the MATLAB/Simulink™, Statemate, or similar tools. most important data types in the automotive domain. However, quality assurance of model-based These model-based technologies allow the developments, especially testing, is still poorly development of high-level models that can be used for supported. Many development projects require simulation in very early stages of the development creation of expensive proprietary testing solutions. process. This in turn is important since automotive In this paper we discuss the characteristics of development is an interdisciplinary business with automotive model-based development processes, the software, electrical, and mechanical engineering consequences for test development and the need to aspects inextricably entwined. Graphical models and reconsider testing procedures in practice. simulation of such models allows engineers to find a Furthermore, we introduce the test tool “TPT” which common functional understanding early in the design masters the complexity of model-based testing in the phase. So, model-based development improves automotive domain. To illustrate this statement we communication within development teams, with present a small automotive case study. customers, or between car manufacturers and suppliers. TPT is based on graphical test models that are not It reduces time to market through component re-use only easy to understand but also powerful enough to and reduces costs by validating systems and software express very complex, fully automated closed loop tests designs up front prior to implementation. Consequently in real-time. TPT has been initially developed by models are often treated as part of the requirements Daimler Software Technology Research. It is already specification, since the models illustrate the required in use in many production-vehicle development functionality in an executable manner. Model-based projects at car manufacturers and suppliers. development provides a development process from requirements to code, ensuring that the implemented 1. Motivation systems are complete and behave as expected. Model- based development allows segregation of concerns; Within only a few years the share of software technical aspects such as fixed-point scaling (i.e. the controlled innovations in the automotive industry has transformation of floating-point algorithms to fixed- increased from 20 percent to 80 percent, and is still point computations), calibration data management, and growing. Forecasts claim that software will determine the representation of signals in memory are separated more than 90% of the functionality of automotive from the core algorithms thereby keeping the models as lean as possible. 0-7695-3127-X/08978-0-7695-3127-4/08 $25.00 $25.00 © 2008 © 2008 IEEE IEEE 483485 DOI 10.1109/ICST.2008.45 2. Model-based testing in practice only a piece of software, but a complex construction that consists of software, hardware, electrical, Along with the growing functionality and the mechanical, and/or hydraulic parts. The development introduction of model-based development processes, of such a system requires the co-design of software and the demands on quality assurance have also increased. hardware components. Consequently an iterative In terms of testing, model-based development enables process is needed with a considerable number of system engineers to test the system in a virtual interim releases of the integrated system. Thoroughly environment when they are inexpensive to fix, i.e. testing these interim releases is crucial to ensure that before the code is actually implemented or integrated requirements inconsistencies and design or on the final hardware (which is called ECU or implementation faults can be uncovered as early as electronic control unit). However, in practice there are possible. This means that the same tests have to be just a few testing procedures that address the repeated again and again over the development cycle. automotive domain-specific requirements of model- Test automation is therefore essential as the manual test based testing sufficiently. The applied test methods and workload over many iterations would be at best tools are often proprietary, ineffective and require expensive and likely not practical, leading to less than significant effort and money. About 15 years ago thorough quality assurance standards. testing embedded devices comprised primarily of four Test automation also simplifies the coordination well-understood areas: (1) electromagnetic between car manufacturers and suppliers during compatibility (EMC tests), (2) electrical tests (e.g. development. Every ECU sample delivered by the short-circuit, creeping current, stress peaks), (3) supplier during development must fulfill at least a environmental tests (i.e. testing under extreme climate predefined acceptance test before being integrated in a conditions), and (4) field tests (on proving ground or new car setup. Finally, testing automotive systems the road). There was no need for dedicated functional often requires test scenarios with a very precise testing methods because the functional complexity was sequence of time-sensitive actions, especially for comparatively low. With the increasing popularity of power-train and chassis systems (in the range of µ-sec). model-based development the engineering discipline of The only way to execute such scenarios is automation. automotive model-based testing has been neglected for a long time. With the promise of model-based 3.2 Portability between integration levels development to prove concepts at early development stages by means of executable models it was assumed One major benefit of model-based development is that testing those models and the derived code is less the possibility to “do things as early as possible”. In important and therefore would not require new terms of testing this means to test the functionality in techniques. the model before the software is implemented and integrated into the final ECU. Between the initial modeling and the integration into the ECU there are 3. Requirements for automotive MBT intermediate integration levels described below. Since the functionality of the system should remain constant The term ‘model-based testing’ is widely used today and independent of the integration level, relevant test with slightly different meanings. In the automotive cases should also be constant throughout the industry it is normally used to describe all testing integration and implementation. activities in the context of model-based development In order to maximize reuse, a test procedure for projects. model-based developments should support the Automotive model-based development has specific portability or reuse of test cases between the various characteristics that require the use of dedicated model- platforms. On the one hand this reduces the effort of based testing approaches. The characteristics of, and test case design tremendously and, on the other hand, the resulting requirements for model-based testing are allows for easy comparison of test results between the discussed below. different integration levels which may be tested on different systems. In addition sharing test cases 3.1 Test automation between different levels also means that test cases can be expressed in a common notation. Test cases can be corrected or extended in a central test model without Automotive systems usually interact with a real- the need to adjust a lot of different test world environment which is under continual control of implementations for the different integration levels. the embedded system. Thus the whole system is not 484486 Although this requirement sounds trivial from a in a cheap and manageable way. Therefore the effort theoretical point of view it is a weak point in today’s spent by PiL testing is worthwhile in almost all cases. testing practice for model-based development because Hardware-in-the-Loop (HiL): When testing the test procedures and test languages are usually embedded system on HiL level the software runs on the specialized for one particular test platform and are very final ECU. However the environment around the ECU