TPT – Automated Embedded Testing

Automated Testing Control Systems Systematic Testing Continuous Behavior

TPT is a test tool used in the testing of embedded control systems. It supports full automation testing in a real-time context and can use continuous as well as discrete signals. Test cases are modelled graphically. Thanks to the built–in platform independence, TPT can be used universally from Model-test (MiL), through Software-test (SiL) to Hardware-in-the-Loop-Test (HiL).

TPT currently offers test support for MATLAB/Simulink models, ‘C’ code on the host processor, ‘C’ code on debugger boards via Tessy, HiL test systems as well as some proprietary environments.

Time Partition Testing

Model-Based Testing in Automotive Industry with TPT Software

Typical Users of TPT Software

IBES INTELLIGENT BUSINESS & EDUCATION SOLUTION JOINT STOCK COMPANY Add. Room 19A , Indochina Building, No. 4 Nguyen Dinh Chieu Street, Dakao Ward, Dist.1, HCMC, Vietnam Tel: 08 37965969 - Fax: 08 37965969 - Cell No. +84 90 3384610

1. Introduction about MBD Testing for AI with TPT software

Time Partition Testing

- A tool for testing control and feedback control systems

- Testing and verifying in all development phases

- Dynamic testing at all integration levels

- Supported Test Cases in different environments

- Dashboard – Interactive UI's for the system under test

2. Main Automotive applications testing by TPT Software

▪ Body controller projects

▪ Interior and exterior lights, seat heating and ventilation, climate, cluster ▪ Powertrain controller projects ▪ Transmission, HV battery, power electronics, hybrid powertrain, engine ▪ ADAS controller projects ▪ Emergency braking, lane departure warning, blind

spot detection ▪ Autonomous driving ▪ Drive dynamics controller projects ▪ Cross-wind control, roll control

3. TPT Test Automation

TPT automates all steps from execution to reporting

3.1 - Test Modeling Technique

Variant modeling

Testlet Testlet Testlet Decision at runtime

Testlet

Time partitioning Parallel processes

Testlet Testlet Testlet

Loop

Hirarch y

3.2 – Manual Testing a) TPT Dashboard

▪ Design your own graphical interface for your system under test ▪ Observe and interact with tests using the Dashboard ▪ Te s t your system manually with the Dashboard (Simulation) ▪ Automatically record test cases from your interactions

▪ Distribute your simulation environment with the Dashboard Player (separate tool at no charge)

b) Macro recorder Dashboard interaction

▪ Record dashboard interaction as test cases ▪ Generate Step Lists

▪ Create new test cases

▪ Integrate recorded tests

into existing step-lists

▪ Post-process step list

manually if required

3.3 – Test case generation methods in TPT

▪ TASMO for structural coverage tests of Simulink models and C-Code

▪ Combination of Equivalence classes

▪ Path and variant combination of TPT automatons

▪ Generate test cases from value ranges

Test cases generation from Value Ranges

▪ TPT test model has many variants of states and transitions ▪ Combination can be built automatically ▪ Tests can be generated

4. Test Execution : TPT tests in different environments…

…and execute as below process:

4.1 – TPT Test process for Matlab/Simulink

• Support of Simulink with/without Plant Models + Opened Loop Control Testing + Closed Loop Control Testing

• Support of internal signals + Feed model-internal signals back to TPT + No modification of the SUT necessary

4.2 - TPT Test process for ASCET

4.3- TPT Test process for C/C++ Code

4.4- TPT Test for FUSION Co-simulation platform

• FUSION support: + FMI : Functional Mock-up Interface + FMUs for co-simulation with other FUSION nodes

4.5- QTRONIC Silver Adapter

▪ Testing of closed loop virtual ECUs with QTRONIC Silver ▪ TPT Integration as a Silver Node ▪ Read, Write and Read/Write access to signals and parameters

4.6- AUTOSAR Support ▪ Test SW-C using TPT on PC ▪ Support of S/R, C/S , IRV, PIMs, Compositions, explicit and implicit communication ▪ AUTOSAR compatibility mode

4.7- SiL and PiL with Debuggers ▪ Support of testing with debuggers on target HW such as ▪ GDB – GNU debugger on PC ▪ PLS-UDE on TPT +UDE on AURIX, Tricore, ARM Cortex etc. ▪ Lauterbach TRACE 32 on ARM, Power Architecture etc.

4.8- HiL Test Execution a) dSPACE HiL Fusion Adapter (ASAM XIL)

▪ Test of dSPACE HiL / VeOS / SCALEXIO components via ASAM XIL API ▪ Change signals ▪ Error simulation ▪ Application via CANape or INCA ▪ Measurement via HiL, CANape/Inca ▪ Diagnosis via CANape

▪ Extensible for other ASAM XIL-based platforms

b) National Instrument – NI : VeriStand ▪ Test National Instrument HiL system via Test Stand ▪ Real-time via test frontloading ▪ Automatic Custom Device Configuration from System Definition File

c) VECTOR CANoe Integration

▪ Te s t VECTOR CANoe Simulation Setups ▪ Stimulation an Observation of

bus signals and system variables

▪ Test driver integrated into

CANoe via custom library

▪ Incubation feature

d) CAN, INCA + CANape platform environment

▪ CAN stimulation: ▪ Map TPT signals to CAN messages

▪ Changing application parameter

▪ via INCA or CANape

▪ Measure

▪ internal signals in controller ▪ Diagnosis via CANape

e) Driver in The Loop Testing – DiL

▪ Via CAN ▪ Via application tools like INCA ▪ Via on-line driver information ▪ information ▪ Message boxes ▪ Acoustic messages ▪ Step list highlighting11

5.TEST ASSESSMENT

There are 4 main scopes of TPT Test Assessment process

3 4

In this Test Assessment process, the special thing that we are not only using Script to write a test cases (customized) but also make it automated

5.1- Manual evaluation of test result

• Data Viewer for data display • Manual setting of test results + Successful + Failed

• Signal Viewer also available as standalone application • Data formats such as tptbin, mat, mdf, csv

5.2- Back to Back regression testing in Simulink

▪ Two test execution configurations -> e.g. MiL and SiL ▪ Automatic code generation ▪ Run both executions automatically ▪ Compare test results automatically including tolerances

5.3- Step based Assessment in Test Step List

Reporting:

5.4- Standard TPT Assessments in GUI

▪ Min-Max ▪ Signal Comparison ▪ Temporal and Logical Constraints

▪ Interactive Condition-Tree

▪ Mode detection

▪ Analysis is always performed in context of a well-defined time interval using temporal regular expression

▪ Every assessment can be analyzed in multiple time intervals: Programming support under Test Scripting:

during [a(t)>17]: TPT.assertAlways(this.getLength()<10.0,“Check for length“) during [light_switch == LIGHT_ON]: TPT.assertAlways(headlight == true,“Check headlight on if switch on“) during [pedal > 50 and speed < 100]: cond := TPT.monotony(v(t), TPT.INCREASING); TPT.assertAlways(cond,“Check for monotony“)

Function: ▪ Compare := TPT.hose(orig(t), ref(t), xtol, ytol)

▪ VehicleBoundsCheck := TPT.checkAlways(v(t) < 250.0); ▪ AccelerationCheck := TPT.monotony(a(t),TPT.INCREASING); ▪ Never := TPT.never(rpm(t), 30); ▪ rpm_average := TPT.filterMA(rpm(t), 30);

6.TEST REPORTING TPT is fully support to generate test report with high accuracy

6.1- Report for Coverage of Requirement

6.2- Report for Coverage of Model and Code

▪ MATLAB/Simulink ▪ MathWorks V&V toolbox (e.g Simulink Coverage will be used from R2017b) ▪ TargetLink ▪ CTC++ ▪ TASMO ▪ C-Code ▪ CTC++ ▪ GNU gcov

7. SUMMARY TPT features

▪ Modeling ▪ Te s t M a n a g e m e n t ▪ Graphical test models ▪ Te s t R a i l ▪ Test step lists or direct definitions ▪ TestLink ▪ Closed-loop (reactive) tests ▪ IBM ALM ▪ Hard real-time enabled (≤100µs cycles) ▪ Jenkins ▪ Signal import ▪ SVN ▪ Signal editor ▪ Requirements Management ▪ Wizard based signal creation ▪ Doors (native DXL and CSV) ▪ Variant handling / one model for all tests ▪ Excel ▪ Test case generation (combinatory) ▪ Polarion ▪ Libraries ▪ Reqtify ▪ Parameter Support ▪ Hardware ▪ Scalars, arrays, curves, and maps ▪ CAN ▪ Parameter import ▪ CANape ▪ Parameter overloading/calibration ▪ INCA ▪ Online parameter calibration ▪ PLS UDE ▪ Execution ▪ NI-HW ▪ Support of many execution platforms ▪ File I/O (Test Data I/O) (MATLAB, fusion ,…) MDF, MD4, DBC, LDF, A2L, DCM, ARXML, ▪ Multiple test sets and execution configurations HDR, CSV, TPTBIN, EXCEL, MAT ▪ Debugger for analysis ▪ Assessment ▪ Online and Offline ▪ Back-to-back Analysis ▪ Temporal conditions ▪ General constraints and analysis per scenario ▪ Flexible offline concept with scripting language ▪ Assessment Libraries

Application Examples

• BMS integration Testing (or: BMC – Battery Management Controller)

▪ Integration testing as above ▪ Module testing with or without cell models ▪ 4 years of testing, thousands of test cases

• GearBox testing on SiL

▪ Driving maneuvers in different environments ▪ Error simulation ▪ Failure reaction tests

Notes:

TPT – Automated Embedded Testing

Test modelling with TPT Examples

Test cases are modelled graphically in a TPT was originally designed for use in a feature-rich environment. Features such wide variety of development projects as parallel and hierarchical test ow, conditional branching as well as support from initial modelling through to pre- for continuous and discrete signals allow production verification at Mercedes. It is the construction of complex test cases. now used by many companies within the

automotive industry. Typical projects For the testing of complex systems TPT offers a powerful approach for systematic using TPT include engine control, power test-case-generation which guarantees train control, driver assistant systems easy interaction and readability even with and interior systems control. a larger number of tests.

Test evaluation Features of TPT

The automatic test evaluation can be done online (i.e. during the test execution) or - Testing using continuous and discrete signals offline (at the end of the test execution). It can formulate complex quality criteria made - Real-time enabled test execution

up from powerful operations such as - Graphical test modelling comparisons with reference data, signal filters, state sequences and timing - Automated execution, conditions. evaluation, documentation and management of tests

- Compatibility with MATLAB/Simulink- Test Documentation Models, C-Code, HiL systems and others Test execution TPT produces a test report of the test execution which contains the most important Test execution with TPT is fully automatic in information regarding the execution and almost every test environment. The core of results of the test case. The information in the execution is TPT’s virtual machine (VM) the report can be configured such that it which allows tests to execute in real-time, remains a readable summary of the hence making it possible to have cycle times execution even for highly complex tests. in the order of microseconds. The VM requires only a few kilobytes of memory so that it may also be used in environments with limited resources

. Integration into development environments

To integrate TPT in an existing test or development environment, it is sufficient to attach the TPT virtual machine. TPT handles this integration itself, the development project’s toolset does not need to be extended or changed for TPT

Time Partition Testing