Unifying Model and Code Verification

. Most controls applications are a combination of model-based generated code and hand code

. How do I efficiently verify this mix of hand code and generated code?

. MathWorks has tools for verifying models and tools for verifying code

. Is there a workflow for me to use these tools in a complementary, optimum way?

2 Case Study: Cruise Control Application

Objective: set cruise control target speed and pedal position based on driver & vehicle inputs

Cruise Control Application (C code) • Hand code components • Model-based Stateflow component • Model-based S-function component

65 mph 3 Case Study: Cruise Control Architecture Hand Code

MBD Gen Code

S-function Code Cruise Control Application

Read Inputs System Inputs Cruise Power Fault Logging System Outputs Brake Target Speed Function Target Speed Vehicle Speed Engaged Scheduler Control Module Coast/Set Pedal Position Pedal Command Accel/Resume Control Module

Write Outputs

4 Workflow for integrated Model-based and Hand Code Components

6 Workflow for integrated Model-based and Hand Code Components Read Inputs

Fault Logging Hand Code Write Outputs

Cruise Control Application S-function Code Pedal Command Control Module

Target Speed Control Module MBD Generated Code

7 MISRA-C – Modeling Guidelines Checks

• Check MISRA compliance (Mdl Advisor)

8 Target Speed Checking Model for MISRA compliance with Model Advisor Control Module

9 Target Speed Checking Model for MISRA compliance with Model Advisor Control Module

10 Target Speed Checking Model for MISRA compliance with Model Advisor Control Module

 Checks model design and code configuration settings  Increases likelihood of generating MISRA C:2012 compliant code 11 Design Error Detection

• Check MISRA compliance (Mdl Advisor) • Check model early for design errors

12 Floats to Integers: Checking Target Speed Control Module the Model for Design Errors

. Simulink Design Verifier identifies design errors on the model . These “dead logic” errors would prevent successful functional testing

13 Root Cause Analysis/Fix of Dead Logic Target Speed Control Module

Design Error Detection reveals this condition can never be false!

. Dead logic due to “uint8” operation on incdec/holdrate*10

. Fix change the order of operation 10*incdec/holdrate

14 Simulation and Test

• Functional testing (simulation)

15 Functional Testing Workflow

Requirements Did we meet our Review functional requirements? behavior

Tests

Design Did we completely Structural coverage test our model? report

16 Simulink Test™

. Authoring Tests . Managing Tests . Execution of Tests

. Key Features – Test harness – Test sequence block for running tests and assessments – Pass-fail criteria, including tolerances, limits, and temporal conditions – Baseline, equivalence, and back-to-back testing – Automatic report generation to document test outcomes

17 Functional Testing of Pedal Command (S-Function) Pedal Command Control Module

. Coverage analysis for the model and the s-function code.

18 Back-to-Back Cmparison Test (ISO 26262)

• SIL mode support • Code coverage (SIL)

19 Check the Generated Code for Equivalent Model Behavior

. Integrated SIL mode support for model-to-code equivalence testing . Coverage report for generated code for a detailed equivalence analysis

20 Optional: Automatic Code Reviews

• Automatic Code Reviews

21 Automatic Code Reviews

Simulink Embedded C source EC Model Coder code Model and code development .

Model IR Code IR IR transformations

Simulink Code Inspector ® Normalized Normalized Model IR Code IR Independent code inspection . SLCI

. Matching

. Static verification tool.

. Checks the generated code against model Code  Traceability Inspection ? report Report  22 Simulink Design Verifier – Automatic Test Generation

• Functional testing • Code coverage (SIL) (simulation)

23 Generate Functional Test Cases Pedal Command Control Module

SL Design Verifier created test cases to cover all possible “Decision/Condition” objectives in the model and s-function code. 24 Simulink Design Verifier – Test Generation for Modules Overview

. Input . Results – Model – Harness model – Input test signals – Coverage metric Test Generation – Unreachable objects – Decision coverage – Condition coverage – Detailed reports – MC/DC – Custom Objectives (requirements-based)

. Optional Input – Modifiable parameter sets – Existing coverage data

25 Simulink Design Verifier – Property Proving Overview

. Proof satisfied or . Proof falsified . Inputs . Counterexample

Property Proving

. Model . Requirements

. Optional Inputs . Assumptions . Detailed reports

26 Simulink Design Verifier – Use Cases

Model Slicing Test Generation

Design Error Detection Property Proving

27 Issues Found on HIL Bench…

. The Cruise Control powered off during fault testing

. And, the Target Speed never exceeded 40 mph

28 Polyspace product family for C/C++

. Polyspace Code Prover – Proves code to be safe and dependable – Deep verification of software components – Perform QA signoff for production ready code

. Polyspace Bug Finder – Quickly find bugs in embedded software – Check code compliance for MISRA and JSF – Intended for every day use by software engineers

Ada language also supported for proving code 29 Customer Lookup Table: Checking the S-Function Code Pedal Command for Runtime Errors Control Module

30 Root Cause Analysis/Fix of S-Function Run-time Errors Pedal Command Control Module

31 Configuring Polyspace from the Model Target Speed Control Module

32 Launching Polyspace from the Model Target Speed Control Module

33 Review Bug Finder MISRA results Target Speed Control Module

34 Reduce MISRA violations with “Code Placement” setting Target Speed Control Module

35 Justify other violations by adding annotation Target Speed Control Module

36 Read Inputs

Hand Fault Logging • Check integrated code for run-time errors Code

Write Outputs Cruise Control Application

What support is S-function there for analyzing Code the integrated code? Pedal Command Control Module

Target Speed Control Module . Check integrated code MBD for run-time errors Generated – Polyspace Code

37 Why code verification? Code verification is exhaustive!

Statically and semantically verifies all possible executions of your code (considering all possible inputs, paths, variable values)

. Proves when code will not fail under any runtime conditions, reducing robustness testing efforts

. Finds unreachable code, runtime errors, boundary conditions and without testing

. Gives insight into runtime behavior and data ranges

38 Creating a Code Prover project to check the Integrated Code

. Read Inputs Target Speed . Write Outputs Control Module . Scheduler Pedal Command . Fault Logging Control Module

39 Code Integration Check with Polyspace: Non-terminating loop in Hand Code Fault Logging

40 Cause of Cruise Control Powering off during fault testing Fault Logging

41 Code Integration Check with Polyspace: Target Speed Dead Code Found in Generated Code Control Module

Vehicle speed signal propagated to Maximum target speed = 90 “CruiseControl_PS.c” [0 … 40]

Unreachable/Dead code

42 Root Cause for Dead Code: Read Inputs Speed Sensor Input Hand Code Changing analog-to-digital converter from 14 to 12-bit results in dead code

MASK – accounts for scaling down for new ADC from 14-bit to 12-bit Overlooked changing CONV_FACTOR for new ADC

43 Workflow Summary: Complementary Model & Code Verification • Check integrated code Hand for run-time errors Code

• Check model early for design errors • Check MISRA compliance (Mdl Advisor) • SIL mode support • Functional testing • Code coverage (SIL) (simulation)

• Check s-function code for run-time errors • Check MISRA compliance (Polyspace)

MBD Generated Code

45 ISO 26262 Structure

ISO 26262-1 • Vocabulary

ISO 26262-2 • Management of functional safety

ISO 26262-3 • Concept phase

ISO 26262-4 • Product development: system level

ISO 26262-5 • Product development: hardware level . Model-Based Design ISO 26262-6 • Product development: software level . Early verification and validation . Code generation ISO 26262-7 • Production and operation

ISO 26262-8 • Supporting processes . Tool classification and qualification ISO 26262-9 • ASIL-oriented and safety-oriented analyses

ISO 26262-10 • Guideline

46 ISO 26262 Tool Qualification Approach - Details

I. Tool Classification II. Tool Qualification

Tool Tool Tool Tool use error confidence impact

cases detection level ASIL

requirements qualification qualification Qualification Increasing TD 3 TCL 3 methods for TCL3 TI 2 Medium TD 2 UC Qualification TCL 2 1..n TD 1 High methods for TCL2

TI 1 TCL 1 Qualification not required

47 Qualification of MathWorks Tools Tool qualification may involve multiple parties

. Tool user MathWorks . Responsible for final tool qualification Pre-classification / in the context of the application qualification based on typical use cases (reference workflows) . Tool vendor TÜV SÜD . Conducts generic pre-classification and Independent Assessment pre- qualification based on reference use cases / reference workflow

ISO 26262 Tool Qualification Kit

. 3rd party assessor (optional) . Provides independent assessment of reference workflow and pre-qualification Tool User artifacts Project-specific adaptation

48 Independent Assessment by TÜV SÜD Example

Certificate Assessment Report

49 A Complementary Model and Code Verification Process …

 Model and code checks before functional testing to minimize rework

 Perform functional, dynamic testing with model and code structural analysis with automation, and reuse of test assets

 Analyze the code to find issues resulting from the integration of o hand code o s-function code o model-based generated code

 Includes formal methods analysis to go beyond functional testing

 Enables more, early testing of the model and code

 Continual increase in design confidence

50 Thank You!

51 Process Compliance Demonstration

Annotated method tables with suggestions on how to use Model-Based Design processes and tools to apply the methods listed in ISO 26262-6

52 ISO 26262 – Walk-through and Inspection

„[..] unit specification design and implementation can be verified at the model level.“

ISO 26262 (2011)

53 Qualification of Model-Based Design Tools

 pre-qualified for all ASILs according to ISO 26262

Simulink Verification and Validation  Simulink Design Verifier  Simulink Test 

Polyspace Bug Finder  Polyspace Code Prover 

Embedded Coder  54 Defects detected by Polyspace Bug Finder

Numerical . Integer division by zero Dynamic memory Dataflow Security / tainted data (38 checkers) . . Float division by zero . Use of previously freed pointer Write without further read . chroot misuse . . Integer conversion overflow . Unprotected dynamic memory allocation Non-initialized variable . Use of dangerous standard function . . Unsigned integer conversion overflow . Release of previously deallocated pointer Non-initialized pointer . Mismatch between data length and size . . Sign change integer conversion . Invalid free of pointer Variable shadowing . Use of non-secure temporary file overflow . Memory leak . Missing or invalid return statement . Sensitive heap not cleared . Float conversion overflow . Dead code . Tainted array index . Integer overflow Programming . Partially access array . Tainted external command . Unsigned integer overflow . Invalid use of = operator . Uncalled function . Tainted integer division . Float overflow . Invalid use of == operator . Pointer to non initialized value converted to . Invalid use of std. library integer routine . Declaration mismatch const pointer . Code deactivated by constant false . Invalid use of std. library float routine . Invalid use of floating point operation Object oriented (12 checkers) condition . Shift of a negative value . Wrap-around on unsigned integer . Object slicing . Unreachable code . Shift operation overflow . Assertion . Copy constructor not called . Useless if . Invalid use of other standard lib. routine . Member not initialized in constructor Static memory . Missing null in string array . … Concurrency . Array access out of bounds . Qualifier removed in conversion . Race condition (atomic / non atomic) . Null pointer . Wrong type used in sizeof . Missing lock/unlock . Pointer access out of bounds . Format string specifiers and arguments . Deadlock . Unreliable cast of function pointer mismatch . Double lock/unlock . Unreliable cast of pointer . Writing to const qualified object . Invalid use of std. library memory routine Good practice Resources management . . Invalid use of standard library string . Large pass-by-value argument Use of previously closed resource routine . More than one statement . Resource leak . Arithmetic operation with NULL pointer . Delete of void pointer . Writing to read-only resource . Wrong allocated object size for cast . Hard-coded buffer size / loop boundary . Local address escape . Unused parameter . Buffer overflow from incorrect string format specifier

55 Defects detected by Polyspace Bug Finder

Numerical . Integer division by zero Dynamic memory Dataflow Security / tainted data (38 checkers) . . Float division by zero . Use of previously freed pointer Write without further read . chroot misuse . . Integer conversion overflow . Unprotected dynamic memory allocation Non-initialized variable . Use of dangerous standard function . . Unsigned integer conversion overflow . Release of previously deallocated pointer Non-initialized pointer . Mismatch between data length and size . . Sign change integer conversion . Invalid free of pointer Variable shadowing . Use of non-secure temporary file overflow . Memory leak . Missing or invalid return statement . Sensitive heap not cleared . Float conversion overflow . Dead code . Tainted array index . Integer overflow Programming . Partially access array . Tainted external command . Unsigned integer overflow . Invalid use of = operator . Uncalled function . Tainted integer division . Float overflow . Invalid use of == operator . Pointer to non initialized value converted to . Invalid use of std. library integer routine . Declaration mismatch const pointer . Code deactivated by constant false . Invalid use of std. library float routine . Invalid use of floating point operation Object oriented (12 checkers) condition . Shift of a negative value . Wrap-around on unsigned integer . Object slicing . Unreachable code . Shift operation overflow . Assertion . Copy constructor not called . Useless if . Invalid use of other standard lib. routine . Member not initialized in constructor Static memory . Missing null in string array . … Concurrency . Array access out of bounds . Qualifier removed in conversion . Race condition (atomic / non atomic) . Null pointer . Wrong type used in sizeof . Missing lock/unlock . Pointer access out of bounds . Format string specifiers and arguments . Deadlock . Unreliable cast of function pointer mismatch . Double lock/unlock . Unreliable cast of pointer . Writing to const qualified object . Invalid use of std. library memory routine Good practice Resources management . . Invalid use of standard library string . Large pass-by-value argument Use of previously closed resource routine . More than one statement . Resource leak . Arithmetic operation with NULL pointer . Delete of void pointer . Writing to read-only resource . Wrong allocated object size for cast . Hard-coded buffer size / loop boundary Checkers inherited from Code Prover . Local address escape . Unused parameter . Buffer overflow from incorrect string format specifier

56 Full list of run-time error checks in Polyspace Code Prover

static void pointer_arithmetic (void) { int array[100]; Green: reliable int *p = array; safe pointer access int i; C++ run-time checks C run-time checks for (i = 0; i < 100; i++) { . Unreachable Code *p = 0; . Unreachable Code p++; . Out of Bounds Array Index . Out of Bounds Array Index } Red: faulty . Division by Zero out of bounds error if (get_bus_status() > 0) { . Division by Zero if (get_oil_pressure() > 0) { . Non-Initialized Variable *p = 5; . Non-Initialized Variable } else { . Scalar and Float Overflow . Scalar and Float Overflow (left shift on signed variables, float underflow i++; . Shift Operations Gray: dead } versus values near zero) unreachable code } . Pointer of function Not Null . Initialized Return Value i = get_bus_status(); . Function Returns a Value . Shift Operations (shift amount in 0..31/0..63, left operand of left shift is if (i >= 0) { . Illegal Dereferenced Pointer *(p - i) = 10; negative) Orange: unproven } . Correctness Condition may be unsafe for some } . Illegal Dereferenced Pointer (illegal pointer access to variable of . Non-Initialized Pointer conditions structure field, pointer within bounds) . Exception Handling (calls to throws, destructor or delete . Correctness Condition (array conversion must not extend range, throws, main/tasks/C_lib_func throws, exception raised is function pointer does not point to a valid function) not specified in the throw list, throw during catch . Non-Initialized Pointer parameter construction, continue execution in__except) . User Assertion . User Assertion . Object Oriented Programming (invalid pointer to member, . Non-Termination of Call (non-termination of calls and loops, arithmetic expressions) call of pure virtual function, incorrect type for this-pointer) . Non-Termination of Call . Known Non-Termination of Call . Non Termination of Loop . Non-Termination of Loop . Absolute Address . Standard Library Function Call . Potential Call . Absolute Address . C++ Specific Checks (positive array size, incorrect typeid . Inspection Points argument, incorrect dynamic_cast on reference)

57 Examples of software safety requirements

Initiation Architecture Design/Implementation Unit Testing Integration Testing

Detect and correct areas with high complexity

Table 1 – Topics to be covered by modelling and coding guidelines

Polyspace Metrics 58 Examples of software safety requirements

Initiation Architecture Design/Implementation Unit Testing Integration Testing

Enforcement of coding guidelines

Polyspace Metrics

Table 1 – Topics to be covered by modelling and coding guidelines Polyspace Metrics

59 Reference Workflow Model-Based Design for ISO 26262

PIL test (Embedded Coder) Traceability matrix analysis Modeling standards checking (Simulink V&V) (IEC Certification Kit) or model vs. code coverage (Simulink V&V)

Simulation / model testing (Simulink, Simulink Test) Model coverage Req. Mgmt. Int. (Simulink V&V)

Simulink/Stateflow Embedded Coder Third-party tool 65 Advanced Reference Workflow Additional Best Practices PIL test (Embedded Coder) Test generation Traceability matrix analysis Modeling standards checking (Simulink V&V) (Simulink Design Verifier) (IEC Certification Kit) or model vs. code coverage (Simulink V&V) Property Proving, Design Error Detection Run-time error detection (Simulink Design Verifier) MISRA-C checking (Polyspace products) (Polyspace products) Simulation / model testing (Simulink, Simulink Test) Model coverage Req. Mgmt. Int. (Simulink V&V)

Simulink/Stateflow Embedded Coder Third-party tool 66 Workflow Summary: Complementary Model & Code Verification • Check integrated code for run-time errors

• Check s-function code for run-time errors • Check MISRA compliance (Polyspace)

67 Model-Based Design Verification and Validation

Requirements Traceability • Navigation and Reports

REQUIREMENTS Modeling Standards Checking User Acceptance Testing • Automatic Review

Complete DESIGN Testing and Test Automation • Test Authoring Integration Environment Models & Test • Coverage Analysis Integration Testing Physical Components • Automatic Test Case Generation

Algorithms Property Proving System-Level System-Level • Quickly find incorrect behavior Integration & Test Specification • Prove correctness of behavior Component Design Code Verification Subsystem Static Analysis on Model and Code Research and Validation Design • Identify defects Data analysis • Prove absence of runtime errorsSubsystem Algorithm • Compliance to coding standardsIntegration & Test Development IMPLEMENTATION Data Modeling Compliance to ISO 26262 VHDL, Structured C, C++ • WorkflowVerilog andText tool qualification

MCU DSP FPGA ASIC PLC Subsystem Implementation 68 Verification on Code Level – Polyspace

. Finds bugs

. Checks coding rule conformance (MISRA/JSF/Custom)

. Provides metrics (Cyclomatic complexity etc.)

. Proves the existence and absence of errors

. Indicates when you’ve reached the desired quality level