Copyright by Jason Eric Trout 2020 The Report Committee for Jason Eric Trout Certifies that this is the approved version of the following Report: Testing Safety-Critical Systems using Model-Based Systems Engineering (MBSE) APPROVED BY SUPERVISING COMMITTEE: Sarfraz Khurshid, Supervisor Philip Terrall Testing Safety-Critical Systems using Model-Based Systems Engineering (MBSE) by Jason Eric Trout Report Presented to the Faculty of the Graduate School of The University of Texas at Austin in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering The University of Texas at Austin May 2020 Dedication To my wife, Katelyn, thank you for your love and support. Acknowledgements I would like to thank Dr. Sarfraz Khurshid for his guidance through many courses at The University of Texas at Austin as well as guidance throughout this report. I would also like to thank Philip Terrall for his mentorship and serving as a reader for this report. v Abstract Testing Safety-Critical Systems using Model-Based Systems Engineering (MBSE) Jason Eric Trout, M.S.E. The University of Texas at Austin, 2020 Supervisor: Sarfraz Khurshid Model-based Systems Engineering (MBSE) provides features for behavioral analysis, requirements traceability, system architecture, simulation, testing, and performance analysis that are imperative for the testing of safety-critical systems. In this report, we present a case study of a simple safety-critical system, and model the system using UML (Unified Modeling Language), SysML (Systems Modeling Language), and AADL (Architecture Analysis and Design Language). We then extend the AADL model with user-defined properties and annexes to augment additional analysis and reporting capabilities relevant to safety-critical systems. As safety and security expectations grow in concert with system complexity, MBSE will become increasingly ingrained in the workflow of the systems and software engineering communities. vi Table of Contents List of Tables ................................................................................................................... viii List of Figures .................................................................................................................... ix Chapter 1: Introduction .......................................................................................................1 Chapter 2: Case Study: Specifications and Initial UML Model .........................................5 Chapter 3: Case Study: SysML Model .............................................................................14 Chapter 4: Case Study: AADL Model ..............................................................................17 Chapter 5: Extending AADL for Safety Analysis ............................................................23 AADL User-Defined Properties for Generating A SWaP Analysis Report ............23 AADL Annex Extension For Generating Attack Trees ............................................28 Chapter 6: Related Work ..................................................................................................32 Chapter 7: Conclusion.......................................................................................................33 References ..........................................................................................................................34 vii List of Tables Table 1: Case Study: Capability Requirements Specification ......................................6 Table 2: Case Study: Software Requirements Specification.........................................6 Table 3: Case Study: AADL SWaP Analysis Report Plugin Methods/Classes ..........26 Table 4: Case Study: AADL Attack Tree Plugin Methods/Classes ............................30 viii List of Figures Figure 1: Case Study: System of Systems (SoS) / Data Flow ........................................5 Figure 2: Case Study: Signal Processing Unit UML Use Case Diagram .......................8 Figure 3: Case Study: Signal Processing Unit UML Class Diagram .............................9 Figure 4: Case Study: Maintain Temperature UML Sequence Diagram .......................9 Figure 5: Case Study: Maintain Temperature UML Activity Diagram .......................10 Figure 6: Case Study: Process Signal UML Sequence Diagram ..................................11 Figure 7: Case Study: Process Signal UML Activity Diagram ....................................12 Figure 8: Case Study: Signal Processing Unit SysML Requirements Diagram ...........15 Figure 9: Case Study: Signal Processing Unit SysML BDD .......................................16 Figure 10: Case Study: AADL devices.aadl...................................................................17 Figure 11: Case Study: AADL integration.aadl .............................................................18 Figure 12: Case Study: AADL data definitions (processes.aadl) ...................................19 Figure 13: Case Study: AADL thread definitions (processes.aadl) ..............................20 Figure 14: Case Study: AADL SignalProcessor definitions (with latencies).................21 Figure 15: Case Study: AADL Latency Analysis Report ..............................................21 Figure 16: Case Study: AADL swap_properties.aadl ....................................................24 Figure 17: Case Study: AADL devices.aadl with SWaP Properties ..............................25 Figure 18: Case Study: AADL integration.aadl Recapitulation .....................................25 Figure 19: Case Study: AADL SWaP Analysis Report .................................................27 Figure 20: Case Study: AADL BNF for the security_specification annex ....................28 Figure 21: Case Study: AADL security_specification annex .........................................29 Figure 22: Case Study: AADL Attack Tree ...................................................................30 ix Chapter 1: Introduction The typical system/software development life cycle (SDLC) consists of six stages: planning, analysis, design, implementation, integration/testing, and support/maintenance. In a waterfall-based SDLC, these stages are performed sequentially, with only one pass during the lifetime of a specific release of a product. In an iterative / agile-based SDLC, on the other hand, these stages are performed sequentially, many times during the lifetime of a specific release of a product, often in scheduled increments known as sprints. A waterfall-based SDLC tends to be used more in the creation of safety-critical systems due to the rigor of analysis and testing requirements requisite for safety certification. Errors in analysis and design in the waterfall-based SDLC that are detected during the integration/testing phase tend to be more costly in monetary and scheduling terms than the more forgiving iterative / agile-based SDL, where errors can be detected and mitigated rapidly during subsequent product sprints. Irrespective of the flavor of SDLC incorporated by an organization, after requirements are accepted and before an engineer begins implementing a system, modeling languages are often used during the design phase to document a system’s architecture. The creation of modeling language-based artifacts is analogous to the creation of blueprints during the design phase used in manufacturing processes. UML (Unified Modeling Language), created by Grady Booch, Ivar Jacobson, and James Rumbaugh at Rational Software in the early to mid-1990s, is a general-purpose visual modeling language that is used to specify, visualize, construct, and document the artifacts of a software system (Rumbaugh et al., 2005). 1 UML provides structural diagrams that emphasize entities that must be present in a system, behavior diagrams that emphasize what must occur in a system, interaction diagrams that emphasize data and control flow amongst the things in the system, and implementation diagrams that show the structure of the run-time system. Whereas the domain of the software engineer is in the implementation phase, the domain of the systems engineer is in the analysis and integration/testing phase. MBSE (Model-based systems engineering) extends modeling languages, such as UML, to make them useful within the domains of systems engineering. MBSE is expected to replace the document-centric approach that has been practiced by systems engineers in the past and to influence the future practice of systems engineering by being fully integrated into the definition of systems engineering processes (INCOSE, 2007). Due to integration/testing issues having higher cost in a waterfall-based SDLC, MBSE focuses on virtual integration. From the MBSE artifacts, systems engineers can integrate/test systems and perform analyses so that major architectural issues can be detected and mitigated before implementation begins. Two modeling languages that are enabling technologies for MBSE are SysML (Systems Modeling Language) and AADL (Architecture Analysis and Design Language). SysML is a graphical modeling language that can be used to visualize and communicate the designs of sociotechnical systems on all scales (Delligatti, 2014). It is used as an architecture modeling language for systems engineering applications, and was created by the SysML Partners’ SysML Open Source Specification Project in 2013. It was subsequently adapted and adopted by the Object Management Group (OMG) as OMG SysML in 2006. SysML is a superset of UML and supports the specification, analysis, design, and V&V of systems and systems