Safety Analysis of an Airbag System using Probabilistic FMEA and Probabilistic Counter Examples Husain Aljazzar1, Manuel Fischer2, Lars Grunske3, Matthias Kuntz1, Florian Leitner1, Stefan Leue1 1 University of Konstanz, Germany 2 TRW Automotive GmbH, Germany 3 Swinburne University, Australia Abstract niques have been integrated into the process [6, 8, 9, 19, 16]. While these techniques are able to establish cause- Failure mode and effects analysis (FMEA) is a technique consequence relationships, they are unable to calculate the to reason about possible system hazards that result from actual failure probabilities. Therefore, stochastic model system or system component failures. Traditionally, FMEA checking was applied to FMEA, leading to a probabilistic does not take the probabilities with which these failures FMEA (pFMEA) process [14]. Currently, this pFMEA pro- may occur into account. Recently, this shortcoming was cess provides no means to help the designer in reducing the addressed by integrating stochastic model checking tech- risk of failures. It only supports the first step of the FMEA niques into the FMEA process. A further improvement is process, which is to identify cause-consequence relation- the integration of techniques for the generation of counter ships and compute the actual hazard probabilities. examples for stochastic models, which we propose in this The contributions of our paper can be described as fol- paper. Counter examples facilitate the redesign of a poten- lows. tially unsafe system by providing information which compo- • We illustrate the usefulness of pFMEA as supported nents contribute most to the failure of the entire system. The by stochastic model checking using the real-life case usefulness of this novel approach to the FMEA process is study of an airbag system. We describe how to map illustrated by applying it to the case study of an airbag sys- the system architecture to a PRISM [23] model and il- tem provided by our industrial partner, the TRW Automotive lustrate how to perform pFMEA on this model. The GmbH. airbag case study results from a collaboration with the automotive supplier TRW Automotive GmbH in Radolfzell, Germany, and is based on real data. Due to 1 Introduction intellectual property concerns of our industrial partner, we are unable to publish the concrete values of compo- nent or overall system failure probabilities1. This does In light of the fact that a failure of a safety-criticalsystem not affect our finding that pFMEA can lead to useful can lead to injuries and even loss of life it is extremely im- failure probability assessment values, as confirmed by portant to provide designers with safety assessment meth- our industrial partner. ods that help to minimise the risk of the occurrence of such • We address the inability of the current pFMEA method disastrous events. One of these methods is failure mode to give guidance in how to improve system depend- and effects analysis (FMEA) [20]. In FMEA, a team of ability by integrating a recently developed technique trained engineers or system designers analyses the cause for finding counter examples in stochastic models. consequence relationships of component failures on system Counter examples provide means to identify those hazards. After having found such a relation, the occurrence parts that contribute most probably to the failure of the probability of that hazard is computed. It is then checked system and thus, provide valuable information for its whether this value is above a certain threshold, defined by redesign. the tolerable hazard probability or rate (THP or THR). If this is the case measures must be taken to reduce the proba- 1Any concrete probability or rate values that this paper presents are ei- bility of the undesired event. ther taken from publicly available specifications, such as the ISO 26262 standard [22], or plausible estimates confirmed by TRW Automotive To support the traditionally time-intensive and error- GmbH. The PRISM model used in the analysis does contain concrete prob- prone FMEA process, functional model checking tech- ability values for the components used. This paper is organised as follows: In Sec. 2 we will briefly the hazard conditions as temporal logical formulae. As a re- introduceFMEA, pFMEA and counter examples in stochas- sult, fault injection experiments can be created where spe- tic model checking. Sec. 3 is devoted to the description of cific failure modes are injected into the system behaviour the airbag system and its PRISM model. In Sec. 4 we will model. Model checking tools can then analyse the conse- describe possible hazard conditions and system failures, and quences on the formalised hazard conditions. Sec. 5 is devoted to the probabilistic FMEA of the airbag system, supported by counter example generation. In Sec. 6 2.2 pFMEA we compare our approach with existing approaches in the FMEA literature. Sec. 7 describes the lessons learnt from A further developmentof the idea to use of model check- this case study. Finally, Sec. 8 concludes the paper with a ing support for FMEA is the approach referred to as proba- summary and an outlook on future research. bilistic FMEA (pFMEA) presented in [14]. Instead of func- tional state-based models pFMEA uses stochastic models, 2 FMEA, pFMEA and Counter Examples in particular discrete and continuous time Markov chains. The hazard conditions are formulated as stochastic tempo- This section explains the basic concepts of the failure ral logical formulae. As a consequence the tolerable hazard model and effect analysis (FMEA) and its probabilistic ex- probabilities can also be integrated into the fomalisation of tension [14]. Furthermore, in Sec. 2.3 we briefly introduce the hazard conditions. Furthermore, to each injected fail- an approach to counter example generation for stochastic ure mode an occurrence probability can be assigned in the model checking. A more detailed account of these topics probabilistic model. A main benefit of pFMEA is the ability can be found in [1, 3, 18]. to probabilistically estimate the likelihood that an injected failure mode will lead to a violation of the hazard condi- 2.1 FMEA tion. The use of stochastic models also avoids a common shortcoming of using functional model checking in FMEA, As described in the introduction, the aim of an FMEA namely that the model checker finds a relationship between is to explore the consequences, such as hazards, of known the injected failure mode and a hazard that is very unlikely component-level failure modes and to propose countermea- to occur in practice. As already noted in [14], one practi- sures to mitigate and reduce the probability that these con- cal problem of pFMEA is the lack of counter examples in sequences occur. The final outcome of an FMEA is a ta- stochastic model checking. This impedes the explanation of ble which documents for each component the set of rele- property violations and hence failure mode / consequence vant component failure modes and for each of these failure relationships found by the stochastic model checker. modes its consequences. Possible failure detection, correc- tion or mitigation mechanisms may also be recommendedin 2.3 Counter Examples in Stochastic Model this table. The structure, number of columns and meaning Checking of columns of the resulting FMEA table may vary in dif- ferent organizations performing FMEA. However, the fol- In stochastic model checking, the property that is to lowing column headings are commonly used [20]: inves- be verified is specified using a variant of temporal logic. tigated component, failure mode, description of the failure The temporal logic used in this paper is CSL (continu- mode/local effect of the failure mode, possible cause for ous stochastic logic) [4, 5]. CSL is a stochastic variant of the failure, effect at the system level, recommended fail- CTL [10]. It is tailored to reason about quantitative system ure detection mechanism, recommended mitigation mech- behaviour, including the performance and dependability of anism, and recommended design changes. For complex a system. Just like in traditional model checking, given an systems with a large number of components and a large appropriate system model and a CSL property, a stochastic number of failure modes per component this table can be- model checking tool such as PRISM can verify automati- come very large. Additionally, it has been reported in [17] cally whether the model satisfies the property. If the model that the table may contain redundant information since dif- refutes the property, a counter example usually helps engi- ferent failure modes can lead to the same consequences. neers to comprehend the reasons of the property violation The FMEA procedure is commonly defined by an iterative and to devise arrangements to fix the error. The computa- process [15] that identifies for all components the possi- tion of counterexamples in stochastic model checking has ble failure modes and identifies their consequences. Re- recently been addressed in [1, 2, 3, 18]. cent approaches [25, 19, 8, 9, 6, 7, 16] aim to support the FMEA process, especially the identification of possi- 2.3.1. Notion of Counter Examples. For our purposes it ble consequences with model checking. The basic idea is to suffices to consider upper bounded properties which require formalise the system’s behaviour as a state-based model and the probability of a property offending behaviour not to ex- ceed a certain upper probability bound. In CSL such prop- rious or even fatal injuries for the occupants. Current airbag erties can be expressed by formulas of the form P≤p(ϕ), systems consist of not only the front airbags but also of side, where ϕ is path formula specifying undesired behaviour of head, knee and a number of further airbags to protect both the the system. Any path which starts at the initial state of the driver and the passengers. the system and which satisfies ϕ is called a diagnostic path.