Semi-formal Reformulation of Requirements for Formal Property Verification Katharina Ceesay-Seitz, Verification Engineer, CERN, Geneva, Switzerland ([email protected]) Hamza Boukabache, Project Manager, CERN, Geneva, Switzerland ([email protected]) Daniel Perrin, Section Leader, CERN, Geneva, Switzerland ([email protected]) Abstract— Ambiguously specified requirements can be a source of risk for safety-critical electronic designs. Requirement specifications in natural language are subject to misinterpretation. A method is proposed that reduces the risk of misinterpretations. Requirements are reformulated into semi-formal properties, which we call Natural Language Properties (NLPs). These statements are composed of natural language patterns which are then translated into SystemVerilog for formal verification. This reformulation is done by an independent verification engineer and then reviewed by the requirements engineer. Applying this technique for the verification of the CERN RadiatiOn Monitoring Electronics (CROME) led to the discovery of a safety-critical fault. Keywords—semi-formal methods; formal property verification; SystemVerilog assertions; functional safety; independence I. INTRODUCTION Project teams in international organizations and companies consist of many people of different backgrounds. They often have different native languages and have previously gained work experience in different countries and cultures. They also have been educated in different universities, which may use different notations and terms to describe the same topic. They may even use different unities to describe physical properties. Even persons with the same mother tongue, trained at the same university in the same subject and working in the same company may interpret the same sentence in different ways. A prominent example for a failure related to intercultural project teams is NASA’s Mars Climate Orbiter. The spacecraft was lost when entering Mars’ atmosphere. The cause of this loss was that the spacecraft’s on-board software used different measurement units than the ground- segment software. The former performed calculations in metric units whereas the latter used English units [1]. Such mismatches can have life threatening consequences. Requirements specifications are often written from the end user’s perspective, which means at system level. When the development happens on Register Transfer Level (RTL), e.g. in VHDL code, a much more detailed specification is needed. Safety standards like IEC 61508 (standard for functional safety of electrical/electronic/programmable electronic systems), ISO 26262 (standard for automotive) or DO-254 (standard for airborne electronics) all specify similar safety life cycles. Typically, these follow the V-model. Validation according to the V-model of IEC 61508 should be based on the functional and safety-requirements [10]. The verification against the design specification does not automatically mean that the system is validated against its requirements specification, even though the design specification is back-traceable to the requirements. The specification is often a refinement of the requirements. This step is usually done by the design engineer, who in many cases is a different person than the requirements engineer. At this point a misinterpretation with possibly fatal consequences can easily happen. A misinterpretation would turn into a legal specification statement against which verification will be performed. Ultimately this means that the implementation is verified to comply to a specification statement that does not reflect the original requirement as it was intended. The proposed technique of using Natural Language Properties (NLPs) aims at discovering such cases. Figure 1 shows where the technique can be located in the V-model. It could be applied directly after writing the FPGA/Software Safety Requirements Specification, before the system is designed and implemented. This has the advantage that the requirements are validated before any further work is based on them. Another option is to write NLPs when writing the verification 1 plan. The latter has the advantage that the design specification is already available. Knowledge of implementation details will often be necessary when writing formal properties. The obvious drawback of validation at a later stage is that its results will lead to modifications of the already implemented system. Ideally a verification engineer would perform the requirements validation with NLPs after the specification of the FPGA/Software Safety Requirements. System Safety System Safety Validation Requirements Validation Verification Programmable System Architecture Electronics Integration Specification Testing FPGA/Software Safety Validation FPGA/Software System Requirements Validation NLP FPGA/Software System Verification Module Integration Design Testing NLP Module Design Verification Module/Unit Testing Specification Time Flow, Trace Code Implementation 2 Figure, 1 V-Model with NLPs The usage of NLPs will be demonstrated on a part of the CERN RadiatiOn Monitoring Electronics (CROME) [2]. CERN, the European Organization for Nuclear Research, operates physics experiments such as the Large Hadron Collider (LHC). The LHC and other facilities accelerate beams of particles nearly up to the speed of light and collide them in order to study the fundamental laws of our universe. These experiments produce ionizing radiation. CERN’s personnel and the general public need to be protected from any unjustified exposure to ionizing radiation. The CROME radiation monitoring system has been newly developed for this purpose. It will replace the old existing systems. It consists of hundreds of autonomous units that are measuring radiation levels. These units are responsible for the automatic detection of dangerous radiation levels based on many parameters, which can be configured in a supervision software and transmitted to the monitors over a network connection. If the calculated ambient equivalent dose rate exceeds the configured limits, the CROME Measurement and Processing Units (CMPUs) notify the alarm units and trigger machine stops. The CMPUs implement several signal processing algorithms. One of them is called “integration”. This algorithm calculates the ambient equivalent dose received by an area. If this dose exceeds a configurable threshold, an alarm/interlock signal is set. Several remote commands can be sent, which can e.g. restart the integration calculation or deactivate the alarm function. The outcomes of 7 conditions at two consecutive measurement times influence the decision whether an alarm should be activated. This leads to 214 possibilities. II. RELATED WORK Property Specification Patterns (SPS) were introduced in [3]. These are recurring patterns in requirement specifications. A pattern consists of a name, an intent and a common specification formalism. The latter could for example be a Linear Temporal Logic (LTL) property. In [4] this approach was assessed for a requirements 2 specification in the automotive domain. They concluded that 70% of the requirements could be mapped to such patterns. SPS could be well combined with the suggested Natural Language Properties. The authors of [5] proposed to use a subset of the English language for the specification of requirements in order to get a clear mapping from English to a formal language. A tool for the translation of such specifications into formal properties was presented in [6]. A limitation is that generated properties cannot span multiple clock cycles. Only 44% of the English language assertions could be automatically translated into SystemVerilog assertions, which means that this approach, though promising, is not yet ready to be applied in real projects. Another tool for the automated translation of English language into formal properties has been presented in [7]. This tool relies on the similarity of sentences in low-level specifications. Sentences like “Signal A remains high for 5 clock cycles” can be automatically translated. High-level statements still need to be translated manually. Another tool for automated translation of natural language comments in Hardware Description Language (HDL) code has been presented in [8]. While the translation works very well for the tested benchmark, the authors highlight a general limitation of automated approaches that use natural language processing. English language constructs like “must be” can refer to the present or the future. A tool needs to be able to correctly translate such statements. From these references it can be seen that the automated translation from requirements to formal properties is not trivial. There are currently no safety qualified tools available to perform this step. Therefore, the manual formulation used in our workflow does not add much overhead. Furthermore, it builds on another important concept in the development and verification of safety-critical systems: independence. Several safety standards require independent reviews and verification. The study presented in [9] also confirmed that the number of discovered faults by an independent verification team is higher than when the designers verify their own product. III. APPLICATION A Natural Language Property (NLP) is a natural language version of a formal SystemVerilog property. Table I provides a one to one mapping from natural language constructs to SystemVerilog. The goal of using NLPs is to facilitate communication between engineers with different backgrounds