Incremental Formal Verification of Hardware Hana Chockler, Alexander Ivrii, Arie Matsliah, Shiri Moran, Ziv Nevo IBM Research – Haifa E-mail:fhanac,alexi,ariem,shirim,[email protected] Abstract—Formal verification is a reliable and fully problem is especially acute in regression verification, automatic technique for proving correctness of hardware where a new version of a hardware design is re-verified designs. Its main drawback is the high complexity of veri- with respect to the same (or very similar) specification. fication, and this problem is especially acute in regression verification, where a new version of the design, differing Since regression verification is a only a preliminary from the previous version very slightly, is verified with (albeit necessary) stage in functional verification of a respect to the same or a very similar property. In this new version, the time and effort allocated to it are usually paper, we present an efficient algorithm for incremental much lower than for the initial verification; in reality, verification, based on the ic3 algorithm, that uses stored since the amount of effort is the same as for the initial information from the previous verification runs in order to improve the complexity of re-verifying similar designs verification, our experience is that regression verification on similar properties. Our algorithm applies both to the is often not performed thoroughly enough, thus possibly positive and to the negative results of verification (that is, leading to lower quality designs.1 A better option would both when there is a proof of correctness and when there be to verify the changes incrementally, that is, to reuse is a counterexample). The algorithm is implemented and the results from the previous execution and only verify experimental results show improvement of up to two orders of magnitude in running time, compared to full verification. the change. Another area in which incremental verification tech- niques are in dire need is coverage computation in formal I. INTRODUCTION verification. Most of the existing work on coverage in formal verification is based on the notion of mutation Today’s rapid development of complex hardware de- coverage, where small mutations are introduced to the formal signs requires reliable verification methods. In design and the mutant designs are checked with re- verification , we verify the correctness of a design with spect to the original specification [HKHZ99], [CKV06b], respect to a desired behavior by checking whether a [CKV06a], [KLS08], [CKP10], with the goal of check- labeled state-transition graph that models the design ing thoroughness of formal specifications. Efficient in- satisfies a specification of this behavior, expressed in cremental verification techniques can reduce the cost of terms of a temporal logic formula or a finite automaton computing mutation coverage, where a large number of [CGP99]. The main advantages of formal verification slightly modified designs need to be checked with respect tools are their reliability (if a design passes verification, to the same property. then it is 100% correct with respect the specification), Several papers view the problem of incremental ver- full automation of the verification process, and the ability ification as an instance of dynamic graph algorithms. of the tools to accompany a negative answer to the In this setting, a design is represented as a graph and correctness query by a counterexample to the satisfaction incremental verification checks the influence of small of the specification in the design [CGMZ95]. changes in the graph (edge insertion and removal) on the The main drawback of the formal verification tech- properties that were previously satisfied in this graph, nology, and the one that prevents it from being even thus reducing the problem of incremental verification more widely used in the hardware industry, is that it to a dynamic graph problem. However, dynamic graph requires significant computational effort, even for mod- connectivity, one of the main problems in dynamic graph erately sized designs. Moreover, when small changes algorithms, and the one that is most relevant to verifi- are introduced into the design or the specification, for cation, is an open problem, hence this reduction is of example due to a bug fix or an upgrade, the whole design limited value in practice [SBS95], [CK03]. A somewhat needs to be re-verified, generally requiring the same related direction is using the reduction to dynamic graph amount of resources as for the initial verification. The 1Our experience is based on participating in the formal verification This work is partially supported by the European Community under of IBM hardware designs, as well as on discussions with formal the call FP7-ICT-2009-5 – project PINCETTE 257647. verification engineers in other companies. problems in order to prove complexity results for LTL finder can be used as a starting point of subsequent ic3’s model checking of evolving designs with non-changing execution, with minor modifications to the algorithm. properties [KW03]. Similarly, our algorithm allows to save a small part of The idea of saving the result of model checking in a counterexample produced by ic3 that is, nevertheless, order to use it for subsequent model-checking queries is sufficient in order to easily reproduce a concrete coun- extensively used in counter-example-guided abstraction terexample and then use this part in order to compute refinement (CEGAR) approach [CGJ+03], where the counterexamples for a modified design (or a modified state-explosion problem is addressed by iterative veri- property), by re-using the saved part “as is” or “patch- fication and refinement of an abstract design. Abstract ing” it to produce a valid counterexample for a modified counterexamples are analyzed and, if spurious, are used version. To improve the performance of the algorithm for in order to guide the refinement process of the abstrac- reusing counterexamples, we propose a simple technique tion for the next iteration of the verification process (see that reduces the size of the partial assignments produced also [LBBO01]). by ic3 by ≈ 30%. In this paper, we present an approach for re-using the The algorithm is implemented in the model-checking result of model-checking a design (either a proof or a engine IVE (incremental verification engine), which is counterexample) for verification of the same or a slightly a part of the formal verification platform of IBM [Rul], different property on the same or a slightly modified [Six] and is checked on the Hardware Model Checking design. Our method applies both for the case where Competition (HWMCC’10) [HWM10] benchmarks as the result of the model-checking query is negative, in well as on a real IBM hardware design. Our results show which case we re-use a counterexample, and where it is a significant speed-up (up to three orders of magnitude) positive, and we re-use the correctness proof. In fact, the compared to the re-run of the same model-checking later scenario is very common in regression verification procedure. We note that the performance of IVE is (since the previous version of the design is assumed to on par with the state-of-the art model checking tools, pass the verification successfully). which makes the speed-up achieved by our incremental verification algorithm even more significant. The basis of our work is the novel ic3 (“Incremental The rest of this paper is organized as follows. The Construction of Inductive Clauses for Indubitable Cor- necessary definitions and an overview of the ic3 algo- rectness”) model checking algorithm, recently proposed rithm are provided in Section II. We describe the main by Aaron Bradley [Bra11]. In addition to being one of contribution of this paper – the invariant finder and the the fastest bit-level verification methods [BEM11], the incremental verification algorithms – in Section III, [BBC+11], the ic3 algorithm lends itself very naturally and present the experimental results of executing our to incremental verification. implementation on known benchmarks and on an IBM We describe an algorithm for saving the relevant design in Section IV. In Section V we summarize our parts of the proof obtained by ic3, and using them contributions and discuss possible directions for future to reproduce the proof (or counterexample) on a new, work. The complete table of running times and speed- possibly modified version of the model. This requires ups achieved by our algorithm on all benchmarks from very little computational effort in case the same proof the HWMCC’10 competition appears in the full version works for the new version as well; in case the same of this paper [Rul]. proof does not apply, our method attempts to “patch” it by extracting the maximum valid part of the previously II. PRELIMINARIES saved information and using it as a basis for proving In this section, we give the necessary definitions for the new version. The latter case, where the original our algorithm and provide overviews of a SAT solver proof does not apply “as is” is the main strength of with incremental capabilities and of the ic3 algorithm. our method. Roughly speaking, the main idea of our algorithm is as follows. First we observe that producing A. Definitions (and saving) re-usable information does not incur any Throughout this paper we consider verification of significant overhead on top of the standard execution of safety properties on finite state machines (FSMs). An ic3; in addition, the saved part is usually very small. We FSM M is a tuple hX; I; T i, where X is a set of Boolean also describe a query-efficient SAT-based algorithm that state variables, such that each assignment s 2 f0; 1gX we call an invariant finder for extracting the maximum corresponds to a state of M, and the predicates I ⊆ valid part of the previously saved information.