Open Archive TOULOUSE Archive Ouverte ( OATAO ) OATAO is an open access repository that collec ts the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited version published in : http://oatao.univ-toulouse.fr/ Eprints ID : 12671 Official URL: http://dx.doi.org/10.1007/978-3-642-39799-8_31 To cite this version : Esparza, Javier and Lammich, Peter and Neumann, René and Nipkow, Tobias and Schimpf, Alexander and Smaus, Jan-Georg A Fully Verified Executable LTL Model Checker. (2013) In: 25th International Conference on Computer Aided Verification (CAV 2013), 13 June 2013 - 19 July 2013 (Saint Petersbourg, Russian Federation). Any corre spondance concerning this service should be sent to the repository administrator: staff -oatao@listes -diff.inp -toulouse.fr A Fully Verified Executable LTL Model Checker ⋆ Javier Esparza 1, Peter Lammich 1, Ren´eNeumann 1, Tobias Nipkow 1, Alexander Schimpf 2, Jan-Georg Smaus 3 1 Technische Universit¨at M¨unchen, {esparza,lammich,neumannr,nipkow}@in.tum.de 2 Universit¨at Freiburg, [email protected] 3 IRIT, Universit´ede Toulouse, [email protected] Abstract. We present an LTL model checker whose code has been completely verified using the Isabelle theorem prover. The checker consists of over 4000 lines of ML code. The code is produced using recent Isabelle technology called the Refinement Framework, which allows us to split its correctness proof into (1) the proof of an abstract version of the checker, consisting of a few hundred lines of “formalized pseudocode”, and (2) a verified refinement step in which mathematical sets and other abstract structures are replaced by implementations of efficient structures like red-black trees and functional arrays. This leads to a checker that, while still slower than unverified checkers, can already be used as a trusted reference implementation against which advanced implementations can be tested. We report on the structure of the checker, the development process, and some experiments on standard benchmarks. 1 Introduction Developers of verification tools are often asked if they have verified their own tool. The question is justified: verification tools are trust-multipliers—they increase our confidence in the correctness of many other systems—and so their bugs may have a particularly dangerous multiplicative effect. However, with the current state of verification technology, proving software correct is dramatically slower than testing it. The strong advances in verification technology of the last two decades would have not been possible if verifiers had only deployed verified tools. In this paper we propose a pragmatic solution to this dilemma, precisely because of the advances in verification: verified reference implementations of standard verification services. Verifiers working on sophisticated techniques to increase efficiency can test their tools against the reference implementation, and so gain confidence in the correctness of their systems. We present a reference implementation for an LTL model checker for finite-state systems `a la SPIN [ 10 ]. The model checker follows the well-known automata-theoretic approach. Given a finite-state program P and a formula φ, two B¨uchi automata are constructed that recognize the executions of P , and all potential executions of P that violate φ, respectively. The latter is constructed using the algorithm of Gerth et al. [7], which—while not the most efficient—is particularly suitable for formal proof ⋆ Research supported by DFG grant CAVA, Computer Aided Verification of Automata because of its inductive structure. Then the product of the two automata is computed and tested on-the-fly for emptiness. For the emptiness check we use an improved version of the nested depth-first search algorithm [6,22]. A reference implementation of an LTL model checker must be fully verified (i. e. it must be proved that the program satisfies a complete functional specification). At the same time, it must be reasonably efficient—as a rule of thumb, the verifier should be able to run a test on a medium-size benchmark in seconds or at most a few minutes. Simultaneously meeting these two requirements poses big challenges. An imperative programming style allows one to use efficient update-in-place and random-access data structures, like hash tables; however, producing fully verified imperative code is hard. In contrast, a functional style, due to its extensive use of recursion and recursively defined data structures like lists and trees, allows for standard proofs by induction; however, efficiency can be seriously compromised. In this paper we choose a functional style (we produce ML code), and overcome the efficiency problem by means of a development process based on refinement. We use the Isabelle Collection and Refinement Frameworks [ 13 ,16 ] presented in Section 5. The Refinement Framework allows us to first prove correct an inefficient but simple formalization and then refine it to an efficient version in a stepwise manner. The Collection Framework provides a pre-proved library of efficient implementations of abstract types like sets by red-black trees or functional versions of arrays, which we use as replacement of hash tables. To prove full functional correctness of executable code, we define the programs in the logic HOL of the interactive theorem prover Isabelle [ 19 ]. More precisely, the programs are defined in a subset of HOL that corresponds to a functional programming language. After proving the programs correct, ML (or OCaml, Haskell or Scala) code can be generated automatically from those definitions [ 8]; like in PVS and Coq, the code generator (which translates equations in HOL into equations in some functional programming language) is part of the trusted kernel of Isabelle. Isabelle proofs are trustworthy because Isabelle follows the LCF architecture and all proofs must go through a small kernel of inference rules. We conduct some experiments on standard benchmarks to check the efficiency of our code. To gain a first impression, for products with 10 6–107 states, our implementation explores 10 4–105 states per second. While this is still below the efficiency of the fastest LTL model checkers, it fulfills the goal stated above: products with 10 6–107 states allow to explore many corner cases of the test space. All supporting material, including the ML code, can be found online at http://cava.in.tum.de/CAV13. 1.1 Related work To the best of our knowledge, we present the first fully verified executable LTL model checker. In previous work [ 21 ], we had presented a formalization of the LTL-to-B¨uchi translation by Gerth et al. [ 7] in Isabelle. However, instead of sets, lists were used everywhere for executability reasons. In the present work, we have re-formalized it using the Refinement Framework, in order to separate the abstract specification (in terms of sets) and the concrete implementation (refining sets to red-black trees, ordered lists etc., as appropriate). This increased modularity and performance. Moreover we have added a second phase that translates the resulting generalized B¨uchi automata into ordinary ones. The first verification of a model checker we are aware of is by Sprenger for the modal µ-calculus in Coq [ 23 ]. No performance figures or larger examples are reported. There is a growing body of verified basic software like a C compiler [ 17 ] or the seL4 operating system kernel [ 11 ]. With Leroy’s compiler we share the functional programming approach. In contrast, seL4 is written in C for performance reasons. But verifying its 10,000 lines required 10–20 person years (depending on what you count). We believe that verifying a model checker at the C level would require many times the effort of our verification, and that verifying functional correctness of significant parts of SPIN is not a practical proposition today. Even seL4, although very performant, was designed with verification in mind. Of course the idea of development by refinement is an old one (see Section 5 for references). A popular incarnation is the B-Method [ 1] for which a number of support tools exist. The main difference is that B aims at imperative programs while we aim at functional ones. 2 Isabelle/HOL Isabelle/HOL [ 19 ] is an interactive theorem prover based on Higher-Order Logic (HOL). You can think of HOL as a combination of a functional programming language with logic. Although Isabelle/HOL largely follows ordinary mathematical notation, there are some operators and conventions that should be explained. Like in functional programming, functions are mostly curried, i. e. of type τ1 → τ2 → τ instead of τ1 × τ2 → τ. This means that function application is usually written f a b instead of f(a, b ). Lambda terms are written in the standard syntax λx. t (the function that maps x to t) but can also have multiple arguments λx y. t , paired arguments λ(x,y ). t , or dummy arguments λ-. t . Names may contain hyphens, as in nested-dfs ; do not confuse them with subtraction. Type variables are written ’a , ’b etc. Compound types are written in postfix syntax: τ set is the type of sets of elements of type τ, similarly for τ list . Lists come with the standard functions length , set (converts a list into a set), distinct (tests if all elements are distinct), and xs!i (returns the ith element of list xs ). The function insert inserts an element into a set. A record with fields l1,...,l n that have the values v1,...,v n is written (|l1 = v1, . , l n = vn|). The field l of a given record r is selected just by function application: l r . In some places in the paper we have simplified formulas or code marginally to avoid distraction by syntactic or technical details, but in general we have stayed faithful to the sources.