Finite-State Abstractions for Probabilistic Computation Tree Logic
Total Page:16
File Type:pdf, Size:1020Kb
Imperial College London Department of Computing Finite-State Abstractions for Probabilistic Computation Tree Logic Daniel Wagner 2011 Supervised by Dr. Michael Huth Submitted in part fulfilment of the requirements for the degree of Doctor of Philosophy in Computing of Imperial College London and the Diploma of Imperial College London Declaration I herewith certify that all material in this dissertation which is not my own work has been duly acknowledged. Selected results from this dissertation have been disseminated in scientific publications as detailed in Chapter 1.10. Daniel Wagner 2 Abstract Probabilistic Computation Tree Logic (PCTL) is the established temporal logic for probabilistic verification of discrete-time Markov chains. Probabilis- tic model checking is a technique that verifies or refutes whether a property specified in this logic holds in a Markov chain. But Markov chains are of- ten infinite or too large for this technique to apply. A standard solution to this problem is to convert the Markov chain to an abstract model and to model check that abstract model. The problem this thesis therefore stud- ies is whether or when such finite abstractions of Markov chains for model checking PCTL exist. This thesis makes the following contributions. We identify a sizeable frag- ment of PCTL for which 3-valued Markov chains can serve as finite abstrac- tions; this fragment is maximal for those abstractions and subsumes many practically relevant specifications including, e.g., reachability. We also de- velop game-theoretic foundations for the semantics of PCTL over Markov chains by capturing the standard PCTL semantics via a two-player games. These games, finally, inspire a notion of p-automata, which accept entire Markov chains. We show that p-automata subsume PCTL and Markov chains; that their languages of Markov chains have pleasant closure proper- ties; and that the complexity of deciding acceptance matches that of proba- bilistic model checking for p-automata representing PCTL formulae. In ad- dition, we offer a simulation between p-automata that under-approximates language containment. These results then allow us to show that p-automata comprise a solution to the problem studied in this thesis. 3 Contents 1 Introduction 8 1.1 Model checking . 10 1.2 Abstraction and refinement . 12 1.3 (Finitary) completeness . 14 1.4 Finite model property . 16 1.5 Abstraction and undecidability . 18 1.6 Witnesses for truth and falsity . 19 1.7 Research agenda of this thesis . 22 1.8 Achievements in this thesis . 24 1.9 Outline of this thesis . 25 1.10 Publications . 27 1.11 Summary of chapter . 28 2 Background on Models and Logics 29 2.1 Kripke structures . 30 2.2 Linear-time and branching-time temporal logic . 36 2.3 Simulation and bisimulation . 51 2.4 Markov chains and Markov decision processes . 60 2.5 Probabilistic branching-time logic: PCTL . 65 2.6 Probabilistic simulation and bisimulation . 76 2.7 Unfoldings . 79 2.8 Summary of chapter . 83 3 Background on Automata and Games 85 3.1 Word automata . 86 3.2 Probabilistic automata . 95 3.3 Alternating tree automata . 99 3.4 Hintikka games . 107 3.5 Stochastic parity games . 109 4 3.6 Summary of chapter . 112 4 Completeness for a Fragment of PCTL 114 4.1 Existing results . 114 4.2 Our setting: Markov chains and PCTL . 117 4.3 Completeness for a fragment of PCTL via 3-valued unfoldings 118 4.4 Summary of chapter . 131 5 A Game Semantics for PCTL 132 5.1 Game semantics . 132 5.2 Winning strategies . 149 5.3 Summary of chapter . 158 6 Completeness for full PCTL via p-Automata 159 6.1 p-Automata . 160 6.2 Acceptance games . 165 6.3 Expressiveness of p-automata . 173 6.4 Simulation of p-automata . 186 6.5 Wrong embedding of Markov chains . 203 6.6 Summary of chapter . 205 7 Evaluation 206 7.1 Related work on completeness of abstractions . 206 7.2 Evaluation of achievements . 209 7.3 Future work . 210 Bibliography 213 5 List of Figures 2.1 A Kripke structure K....................... 33 2.2 A labelled transition system (T; s0)............... 34 2.3 A partial Kripke structure (K; s0)................ 35 2.4 Hasse diagram of the information order on ftt; ?; ffg...... 36 2.5 A Kripke structure C....................... 53 2.6 A Kripke structure A....................... 53 2.7 A Kripke structure M....................... 54 2.8 A Kripke structure N....................... 55 2.9 A Kripke structure L....................... 56 2.10 A Kripke structure A....................... 58 2.11 A Kripke structure C....................... 58 2.12 A labelled discrete-time Markov chain M............ 63 2.13 A 3-valued discrete-time Markov chain M............ 64 2.14 A Markov decision process M................... 65 2.15 A Markov chain M........................ 68 2.16 A Markov chain M........................ 69 2.17 A Markov chain M........................ 78 2.18 A Markov chain N......................... 78 2.19 A Markov chain B......................... 80 2.20 A Markov chain M........................ 82 s0 2.21 The finite unfolding M3;3..................... 82 3.1 A deterministic word automaton A................ 88 3.2 A deterministic Büchi automaton A............... 90 3.3 A deterministic Büchi automaton B............... 91 3.4 A non-deterministic Büchi automaton C............. 94 3.5 A non-deterministic Büchi automaton A............. 95 3.6 A probabilistic automaton A................... 97 3.7 A probabilistic automaton C................... 98 6 3.8 A probabilistic automaton D................... 98 3.9 A Kripke structure M....................... 104 3.10 An accepting run of B on M................... 104 4.1 A labelled Markov chain M.................... 124 s0 4.2 The finite unfolding M2 ..................... 124 4.3 Maximal height of an unfolding. 127 4.4 A concrete Markov chain M................... 129 6.1 Graph GA of automaton A.................... 163 6.2 A Markov chain M........................ 164 6.3 Case 3 of acceptance game. 171 6.4 Case 1 of acceptance game. 172 6.5 A Markov chain M........................ 204 7 1 Introduction Computing is becoming an ubiquitous part of our lives and an irreplaceable factor in our modern society. Our daily lives and our society as a whole are becoming more and more reliant and dependent on computer systems and their well-functioning. At the same time the various computer systems and especially interaction between them become more and more complex. The systems grow in size and in complexity as well as in sheer number and interconnectivity. Few systems are truly self-contained and many of them interact with each other and with changing and partially unknown environments. The need to understand such systems and their interaction, and the need to be sure about their functionality and their behaviour is becoming more and more pressing. The size and complexity of such systems makes manual inspections obviously infeasible. Furthermore one would like to be able to audit a system on more than one level between the high- level specification and the source code. Actually, one would like to audit systems at all levels between high-level specification and source code, or at least to understand the relationship between the properties of the system in its various descriptions. The result of such an audit should be a rigorous verification and should be fully integrated in the engineering process. The importance of (automatic) formal verification of system behaviour has been recognised early on in the history of computing. According to Cliff Jones [107] already some of the work by Herman Goldstine and John von Neumann [83], by Alan Turing [171] and somehow even the “pre-electronic” work by Charles Babbage [15] was concerned with (the formal verification of) the correctness of programs. It was in the 1960s that the research in formal verification and program analysis truly took off the ground [95, 137, 119]. Since then – with the growing importance of computer systems for industry and society – the need for efficient automatic verification techniques is becoming increasingly important, both economically and socially. Although posed early on, the problem is far from solved. Essentially, 8 the underlying, fundamental problems are undecidable and so incomplete methods are required. Many verification techniques have been suggested, analysed and deployed in applications, but all of them – even for decidable settings – struggle to keep up with the growth in size and complexity of current systems. Formal verification of system behaviour therefore remains an important and active research area. The importance of formal verification was recently acknowledged by Tony Hoare’s declaration of The verifying compiler as one of the Grand Chal- lenges in Computing [96], and by the UK Computing Research Commit- tee (UKCRC) whose Grand Challenge 6 is Dependable Systems Evolution [183, 182]. These Verification Grand Challenges found strong support in the scientific community. Motivating examples, cases of support, different approaches and perspectives were presented, for example, in the position papers presented at the conference “Verified Software: Theories, Tools, Ex- periments”, which was held at the ETH Zürich in 2005 [2]. As the motivation for their Grand Challenge 6 the UKCRC formulates the general need for a notion of justification and dependability: The vision of GC6 is of a future in which all computer systems justify the trust that society increasingly places in them. De- pendability is a multi-faceted notion which includes fault toler- ance, requirements engineering and verification amongst other topics. This broad definition is not restricted to formal verification but also includes other evaluations of evolving and interacting systems. The available means for evaluating systems are broad, ranging from testing and numerical sim- ulation to the formally rigorous approaches such as theorem proving and model checking. In model checking [69, 42, 155, 154], which this thesis is concerned with, one tries to formally answer a query about a system, such as “After each computation, will the program go back to its idle mode?” In model checking the returned answer is usually rigorously proved as opposed to the approximative nature of, say, testing and Monte Carlo simulation.