Model Checking Infinite-State Systems: Generic and Specific Approaches Anthony Widjaja To I V N E R U S E I T H Y T O H F G E R D I N B U Doctor of Philosophy Laboratory for Foundations of Computer Science School of Informatics University of Edinburgh 2010 Abstract Model checking is a fully-automatic formal verification method that has been ex- tremely successful in validating and verifying safety-critical systems in the past three decades. In the past fifteen years, there has been a lot of work in extending many model checking algorithms over finite-state systems to finitely representable infinite- state systems. Unlike in the case of finite systems, decidability can easily become a problem in the case of infinite-state model checking. In this thesis, we present generic and specific techniques that can be used to derive decidability with near-optimal computational complexity for various model checking problems over infinite-state systems. Generic techniques and specific techniques pri- marily differ in the way in which a decidability result is derived. Generic techniques is a “top-down” approach wherein we start with a Turing-powerful formalism for infinite- state systems (in the sense of being able to generate the computation graphs of Turing machines up to isomorphisms), and then impose semantic restrictions whereby the desired model checking problem becomes decidable. In other words, to show that a subclass of the infinite-state systems that is generated by this formalism is decidable with respect to the model checking problem under consideration, we will simply have to prove that this subclass satisfies the semantic restriction. On the other hand, spe- cific techniques is a “bottom-up” approach in the sense that we restrict to a non-Turing powerful formalism of infinite-state systems at the outset. The main benefit of generic techniques is that they can be used as algorithmic metatheorems, i.e., they can give unified proofs of decidability of various model checking problems over infinite-state systems. Specific techniques are more flexible in the sense they can be used to derive decidability or optimal complexity when generic techniques fail. In the first part of the thesis, we adopt word/tree automatic transition systems as a generic formalism of infinite-state systems. Such formalisms can be used to gener- ate many interesting classes of infinite-state systems that have been considered in the literature, e.g., the computation graphs of counter systems, Turing machines, push- down systems, prefix-recognizable systems, regular ground-tree rewrite systems, PA- processes, order-2 collapsible pushdown systems. Although the generality of these formalisms make most interesting model checking problems (even safety) undecid- able, they are known to have nice closure and algorithmic properties. We use these nice properties to obtain several algorithmic metatheorems over word/tree automatic systems, e.g., for deriving decidability of various model checking problems including recurrent reachability, and Linear Temporal Logic (LTL) with complex fairness con- i straints. These algorithmic metatheorems can be used to uniformly prove decidability with optimal (or near-optimal) complexity of various model checking problems over many classes of infinite-state systems that have been considered in the literature. In fact, many of these decidability/complexity results were not previously known in the literature. In the second part of the thesis, we study various model checking problems over subclasses of counter systems that were already known to be decidable. In particu- lar, we consider reversal-bounded counter systems (and their extensions with discrete clocks), one-counter processes, and networks of one-counter processes. We shall de- rive optimal complexity of various model checking problems including: model check- ing LTL, EF-logic, and first-order logic with reachability relations (and restrictions thereof). In most cases, we obtain a single/double exponential reduction in the previ- ously known upper bounds on the complexity of the problems. ii Acknowledgements First and foremost, I would like to express my deepest gratitude to my supervisor Leonid Libkin for his teaching, guidance and sound advice throughout my PhD studies. I have learnt a lot from him about many aspects of logic and automata, which I would unlikely have looked at otherwise. His teaching and discussions have always been a source of inspirations. Leonid has also helped me with many academic and personal issues that I encountered throughout my PhD studies. I cannot thank him enough for all these helps. I would also like to thank my co-supervisor Richard Mayr for his guidance and his inspiring introduction to the area of verification of infinite-state systems. I have lost count of how many hours of useful discussions about research which we have had in the last two years. I am honored to have Luke Ong and Colin Stirling as my PhD examiners. Thank you for going through the thesis and providing a lot of useful feedback. I would also like to thank Stefan G¨oller and Matthew Hague for the fruitful col- laborations that we have had throughout my PhD studies. I look forward to work- ing closely together again in the near future. In addition, I am grateful to many other colleagues presently or previously at the School of Informatics with whom I have had many useful discussions: Shunichi Amano, Lorenzo Clemente, Claire David, Christophe Dubach, Kousha Etessami, Floris Geerts, Luis Fabricio Wanderley Goes, Julian Gutierrez, Xibei Jia, Shuai Ma, Filip Murlak, Vasileios Porpodas, Juan Reutter, Rahul Santhanam, Karthik T. Sundararajan, Tony Tan, and Yinghui Wu. I would also like to thank colleagues from other institutes with whom I have had useful research dis- cussions throughout my PhD studies: Vince Barany, Pablo Barcelo, Eryk Kopczynski, Jerome Leroux, Christof L¨oding, Christophe Morvan, and Sanming Zhou. I would also like to acknowledge many teachers and colleagues from Australia who have been a great source of inspiration. In particular, I thank James Bailey, Harald Sondergaard, and Sanming Zhou for introducing me to theoretical computer science. I am also indebted to my mentors Mike Ciavarella and Benjamin Rubinstein for their personal and academic advice. My PhD studies have been made possible by Overseas Research Students Award Scheme and EPSRC grant E005039. Thank you for the generous support. Last but not least, I would like to thank my family and my girlfriend who have supported me personally throughout my PhD studies. This thesis is dedicated for you. iii Declaration I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other degree or professional qualification except as specified. (Anthony Widjaja To) iv Table of Contents 1 Introduction 1 1.1 Specificapproaches........................... 4 1.2 Genericapproaches ........................... 8 1.3 Contributions .............................. 9 1.4 Organizations .............................. 12 2 Preliminaries 13 2.1 Generalnotations ............................ 13 2.2 Automatatheory ............................ 14 2.2.1 Automataoverfinitewords. 15 2.2.2 Automata over ω-words .................... 22 2.2.3 Automataovertrees . 23 2.2.4 Automataoverinfinitetrees . 29 2.2.5 Pushdownautomataandcontext-freegrammars . 30 2.2.6 Parikh’sTheoremandsemilinearsets . 31 2.3 Computabilityandcomplexitytheory . 33 2.3.1 Computability ......................... 33 2.3.2 Complexitytheory . .. .. .. .. .. .. 34 2.4 Structuresandtransitionsystems . .. 37 2.5 Logicsandproperties . .. .. .. .. .. .. .. 38 2.5.1 Safety,liveness,andfairness . 39 2.5.2 Reachability and recurrent reachability . .. 40 2.5.3 FO:First-orderlogic . .. .. .. .. .. .. 41 2.5.4 FOREG(Reach): FO withregularreachability . 44 2.5.5 HM-logic:Hennessy-MilnerLogic . 45 2.5.6 EFREG-logic: HM-logicwithregularreachability . 46 2.5.7 CTL:ComputationTreelogic . 47 v 2.5.8 LTL:LinearTemporalLogic . 48 2.5.9 Otherlogics........................... 49 2.5.10 Modelcheckingcomplexity . 50 I Generic Approaches: Algorithmic Metatheorems 53 3 Word/Tree-automatic Systems 55 3.1 Word-automaticsystems . 56 3.1.1 Basicdefinitions ........................ 56 3.1.2 Examples............................ 58 3.1.3 Basicclosureandalgorithmicresults . 62 3.1.4 Negativeresults.. .. .. .. .. .. .. 67 3.2 Tree-automaticsystems. 69 3.2.1 Basicdefinitions ........................ 69 3.2.2 Examples............................ 71 3.2.3 Basicclosureandalgorithmicresults . 73 3.2.4 Negativeresults.. .. .. .. .. .. .. 75 3.3 Othergenericframeworks . 75 3.3.1 Length-preserving word-automatic transition systems..... 75 3.3.2 Presburgertransitionsystems. 77 3.3.3 Rationaltransitionsystems. 78 3.3.4 ω-wordautomatictransitionsystems. 80 4 Algorithmic metatheorems for recurrent reachability 81 4.1 Theword-automaticcase . 83 4.1.1 Maintheorem.......................... 83 4.1.2 Proofofthemaintheorem . 84 4.1.3 Asmallwitnessforrecurrentreachability . .. 91 4.1.4 Twoappetizerexamples . 92 4.2 Thetree-automaticcase. 95 4.2.1 Maintheorem.......................... 95 4.2.2 Preliminaries .......................... 96 4.2.3 Proofofthemaintheorem . 97 4.2.4 Anappetizerexample . 105 4.3 GeneralizedB¨uchiconditions. 106 vi 4.3.1 Word-automaticsystems . 107 4.3.2 Tree-automaticsystems. 107 4.3.3 Applications .......................... 108 4.4 Recurrentreachabilityviaapproximation . .... 110 5 Algorithmic metatheorems