Technische Universität München Institut für Informatik Proving Theorems of Higher-Order Logic with SMT Solvers Sascha Böhme Vollständiger Abdruck der von der Fakultät für Informatik der Technischen Universität München zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzender: Univ.-Prof. Dr. Felix Brandt Prüfer der Dissertation: 1. Univ.-Prof. Tobias Nipkow, Ph.D. 2. Univ.-Prof. Dr. Andrey Rybalchenko Die Dissertation wurde am 24. 10. 2011 bei der Technischen Universität Mün- chen eingereicht und durch die Fakultät für Informatik am 05. 03. 2012 ange- nommen. Zusammenfassung Der Nachfrage nach erhöhter Beweisautomatisierung für den interaktiven Theorembe- weiser Isabelle folgend zeigt diese Arbeit, wie SMT-Löser den Stand der Technik ver- bessern können. SMT-Löser sind automatische Theorembeweiser mit Entscheidungs- verfahren für verschiedene Theorien wie zum Beispiel Gleichheit und lineare Arith- metik. Unsere Hauptbeiträge sind eine korrekte Übersetzung von Isabelles höherstufi- ger Logik in die erststufige Logik von SMT-Lösern sowie die effiziente Rekonstruktion von Beweisen, die vom SMT-Löser Z3 gefunden wurden. Mittels einer umfangreichen Evaluierung belegen wir, dass viele Theoreme nun automatisch und beinahe augen- blicklich bewiesen werden können, wofür früher zeitraubendes Nachdenken seitens des Benutzers nötig war. Das Überprüfen von Z3-Beweisen ist dank unserer aufwendi- gen Optimierungen schnell. Weitere Beiträge sind ein neues Werkzeug und eine neue Methode, um funktionale Korrektheit von C-Code im Zusammenspiel mit dem auto- matischen Programmbeweiser VCC zu beweisen. Wir demonstrieren ihre Eignung für Implementierungen aus der Praxis anhand von Baum- und Graphalgorithmen. Abstract Following the demand for increased proof automation in the interactive theorem prover Isabelle, this thesis describes how satisfiability modulo theories (SMT) solvers improve on the state of the art. SMT solvers are automatic theorem provers with decision pro- cedures for several theories such as equality and linear arithmetic. Our main contribu- tions are a sound translation from Isabelle’s higher-order logic to the first-order logic of SMT solvers, and efficient checking of proofs found by the solver Z3. Based on a large evaluation, we find that many theorems can now be proved automatically and almost instantaneously for which time-consuming user ingenuity was previously re- quired. Checking Z3’s proofs is fast thanks to our extensive optimizations. Further contributions are a new tool and methodology to verify the functional correctness of C code in conjunction with the automatic program verifier VCC. We demonstrate their suitability on real-world implementations of tree and graph algorithms. Acknowledgments First and foremost, I am deeply indepted to Tobias Nipkow who introduced me to the great world of theorem proving and who invited me to join his group. I appreciate his guidance and his questions as well as the possibility to always ask him for advice that together broadened my knowledge beyond the topics of this thesis. Many thanks go to all members of the Isabelle group at TUM: Clemens Ballarin, Ste- fan Berghofer, Jasmin Christian Blanchette, Lukas Bulwahn, Amine Chaieb, Johannes Hölzl, Dongchen Jiang, Cezary Kaliszyk, Alexander Krauss, OndˇrejKunˇcar, Peter Lam- mich, Lars Noschinski, Steven Obua, Andrei Popescu, Thomas Türk, Christian Urban and Markus Wenzel. They all contributed to the pleasant and productive atmosphere of the group and also to great after-work activities. To some of my colleagues I owe special thanks. Stefan Berghofer patiently spent hours in answering all my questions related to Isabelle, Makarius tought me to appreciate good software engineering, Jas- min motivated me to consider practical solutions instead of overly general and artificial ones, and Alex made me understand many things that I had not even known before. I thank Alex, Jasmin and Andrei for reading and commenting earlier versions of this the- sis. In addition, Jasmin deserves special thanks for his advices on language and written style that contributed to the readability of this thesis. Several parts of this thesis are the product of joint work. I have collaborated with Eyad Alkassar, Rustan Leino, Kurt Mehlhorn, Michał Moskal, Larry Paulson, Chris- tine Rizkallah, Wolfram Schulte, Thomas Sewell, Tjark Weber and Burkhart Wolff. In addition, Nikolaj Bjørner and Leonardo de Moura provided many insights into Z3. Mark Hillebrand and Dirk Leinenbach helped me understand VCC. I thank them all. Nikolaj invited me to a short research visit at MSR, and Michał invited me to a research internship at MSR, and I am grateful to both of them for these opportunities. Finally, my parents, my brother and Jana deserve special thanks for supporting me, especially pushing me to finish this thesis, and accepting that I spent only little time with them during the last months. This thesis was partly supported by BMBF under grant 01IS07008. Contents 1. Introduction1 1.1. Motivation....................................1 1.2. Contributions..................................2 1.3. Isabelle/HOL..................................4 1.3.1. Higher-Order Logic..........................4 1.3.2. LCF-Style Kernel and Proof Automation..............6 1.3.3. Sledgehammer.............................7 1.4. Satisfiability Modulo Theories Solvers....................8 1.4.1. Many-Sorted First-Order Logic....................8 1.4.2. SMT Solver Architecture........................9 1.5. Structure of This Thesis............................. 11 2. Integrating SMT Solvers as Oracles 13 2.1. Introduction................................... 13 2.2. Translation.................................... 14 2.2.1. Monomorphization........................... 15 2.2.2. Lambda-Lifting............................. 17 2.2.3. Explicit Applications.......................... 18 2.2.4. Erasure of Compound Types..................... 19 2.2.5. Separation of Formulas from Terms................. 20 2.2.6. Translation into MSFOL........................ 20 2.3. Theories and Interpreted Constants...................... 22 2.4. Extra-Logical Information........................... 25 2.5. Evaluation.................................... 26 2.5.1. Experimental Setup........................... 27 2.5.2. Benefits from SMT Solvers....................... 28 2.5.3. Benefits from Decision Procedures and Extra-Logical Information 30 2.6. Related Work.................................. 34 3. Reconstructing Z3 Proofs 37 3.1. Introduction................................... 37 3.2. Z3 Proof Format................................. 38 3.3. Proof Reconstruction.............................. 44 3.3.1. Local Definitions............................ 47 3.3.2. Skolemization.............................. 47 3.3.3. Conjunctions and Disjunctions.................... 49 ix x Contents 3.3.4. Theory Reasoning........................... 50 3.3.5. Rewriting................................ 51 3.4. Evaluation.................................... 53 3.4.1. Judgment Day Benchmarks...................... 54 3.4.2. SMT-COMP Benchmarks....................... 56 3.5. Related Work.................................. 59 4. Verifying Imperative Programs 61 4.1. Introduction................................... 61 4.2. VCC: An Automatic Program Verifier for C................. 61 4.2.1. The VCC Annotation Language................... 62 4.2.2. The VCC Memory Model....................... 64 4.2.3. Generating Verification Conditions.................. 65 4.2.4. Associating Errors to Source Code Locations............ 67 4.3. HOL-Boogie................................... 67 4.3.1. Debugging and Proving Verification Conditions.......... 69 4.3.2. Integration with VCC and Isabelle/HOL.............. 71 4.3.3. A Simpler Heap Model........................ 73 4.4. Case Study: Binary Search Trees........................ 74 4.4.1. Abstract Specification of Binary Search Trees............ 75 4.4.2. First Attempt: Combining Automatic and Interactive Proofs... 77 4.4.3. Second Attempt: Purely Automatic Proofs............. 78 4.5. Abstract Cooperation.............................. 78 4.6. Case Study: Maximum Cardinality Matching in Graphs.......... 82 4.6.1. Data Structures of the MCM Checker................ 84 4.6.2. Specification for the MCM Checker................. 84 4.6.3. Abstracting the Checker Specification and Combining the Results 85 4.7. Related Work.................................. 86 5. Conclusion 89 5.1. Proof Automation................................ 89 5.2. Program Verification.............................. 90 5.3. Future Work................................... 90 A. Z3 Proof Rule Names 93 B. A Binary Search Tree Implementation in C 95 List of Figures 101 List of Tables 103 Bibliography 105 Chapter 1. Introduction Contents 1.1. Motivation...................................1 1.2. Contributions.................................2 1.3. Isabelle/HOL.................................4 1.4. Satisfiability Modulo Theories Solvers..................8 1.5. Structure of This Thesis........................... 11 1.1. Motivation Interactive theorem provers such as Isabelle/HOL [130], HOL4 [78] and Coq [20] rely on automatic methods to discharge simple proof steps without much user guidance. More precisely, showing that a theorem holds with such a prover requires the user to find a proof sketch whose intermediate steps can be established by the prover’s au- tomatic methods. These methods, too, must be chosen and, in most cases, also