A Framework for Automated Concurrency Verification
Total Page:16
File Type:pdf, Size:1020Kb
A Framework for Automated Concurrency Verification Matthew Bernard Windsor Ph.D University of York Computer Science April 2019 Abstract Reasoning systems based on Concurrent Separation Logic make verifying complex concurrent algorithms readily possible. Such algorithms contain subtle protocols of permission and resource transfer between threads; to cope with these intricacies, modern concurrent separation logics contain many moving parts and integrate many bespoke logical components. Verifying concurrent algorithms by hand consumes much time, effort, and expertise. As a result, computer-assisted verification is a fertile research topic, and fully automated verification is a popular research goal. Unfortunately, the complexity of modern concurrent separation logics makes them hard to automate, and the proliferation and fast turnover of such logics causes a downward pressure against building tools for new logics. As a result, many such logics lack tooling. This dissertation proposes Starling: a scheme for creating concurrent pro- gram logics that are automatable by construction. Starling adapts the existing Concurrent Views Framework for sound concurrent reasoning systems, overlay- ing a framework for reducing concurrent proof outlines to verification conditions in existing theories (such as those accepted by off-the-shelf sequential solvers). This dissertation describes Starling in a bottom-up, modular manner. First, it shows the derivation of a series of general concurrency proof rules from the Views framework. Next, it shows how one such rule leads to the Starling framework itself. From there, it outlines a series of increasingly elaborate frontends: ways of decomposing individual Hoare triples over atomic actions into verification conditions suitable for encoding into backend theories. Each frontend leads to a concurrent program logic. Finally, the dissertation presents a tool for verifying C-style concurrent proof outlines, based on one of the above frontends. It gives examples of such outlines, covering a variety of algorithms, backend solvers, and proof techniques. 2 Contents Abstract 2 Contents 3 Listings 5 Acknowledgements 7 Declaration 9 Collaboration history . 9 Publication history . 10 1 Introduction 11 1.1 Contribution . 13 1.2 Structure . 14 2 Background 15 2.1 Atomicity and atomic actions . 15 2.2 Formal methods . 16 2.3 Floyd/Hoare-style program logics . 19 2.4 The Views framework . 24 2.5 Automating program-logic proofs . 30 3 Proof-Specific Views Instances 33 3.1 Disposable views instances . 34 3.2 Decomposing proof outlines . 34 3.3 Case study: Peterson’s algorithm . 37 3.4 Free views instances . 40 3.5 Axiomatisation templates . 41 3.6 Automation-friendly templates . 42 3.7 Summary . 48 4 The Starling Framework 49 4.1 Backends . 51 4.2 Frontends . 55 3 CONTENTS 4.3 Syntactic definers . 56 4.4 µStarling . 58 4.5 Summary . 59 5 Adding Local-State Reasoning 61 5.1 The Local Views Framework .......................... 62 5.2 The LVF programming language . 68 5.3 Soundness of the LVF . 71 5.4 Embedding the LVF in the CVF . 75 5.5 Local states in backends . 77 5.6 loStarling . 79 5.7 Atomicity of local actions . 82 5.8 Summary . 84 6 A Practical Starling Frontend 85 6.1 Practical views semigroups . 86 6.2 View expressions . 89 6.3 Structured propositions . 95 6.4 Guarded views . 103 6.5 Guarded syntactic definers . 107 6.6 The gStarling frontend . 113 6.7 Iterated views and other extensions . 120 6.8 Summary . 123 7 Automating Proofs with Starlingtool 125 7.1 The Cview language . 125 7.2 The Starlingtool frontend . 131 7.3 Summary . 138 8 Case Studies and Validation 139 8.1 Case studies . 139 8.2 Testing Starlingtool ............................... 162 8.3 Mechanisation . 163 8.4 Related work . 164 8.5 Summary . 168 9 Conclusions and Further Work 169 9.1 Conclusions . 169 9.2 Further work . 170 A Additional Definitions 177 A.1 Trivial definitions . 177 A.2 Backend interfaces . 177 A.3 Syntactic definers . 179 A.4 Derivation of outline flattening . 179 4 A.5 Operations on view lists . 182 A.6 Cview ....................................... 182 A.7 Full Circular Buffer . 185 B Proofs 187 B.1 The free views instance . 187 B.2 Views algebras . 187 B.3 Local Views Framework . 199 Glossaries 201 Glossary . 201 Symbols . 204 Bibliography 207 Listings 2.1 Owicki–Gries multiple counter (incomplete) . 21 2.2 Owicki–Gries multiple counter (corrected) . 21 7.1 Ticket lock (static) . 132 8.1 Peterson’s algorithm: Cview proof using Z3 . 141 8.2 Circular buffer (static) . 144 8.3 ARC (static) . 147 8.4 ARC (GRASShopper auxiliary) . 149 8.5 ARC (dynamic) . 149 8.6 Auxiliary GRASShopper module clhmodule.spl............. 153 8.7 CLH lock: Cview proof using GRASShopper . 153 8.8 Ticket lock: modified constraint set for use with HSF . 156 8.9 Ticket lock: induced specification failure . 158 8.10 Ticket lock: induced code failure . 158 8.11 Peterson’s algorithm: induced code failure . 160 8.12 Attempted proof of a flawed mutual exclusion algorithm [1] . 160 8.13 Ticket lock (CAPER) . 166 8.14 Peterson (Threader) . 167 Full circular buffer . 185 5 Acknowledgements This work would not have been possible without the input and guidance from my two su- pervisors, Mike Dodds (from 2014 to 2017) and Radu Calinescu (2017 onwards). Both have gone above and beyond the call of duty to help me in my journey — especially in times where the path has been unclear, or fraught. I am grateful to them for not giving up on me, especially when times have been difficult — and whenever I was a challenging student! I’m grateful to Jeremy Jacob and Ana Cavalcanti for several reasons. Jeremy, my internal assessor, has been a reassuring and stabilising force throughout my PhD journey; Ana, my undergraduate project supervisor in 2014, helped me to start the journey in the first place. My gratitude goes back earlier, though: Jeremy and Ana’s Programming: Correctness by Construction course taught me the beauty and elegance of formal methods, and was a fundamental influence on the direction I have taken ever since. I also thank Matthew Parkinson for his invaluable contributions throughout the PhD. This dissertation builds extensively on both his published work and insights he and Mike have shared with me over the four years. When I did not understand part of the Views framework, or was struggling with a piece of Starling’s theory, ‘MattP’ was often the first port of call. Many more people have helped me on my journey, and I cannot thank them all in one page. I especially thank: Ben Simner for his work as an undergraduate intern on Starlingtool; Claudio Russo for hosting me as an intern at Microsoft Research, Cambridge; John Wicker- son and Alastair Donaldson, my bosses at Imperial College, London; my office-mates Ivaylo, Rudy, Michael, Gia, Tim, and José; and all of my friends who have helped me survive my PhD. I gratefully note that an EPSRC Doctoral Training Grant funded my PhD. I also thank the Interface Reasoning for Interacting Systems project, which funds my post-doctoral work. Finally, I thank my family, especially my parents Angela (1966–2018) and Stephen. They never gave up on me, even at my worst and my lowest, always seeing the best in me. I hope this dissertation does them proud. 7 ACKNOWLEDGEMENTS To Mum 8 Declaration I, Matthew Bernard Windsor, declare: that this dissertation presents original work; that I identify parts of said work developed in collaboration, and the collaborating parties, below this declaration; that I am the sole author of the dissertation, though parts of its text draw heavily from publications written in collaboration with other authors, which I also identify below this declaration; that I acknowledge all sources, including said publications, as refer- ences; and that this work has not been presented for a previous award at this, or any other, university. Collaboration history • Starling, in general, develops a previously unpublished idea by Matthew Parkinson and Mike Dodds; • The definitions of the thread-local semantic judgement and multi-threaded action judgement in § 5.2 stem from conversations between the author and Matthew Parkinson in March 2018. The local-state action judgement is original work, but based on a thread- aware version suggested by Matthew Parkinson; • Discussion of Starling’s relationship with thread modularity, and its contrasts with that of Owicki-Gries, stem from a conversation with Andreas Podelski at CAV’2017; • Starlingtool is mainly the author’s own work, but contains code written by Mike Dodds (including initial GRASShopper support, and examples including the lock-coupling list and allocatable ARC); Matthew Parkinson (including efficiency optimisations, bug-fixes, and initial phase timing support); and Ben Simner (including colourised error formatting, optimisation passes, continuous integration and regression testing infrastructure, initial support for iterated views, and work on the ARC proof )1; • The mechanisation is entirely the author’s own work, except for the adapted use of parts of the original mechanisation of the Starling proof rule contributed by Mat- thew Parkinson. The mechanisation’s treatment of the Concurrent and Local Views frameworks take inspiration, but not direct code, from the original Concurrent Views Framework Coq mechanisation. 1See also: https://github.com/MattWindsor91/starling-tool/graphs/contributors. 9 DECLARATION Publication history The following parts of the dissertation have been previously published, in some form, as the paper Starling: Lightweight Concurrency Verification with Views [2]: • the deriving-views proof rule (Definition 3.12), and its derivation from Views axiom soundness; • discussion of Starlingtool’s Z3 and GRASShopper backends (included here in the relev- ant case studies); • the ARC and CLH lock proofs (Listing 8.3, Listing 8.5, Listing 8.7) 10 Chapter 1 Introduction.