NETWORK CONTROL PLANE SYNTHESIS AND VERIFICATION RYAN ANDREW BECKETT ADISSERTATION PRESENTED TO THE FACULTY OF PRINCETON UNIVERSITY IN CANDIDACY FOR THE DEGREE OF DOCTOR OF PHILOSOPHY RECOMMENDED FOR ACCEPTANCE BY THE DEPARTMENT OF COMPUTER SCIENCE ADVISER:DAVID WALKER SEPTEMBER 2018 © Copyright by Ryan Andrew Beckett, 2018. All rights reserved. Abstract Computer networks have become an integral part of modern infrastructure, and as the world con- tinues to become increasingly interconnected and more devices come online, the importance of networks will only continue to grow. A critical component of networks is a process called routing, whereby the network determines how to move data from point A to point B as changes occur dy- namically (e.g., when new devices connect or equipment fails). Routing is traditionally achieved through the manual configuration of one or more distributed protocols that exchange messages about available routes to different destinations. Manual configuration lets a network operator tune various low-level protocol parameters to accommodate the different economic-, performance-, and robustness-related objectives that they may have for the network. Unfortunately, the low-level na- ture of existing configuration primitives and the scale of modern networks makes it difficult for humans to predict the impact of configuration on all possible runtime behaviors of the network, often resulting in configuration bugs. This dissertation develops two complementary techniques for proactively finding and prevent- ing bugs in configurations. The first technique is verification. Given a collection of router config- urations and a high-level specification of what the network should do (e.g., certain devices should be reachable), verification aims to ensure that the configurations implement this high-level specifi- cation correctly for all possible network behaviors. To address this problem, we develop a formal model of network routing protocols and show how many common protocols can be translated to logic constraints that existing constraint solvers can solve to find (or prove the absence of) bugs. The second technique is synthesis. Given a high-level specification of what the network should do, synthesis aims to produce a collection of configurations that faithfully implement the specification for all possible dynamic network conditions. We develop a new high-level language for describing end-to-end network behavior and demonstrate an efficient synthesis algorithm that can generate correct configurations. Throughout the development of both techniques, we show the importance of “abstraction” in speeding up each technique by several orders of magnitude. iii Acknowledgements I want to start by thanking my advisor David Walker for all his teaching, patience, and guid- ance over the years. Dave has made an otherwise arduous journey a delight. It has also been a pleasure to work with many great co-authors over the years, including Jitu Padhye, Ratul Mahajan, Todd Millstein, Sharad Malik, Aarti Gupta, Jennifer Rexford, Michael Greenberg, Shuyuan Zhang, Kelvin Zou, and Ang Chen – all of whom have taught me a great deal through our interactions. The members of the PL group at Princeton have also been a great source of support whether it was getting feedback on ideas and presentations or just taking my mind off of things. I hope I have been an equal source of support in return. I am grateful for the advice and attention from my disserta- tion committee: Aarti Gupta, Ratul Mahajan, Jennifer Rexford, Nick Feamster, and David Walker. Their feedback and efforts have greatly improved the quality of this thesis. This dissertation would also not have been possible without my undergraduate advisor Paul Reynolds, who introduced me to programming languages and opened my eyes to the possibility that graduate school might be something I would enjoy pursuing. I would like to thank my family – my mother, father, and my brother Matthew – who have been nothing but supportive over the years (even when I call for help with silly problems). Finally, much of the work in this dissertation was supported by NSF grants 1111520, 1525936, and 1703493 as well as funding from Cisco, Facebook, and Google. iv Contents Abstract . iii Acknowledgements . iv List of Figures . xi 1 Introduction 1 1.1 Network Routing . .4 1.2 Network Configuration . .7 1.2.1 How Configuration Influences Routing . .7 1.2.2 The Cost of Misconfiguration . .8 1.3 Verification of Configurations . 10 1.4 Synthesis of Configurations . 12 1.5 Abstraction: Scaling Network Analysis . 13 1.6 Summary of Contributions . 14 1.7 Additional Comments . 15 2 Background 17 2.1 Data Plane . 17 2.1.1 Longest Prefix Matching (LPM) . 18 2.1.2 Access Control Lists (ACLs) . 18 2.2 Control Plane . 19 2.2.1 Static Routing . 20 v 2.2.2 Dynamic protocols . 21 2.2.3 iBGP . 25 2.2.4 Route Reflectors . 28 2.2.5 Route Redistribution . 28 2.2.6 Route Aggregation . 29 2.2.7 Multipath Routing . 29 3 Control Plane Verification 31 3.1 Related Work . 31 3.1.1 Analysis without formal semantic models. 32 3.1.2 Analysis with formal semantic models. 33 3.2 Overview of the approach . 36 3.3 Motivating Example . 38 3.4 Stable Routing Problem . 41 3.4.1 SRP Definition . 41 3.4.2 SRP Solution . 43 3.4.3 Modeling Common Routing Protocols . 44 3.5 Translation to SMT . 47 3.5.1 Overview . 47 3.5.2 Design Decisions and Limitations . 48 3.5.3 Encoding the Packet . 49 3.5.4 Encoding the Control Plane . 50 3.5.5 Encoding the Data Plane . 58 3.5.6 Encoding Properties . 58 3.6 Generalizing the Model . 59 3.6.1 Route redistribution . 59 3.6.2 Static route recursive lookup . 60 3.6.3 Aggregation . 60 vi 3.6.4 Multipath routing . 60 3.6.5 BGP community regexes . 61 3.6.6 iBGP . 61 3.6.7 Route reflectors . 62 3.6.8 Multi-exit discriminator (MED) . 62 3.7 Property Expressiveness . 63 3.7.1 Reachability and isolation . 63 3.7.2 Waypointing . 64 3.7.3 Bounded or equal path length . 64 3.7.4 Disjoint paths . 64 3.7.5 Forwarding loops . 64 3.7.6 Black holes . 65 3.7.7 Multipath consistency . 65 3.7.8 Neighbor or path preferences . 66 3.7.9 Load balancing . 67 3.7.10 Aggregation and leaking prefixes . 68 3.7.11 Local equivalence . 68 3.7.12 Full equivalence . 69 3.7.13 Stability and Uniqueness . 69 3.7.14 Wedgies . 70 3.7.15 Fault tolerance . 70 3.7.16 Fault-invariance testing . 70 3.8 Optimizations . 71 3.8.1 Hoisting . 71 3.8.2 Network Slicing . 73 3.9 Implementation . 74 3.10 Evaluation . 75 vii 3.10.1 Finding Errors in Real Configurations . 75 3.10.2 Verification Performance . 76 3.10.3 Optimization Effectiveness . 80 3.11 Summary . 80 4 Control Plane Verification with Abstraction 82 4.1 Related Work . 83 4.2 Overview . 84 4.3 Abstraction Definitions . 89 4.3.1 Effective Abstraction Conditions . 90 4.4 Control Plane Equivalence . 93 4.4.1 Loop-free protocols . 94 4.4.2 Static routing . 95 4.4.3 Forwarding path equivalence . 95 4.4.4 BGP with Loop Prevention . 96 4.4.5 Properties preserved . 100 4.4.6 Properties not preserved . ..