Tragedy of the routing table: An analysis of collective action amongst Internet network operators by Stephen Robert Woodrow B.Sc. Computer Engineering, University of Manitoba (2007) Submitted to the Engineering Systems Division and Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degrees of Master of Science in Technology and Policy and Master of Science in Electrical Engineering and Computer Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 2011 c Massachusetts Institute of Technology 2011. All rights reserved. Author........................................................................... Engineering Systems Division Department of Electrical Engineering and Computer Science June 10, 2011 Certified by . Karen R. Sollins Principal Research Scientist, Computer Science and Artificial Intelligence Laboratory Thesis Supervisor Accepted by . Leslie A. Kolodziejski Professor of Electrical Engineering Chair, Department Committee on Graduate Students Accepted by . Dava J. Newman Professor of Aeronautics and Astronautics and Engineering Systems Director, Technology and Policy Program 2 Tragedy of the routing table: An analysis of collective action amongst Internet network operators by Stephen Robert Woodrow Submitted to the Engineering Systems Division and Department of Electrical Engineering and Computer Science on June 10, 2011, in partial fulfillment of the requirements for the degrees of Master of Science in Technology and Policy and Master of Science in Electrical Engineering and Computer Science Abstract This thesis analyzes and discusses the effectiveness of social efforts to achieve collective action amongst Internet network operators in order to manage the growth of the Internet routing table. The size and rate of growth of the Internet routing table is an acknowledged challenge impeding the scalability of our BGP interdomain routing architecture. While most of the work towards a solution to this problem has focused on architectural improve- ments, an effort launched in the 1990s called the CIDR Report attempts to incentivize route aggregation using social forces and norms in the Internet operator community. This thesis analyzes the behavior of Internet network operators in response to the CIDR Report from 1997 to 2011 to determine whether the Report was effective in achieving this goal. While it is difficult to causally attribute aggregation behavior to appearance on the CIDR report, there is a trend for networks to improve their prefix aggregation following an appearance on the CIDR Report compared to untreated networks. This suggests that the CIDR Report did affect network aggregation behavior, although the routing table contin- ued to grow. This aggregation improvement is most prevalent early in the study period and becomes less apparent as time goes on. Potential causes of the apparent change in efficacy of the Report are discussed and examined using Ostrom’s Common Pool Resource frame- work. The thesis then concludes with a discussion of options for mitigating routing table growth, including the continued use of community forces to better manage the Internet routing table. Thesis Supervisor: Karen R. Sollins Title: Principal Research Scientist, Computer Science and Artificial Intelligence Labora- tory 3 4 Acknowledgments This thesis marks the culmination of my journey at MIT—arguably not the smoothest (or most direct) of sails. While often challenging, my time at MIT has also been stimulat- ing and broadening, and has caused me to grow in ways that I am not sure I would have needed to or been able to elsewhere. I am grateful for the opportunity I have had to deeply study and grow even more excited about the Internet—a space where the interplay between technology, policy, and business is incredibly rich and interesting. Participating in network operator communities, talking with experts directly, reading papers, writing code, and ana- lyzing data have all contributed to this interest and excitement, and to this thesis. I am also grateful for this opportunity to obtain the CS education that I “missed” during my under- graduate computer engineering career, while simultaneously learning about other ways to think about and address the world, its institutions, and its challenges through TPP. While probably contributing to what I perceived to be somewhat uncertain seas of my graduate school career, I must ultimately thank my adivsors Karen and Dave for allowing me the flexibility and freedom to spend my time fully on what would eventually and some- what unexpectedly become my thesis research. You might say I was given enough rope to hang myself, but thankfully, with their help, I managed to avoid such a fate. With this long leash and the problems that inevitably occur when doing things for the first time, I gained an appreciation for what it really means to do research for a living, including the most important first steps of picking a good question to answer and being excited about the work. I also learned how important it is to talk to people about your work, for feed- back, criticism, new ideas, or in the case of Internet operators, an understanding of how the Internet actually works, rather than how we might learn about it at the university. Finally (last but not least), the obligatory round of thanks to and acknowledgment of those who helped and influenced me during this effort: My first and greatest thanks are to Karen Sollins, my adivsor. She first helped me by hiring me when I was searching for an RA position related to my interests in the Internet in November of 2008, and has been a constant source of encouragement, understanding, and good advice on both my career as a graduate student and beyond since then. Karen also took efforts to involve me in a number of interesting projects and has offered much feedback and helpful perspectives on this work, for which I am greatly appreciative. I would also like to thank David Clark for his feedback and advice as he became in- volved in my project as I changed to focus on what would ultimately become this thesis. His experience with and insights about Internet policy and economics together with his technical knowledge was very helpful, as was his suggestion to use Elinor Ostrom’s CPR framework to think about the Internet routing table. Steven Bauer and I first met when I was searching for an RA, but we ultimately became better friends when he returned to MIT to work on another project. I would argue that Steve has helped keep me sane, as he has been a great sounding board, a source of strong feedback, and an advocate, as well as someone to “geek out” with on topics such as Python, TCP, and startups. Thanks for all these things, Steve. Arthur Berger has been invaluable in helping me plan my investigation and analysis 5 and offering suggestions and assistance in interpreting the resulting data, for which I am immensely grateful. Arthur and I also had several discussions about my professional aspi- rations that were helpful in my deciding what to do next, and I appreciate his thoughts and advice to this end. I would like to thank a number of members of the Internet community for volunteering to talk with me about some of the various questions I have had about the Internet. In par- ticular, these individuals’ insights and opinions were extremely valuable in understanding how the Internet works in practice, which is often quite different from theory and academic viewpoints. Thank you Martin Hannigan, Geoff Huston, Tony Li, Bruce Davie, and Jeff Schiller. Thanks to Susan Perez, the ANA administrative assistant, for always being incredibly helpful and cheerful. I must acknowledge my officemates in 32-G806 for always providing good company and discussion—especially Jesse Sowell and Chintan Vaishnav—who have offered research advice and discussion when asked, and passionate conversation always. Thanks are due to my family, and in particular my mother, for their support and encour- agement through this process. It started with the drive down to Massachusetts in 2008 and has continued to the present with the Calvin and Hobbes comic strip that has graced my inbox daily since April. Finally, I acknowledge the financial support of the Intel Corporation, the MIT Com- munications Futures Program, the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Office of Naval Research under contract N00014-08-1-0898. 6 Contents 1 Introduction 13 1.1 Context and motivation . 14 1.2 The case of the CIDR Report . 17 1.3 The routing table as a Common Pool Resource . 18 1.4 Contributions of this thesis . 22 1.5 Roadmap for remaining chapters . 22 2 Background 25 2.1 Routing and Internet operations . 25 2.2 CIDR-ization of the Internet and the CIDR Report . 36 2.3 Coordinating Internet operations . 44 3 Analyzing the CIDR Report 47 3.1 Analytical approach . 47 3.2 Data sources & data collection . 49 3.3 Data preprocessing . 51 3.4 Implementing the aggregation report . 53 3.5 Analyzing the aggregation report . 58 4 CIDR Report characteristics & influence on AS behavior 63 4.1 Data and methodological quality . 64 4.2 Characteristics of the CIDR Report . 68 4.3 Analysis of AS behavior after appearing on the CIDR Report . 76 5 The CIDR Report as a mechanism for inducing collective action 91 5.1 Is the CIDR Report accurate? . 91 5.2 Is the CIDR Report effective? . 94 5.3 What caused the decrease in treatment effect over time? . 96 5.4 Learning from the CIDR Report and the routing table CPR system . 100 6 Related work 105 6.1 Analyses of routing table growth & deaggregation . 105 6.2 Improving interdomain routing scalability . 107 6.3 Non-technical solutions to routing table growth & deaggregation . 110 6.4 The Internet operations community and the social forces within . 111 7 7 Managing future routing table growth 113 7.1 Is routing table growth a problem? .