PAC.C: A UNIFIED CONTROL ARCHITECTURE FOR PACKET AND CIRCUIT NETWORK CONVERGENCE A DISSERTATION SUBMITTED TO THE DEPARTMENT OF ELECTRICAL ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Saurav Das June 2012 © 2012 by Saurav Das. All Rights Reserved. Re-distributed by Stanford University under license with the author. This work is licensed under a Creative Commons Attribution- Noncommercial 3.0 United States License. http://creativecommons.org/licenses/by-nc/3.0/us/ This dissertation is online at: http://purl.stanford.edu/cm644gk6181 ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Nick McKeown, Primary Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Fouad Tobagi I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Guru Parulkar Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost Graduate Education This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives. iii To my parents iv Abstract Service providers today face several challenges. By all accounts Internet traffic is growing at 40-50% per year, necessitating costly upgrades to carrier infrastructure. Yet carriers do not see a commensurate increase in revenue, nor do they see relative reductions in capital and operational expenditures (Capex and Opex). Part of the problem is that service providers today separately own and operate two distinct networks: packet-switched IP/MPLS networks and circuit-switched TDM/WDM Transport networks. These networks are typically planned, designed and managed by separate divisions even within the same organization, leading to substantial management overhead, functionality/resource duplication, and increased Capex/Opex. This is clearly an expensive and inefficient way to run networks. There have been other attempts to unify the control and management of circuit and packet switched networks – essentially run one converged network instead of two – but none have taken hold. In this thesis, we propose a simple way to unify both types of network using an emerging concept called Software Defined Networking (SDN). SDN advocates the separation of data and control planes in networks; where the data-plane can be abstracted and represented to external software-controllers running a Network Operating System (NetOS). All network control functions are implemented as applications on top of the NetOS. The applications make control decisions that manipulate an annotated-map of the network presented to them and kept consistent by the NetOS. In turn the NetOS translates the map-manipulations into data-plane reality by programming the data-plane switch flow- tables via a switch-API like OpenFlow. As circuits can readily be defined as flows, the basic idea is that a common-flow abstraction fits well with both packet and circuit switches; provides a common paradigm for control using a common-map abstraction; and makes it easy to control, jointly optimize, and insert new functionality into the network. We call our SDN based solution pac.c for packet and circuit .network convergence. v We defined the common-flow abstraction as flow-tables that take the form of lookup- tables in packet switches and cross-connect tables in circuit switches. Together with a switch-API like OpenFlow, which we extended for circuit switches, it abstracts away layer and vendor specific hardware and interfaces, while providing a flexible forwarding plane for manipulation by a common control plane. The common-map abstraction was defined as one which provides full visibility into both packet and circuit switched networks, while abstracting away the complexity of state-dissemination from applications, allowing the latter to be implemented in a centralized manner. We built several prototypes to demonstrate and verify our architectural constructs. Our complete pac.c prototype emulates an inter-city carrier network, with access packet- switches in three cities, interconnected by hybrid packet-optical switches in the backbone, all under OpenFlow/SDN control. With this prototype, we verified the simplicity and extensibility of our architectural solution, compared to current state-of-the- art industry practice. More importantly, we presented qualitative architectural insights into why our solution fares better; and gave reasons why our control solution can succeed where GMPLS - the only previous attempt at unified control over packets and circuits - failed. Finally, we identified and demonstrated several new networking capabilities enabled at the packet-circuit interface, and offered architectural solutions to a number of deployment challenges faced by any new control solution. To demonstrate the benefits of reduced Total Cost of Ownership (TCO), we designed and analyzed today’s IP networks and contrasted it with a converged packet-circuit network based on our control architecture. We found nearly 60% Capex savings and 40% Opex savings. More importantly the savings are insensitive to varying traffic matrices and grow as we dimension the network for increasing traffic demand. And finally, we introduced the map-abstraction in MPLS networks and demonstrated how existing packet services like traffic engineering can be replicated in an SDN based network, without the complexities of the IP/MPLS control plane. In doing so we drew parallels with SDN based control for packets and circuits. vi To summarize, we have proposed, designed, analyzed and demonstrated a converged IP/MPLS/Optical network architecturally based on SDN. The common platform helps reduce expenditures, provides existing services, and helps carriers innovate by easing the introduction of new revenue-generating services that differentiate them from other carriers. Our work is in the early stages but with further development, if these ideas are adopted by service providers, its main impact would be that they can remain profitable as the Internet grows. As a result they would then have greater incentive to invest in their networks, which in-turn could benefit society immensely. vii Acknowledgements I am proud to call myself Nick McKeown’s student. The last few years have been an awe-inspiring journey. I have been a part, but I have also sat back and watched, as an idea was taken through inception, grown through hard work, passion and commitment and then unleashed into the larger networking community to spawn off what might one day become an industry of its own. Through it all, Nick still found time to guide me; to help me understand what makes a good networking systems researcher; and to instill confidence in me to challenge the status-quo, question pre-conceived notions, and dare to “change the world”. I am deeply grateful. Likewise it is hard to overstate the impact that my co-adviser Guru Parulkar has had on my growth and progress as a PhD student and reseracher. It is Guru that jump-started this work (literally by convincing Ciena’s CTO Stephen Alexander to work with me). It is Guru who has on innumerable occasions found clarity in my clouded thoughts and muddled ideas. And it is Guru who has given me direction and sustained my drive through these long years. It is an honor to be Guru’s shishya. I would also like to thank Fouad Tobagi for all the classes I have taken with him and his invaluable comments and feedback on my thesis. I reserve special thanks for three of my colleagues, each of whom has been instrumental in my thesis progress. Vinesh Gudla helped me put together the world’s first prototype for an OpenFlow controlled packet-optical network; Yiannis Yiakoumis, moved out of his comfort zone to work with me to build the complete pac.c prototype and emulated WAN testbed; and Ali Reza Sharafat helped me demonstrate for the first time an MPLS service like Traffic Engineering without the IP/MPLS control plane. Each of them worked with me at a crucial stage of my PhD and helped me make forward progress for which I cannot thank them enough. viii There are also a number of people in the industry I would like to acknowledge. Firstly I would like to thank Shinji Yamashita for believing in me and my story long before others did. Shinji-san was instrumental in getting my work off the ground by providing the resources and support I needed from Fujitsu, Japan. Likewise I am deeply grateful to Lyndon Ong, Daniel Getachew, Preeti Singh, Joe Berthold and all my friends in Ciena and Ciena-India, for working with me, training me, schooling me and supporting my work. I am also much obliged to Hans-Martin Foisel, Andreas Gladisch, Fritz-Joachim Westphal, Michael Duser, Christoph Gerlach and all my friends at T-Systems/ Deutsche Telekom for the wonderful summer I spent in Berlin learning how Transport Networks worked. Finally I would like to thank Ori Gerstel from Cisco for inspiration, guidance and encouragement to think along these lines. It has been a privilege to be a member of the McKeown Group. Every single group member has directly or indirectly had a positive impact on my work. This work would simply not have been possible without the individual and collective efforts of each group member – one only has to comb the references to realize the significance and magnitude of their contribution. Two key figures require special mention – Boris Grek and Jacob Sun. Boris is the whole reason why I started pursuing a PhD in the first place.