VERSION 2.01 1 Software-Defined Networking: A Comprehensive Survey Diego Kreutz, Member, IEEE, Fernando M. V. Ramos, Member, IEEE, Paulo Verissimo, Fellow, IEEE, Christian Esteve Rothenberg, Member, IEEE, Siamak Azodolmolky, Senior Member, IEEE, and Steve Uhlig, Member, IEEE Abstract—The Internet has led to the creation of a digital Index Terms—Software-defined networking, OpenFlow, net- society, where (almost) everything is connected and is accessible work virtualization, network operating systems, programmable from anywhere. However, despite their widespread adoption, networks, network hypervisor, programming languages, flow- traditional IP networks are complex and very hard to manage. based networking, scalability, dependability, carrier-grade net- It is both difficult to configure the network according to pre- works, software-defined environments. defined policies, and to reconfigure it to respond to faults, load and changes. To make matters even more difficult, current networks are also vertically integrated: the control and data I. INTRODUCTION planes are bundled together. Software-Defined Networking (SDN) The distributed control and transport network protocols run- is an emerging paradigm that promises to change this state of affairs, by breaking vertical integration, separating the network’s ning inside the routers and switches are the key technologies control logic from the underlying routers and switches, promoting that allow information, in the form of digital packets, to (logical) centralization of network control, and introducing the travel around the world. Despite their widespread adoption, ability to program the network. The separation of concerns traditional IP networks are complex and hard to manage [1]. introduced between the definition of network policies, their To express the desired high-level network policies, network implementation in switching hardware, and the forwarding of traffic, is key to the desired flexibility: by breaking the network operators need to configure each individual network device control problem into tractable pieces, SDN makes it easier to separately using low-level and often vendor-specific com- create and introduce new abstractions in networking, simplifying mands. In addition to the configuration complexity, network network management and facilitating network evolution. environments have to endure the dynamics of faults and In this paper we present a comprehensive survey on SDN. We adapt to load changes. Automatic reconfiguration and response start by introducing the motivation for SDN, explain its main mechanisms are virtually non-existent in current IP networks. concepts and how it differs from traditional networking, its roots, and the standardization activities regarding this novel paradigm. Enforcing the required policies in such a dynamic environment Next, we present the key building blocks of an SDN infrastructure is therefore highly challenging. using a bottom-up, layered approach. We provide an in-depth To make it even more complicated, current networks are analysis of the hardware infrastructure, southbound and north- also vertically integrated. The control plane (that decides how bound APIs, network virtualization layers, network operating to handle network traffic) and the data plane (that forwards systems (SDN controllers), network programming languages, and network applications. We also look at cross-layer problems such traffic according to the decisions made by the control plane) as debugging and troubleshooting. In an effort to anticipate the are bundled inside the networking devices, reducing flexibility future evolution of this new paradigm, we discuss the main and hindering innovation and evolution of the networking ongoing research efforts and challenges of SDN. In particular, infrastructure. The transition from IPv4 to IPv6, started more we address the design of switches and control platforms – with than a decade ago and still largely incomplete, bears witness arXiv:1406.0440v3 [cs.NI] 8 Oct 2014 a focus on aspects such as resiliency, scalability, performance, security and dependability – as well as new opportunities for to this challenge, while in fact IPv6 represented merely a carrier transport networks and cloud providers. Last but not protocol update. Due to the inertia of current IP networks, least, we analyze the position of SDN as a key enabler of a a new routing protocol can take 5 to 10 years to be fully software-defined environment. designed, evaluated and deployed. Likewise, a clean-slate approach to change the Internet architecture (e.g., replacing D. Kreutz and F. Ramos are with the Department of Informatics of IP), is regarded as a daunting task – simply not feasible in Faculty of Sciences, University of Lisbon, Lisbon, 1749-016 Portugal e-mail: [email protected], [email protected]. practice [2], [3]. Ultimately, this situation has inflated the P. Ver´ıssimo is with the Interdisciplinary Centre for Security, Reliability capital and operational expenses of running an IP network. and Trust (SnT), University of Luxembourg, 4 rue Alphonse Weicker, L-2721 Software-Defined Networking (SDN) [4], [5] is an emerging Luxembourg. e-mail: [email protected]. C. Esteve Rothenberg is with the School of Electrical and Com- networking paradigm that gives hope to change the lim- puter Engineering (FEEC, University of Campinas, Brazil. e-mail: itations of current network infrastructures. First, it breaks [email protected]. the vertical integration by separating the network’s control S. Azodolmolky is with Gesellschaft fur¨ Wissenschaftliche Datenverar- beitung mbH Gottingen¨ (GWDG), Am Faßberg 11, 37077 Gottigen,¨ Germany. logic (the control plane) from the underlying routers and e-mail: [email protected]. switches that forward the traffic (the data plane). Second, S. Uhlig is with Queen Mary University of London. is with Queen Mary, with the separation of the control and data planes, network University of London, Mile End Road, London E1 4NS, United Kingdom. e-mail [email protected]. switches become simple forwarding devices and the control Manuscript received May 31, 2014. logic is implemented in a logically centralized controller (or VERSION 2.01 2 Network Applica4on(s) SDN momentum was strong enough to make Google, Face- book, Yahoo, Microsoft, Verizon, and Deutsche Telekom fund Open northbound API Open Networking Foundation (ONF) [10] with the main goal Controller Pla+orm of promotion and adoption of SDN through open standards development. As the initial concerns with SDN scalability Open southbound API were addressed [11] – in particular the myth that logical centralization implied a physically centralized controller, an issue we will return to later on – SDN ideas have matured and evolved from an academic exercise to a commercial success. Google, for example, has deployed a software-defined network to interconnect its data centers across the globe. This production network has been in deployment for 3 years, helping the company to improve operational efficiency and sig- Data forwarding elements (e.g., OpenFlow switches) nificantly reduce costs [8]. VMware’s network virtualization Network Infrastructure platform, NSX [12], is another example. NSX is a commercial solution that delivers a fully functional network in software, Fig. 1. Simplified view of an SDN architecture. provisioned independent of the underlying networking devices, entirely based around SDN principles. As a final example, the world’s largest IT companies (from carriers and equipment network operating system1), simplifying policy enforcement manufacturers to cloud providers and financial-services com- and network (re)configuration and evolution [6]. A simplified panies) have recently joined SDN consortia such as the ONF view of this architecture is shown in Figure1. It is important and the OpenDaylight initiative [13], another indication of the to emphasize that a logically centralized programmatic model importance of SDN from an industrial perspective. does not postulate a physically centralized system [7]. In fact, A few recent papers have surveyed specific architectural the need to guarantee adequate levels of performance, scala- aspects of SDN [14], [15], [16]. An overview of OpenFlow bility, and reliability would preclude such a solution. Instead, and a short literature review can be found in [14] and [15]. production-level SDN network designs resort to physically These OpenFlow-oriented surveys present a relatively simpli- distributed control planes [7], [8]. fied three-layer stack composed of high-level network services, The separation of the control plane and the data plane controllers, and the controller/switch interface. In [16], the can be realized by means of a well-defined programming authors go a step further by proposing a taxonomy for SDN. interface between the switches and the SDN controller. The However, similarly to the previous works, the survey is limited controller exercises direct control over the state in the data- in terms of scope and it does not provide an in-depth treatment plane elements via this well-defined application programming of fundamental aspects of SDN. In essence, existing surveys interface (API), as depicted in Figure1. The most notable lack a thorough discussion of the essential building blocks example of such an API is OpenFlow [9], [10]. An OpenFlow of an SDN such as the network operating systems, program- switch has one or more tables of packet-handling rules (flow ming languages, and interfaces. They also fall short on the table). Each rule