Noname manuscript No. (will be inserted by the editor) OpenFlow Data Planes Performance Evaluation Leonardo C. Costa · Alex B. Vieira · Erik de Britto e Silva · Daniel F. Macedo · Luiz F. M. Vieira · Marcos A. M. Vieira · Manoel da Rocha Miranda Junior · Gabriel Fanelli Batista · Augusto Henrique Polizer · Andr´e Vin´ıciusGomes Santos Gon¸calves · Geraldo Gomes · Luiz H. A. Correia Received: date / Accepted: date Abstract Software-Defined Networks, in its essence, is the separation of the data and control planes of switching devices. The OpenFlow (OF) protocol is the most popular SDN protocol today, being available in many switches. This is due to the low implementation cost as well as the potential for inno- vative solutions in the network. Although OF is being used in many research papers and production networks, as far as we know, no work on the liter- ature performs an extensive evaluation of the OpenFlow data planes. This evaluation helps network administrators choose which switch to use in their networks. Meanwhile, researchers are made aware of the limitations of existing OF switches. This article evaluates the performance and maturity of OF 1.0 on eleven hardware and software switches using POX, and also of the newer OF 1.3 on five switches using the ONOS controller. Our findings indicate that the performance variations among OF switches are significant. Packet delays vary by one order of magnitude in the evaluated equipment. Meanwhile, there is no performance impact when changing the packet size. Hence, the results highlight that researchers must be aware of issues such as variable jitter, the number of matches in tables as well as missing OF match fields. Leonardo C. Costa, Alex Borges Vieira Computer Science Department Universidade Federal de Juiz de Fora, Juiz de Fora, Brazil E-mail: [email protected] Erik de Britto e Silva, Daniel F. Macedo, Luiz F. M. Vieira, Marcos A. M. Vieira, Manoel da Rocha Miranda Junior, Gabriel Fanelli Batista, Augusto Henrique Polizer Computer Science Department Universidade Federal de Minas Gerais, Belo Horizonte, Brazil Augusto H. Polizer, Andr´eV. G. S. Gon¸calves, Geraldo Gomes, Luiz H. A. Correia Computer Science Department Universidade Federal de Lavras, Lavras, Brazil 2 Leonardo C. Costa et al. Keywords OpenFlow, Software-Defined Networks, data plane, performance evaluation 1 Introduction Software-Defined Networking (SDN) is a technology that is reshaping the way networks are operated and developed [12]. SDN simplified the creation of new networking solutions, mainly because of the separation of the data and con- trol planes. The key benefits of this are the fast deployment of new services, which drive down the costs of network operators. Because of that, SDN is becoming a key architecture component on 5G systems [10] and Wireless En- terprise Networks [24]. SDN can even be used to support matching rules with domain names [36]. Furthermore, researchers are now able to perform tests in real environments, without affecting the traffic or the availability of produc- tion networks [12]. In this context, there is no doubt that the SDN paradigm attracted attention from both academia and industry. There are several sophisticated SDN implementations, such as P4 (Pro- gramming Protocol-independent Packet Processors) and POF (Protocol obliv- ious Forwarding) [5, 37]. However, OpenFlow is the most popular standard in production networks and research. This occurs because vendors can implement OpenFlow without significant difficulties. Despite being widely employed in industry and academia, as far as we know, no work has evaluated of the existing OpenFlow data planes. Such a study is important for operators because it provides an assessment that may guide the deployment of OpenFlow on their network. For researchers, this is useful to grasp the limitations of OpenFlow in current hardware. With this information in mind, better emulators and simulators can be built, by representing the performance and features of those switches more realistically . Many papers evaluate the performance of the OpenFlow Controllers [1, 4, 16, 38], however, most of them do not measure how the implementation of the data plane affects the network performance. Only [1, 4, 38] evaluate the data plane, however, their work does not measure the data plane separately from the control plane. It is important to properly investigate the data plane because the performance of the controller only affects the first packet in a flow. The data plane, however, will define the performance for the entire flow. Given this context, this article evaluates the performance and maturity of the main features of OpenFlow 1.0 and 1.3 on both hardware and software switches. We consider a wide number of switches, varying from various off-the- shelf equipment to open source implementations of software switches running on PCs or embedded platforms (e.g. home routers). In a controlled environ- ment, we systematically evaluate the implementation of the OpenFlow data planes by: (i) measuring the performance of the switch in terms of latency and jitter, (ii) assessing how the OpenFlow mode compares to the legacy L2 switching mode, and (iii) evaluating how OpenFlow operations (such as rule OpenFlow Data Planes Performance Evaluation 3 matches, packet rewrites, flowstats, and packetstats reports) perform. This article extends our previous work [8]1 by: { Adding OpenFlow 1.3 performance evaluation experiments; { Adding one more commercial switch to the comparison of OpenFlow 1.0 switches (Zodiac FX); { Testing a highly optimized version of Open vSwitch using the Intel DPDK suite of Linux kernel modifications, to understand how OS optimizations may improve the performance of software switches; { Adding high-performance switch evaluation; Our results show that OpenFlow's performance varies significantly. For example, packet delays vary by one order of magnitude among the evaluated switches, while the performance is not affected by the packet size. Moreover, we also note that some hardware switches perform as well as software switches, which may indicate a software implementation within the hardware. In sum, our article presents a glimpse into how OpenFlow is currently implemented on real devices. This is important for network administrators and academics to understand how OF switches work, allowing them to pick the best switch for their network. The remainder of this article is structured as follows. Section 2 presents related work. Section 3 describes the evaluation methodology. Sections 4 and 5 discuss the results for OpenFlow 1.0 and OpenFlow 1.3, respectively. Section 6 concludes the article and presents future work. 2 Related Work OpenFlow [23] is an open protocol in which a central element can inspect and modify the flow tables of the switching elements. This is possible via a standard API, where a computer (the control plane) sends queries and configuration messages to the network equipment (data plane) [12, 35]. OpenFlow is the most popular SDN platform in academia and industry [21]. Companies such as HP, NEC, Pronto, Extreme, Cisco, Brocade, Juniper, and Huawei already sell OF-capable devices [12]. Thus, it is important to measure the performance of OpenFlow-enabled switches. This section presents the state of the art on OpenFlow evaluation, which can be divided into categories. First, we highlight the studies that performed evaluation campaigns. Then, we tackle the works defining tools and frame- works for performance evaluation, followed by analytical models of OF perfor- mance. Finally, we map how the data plane performance influences the design of new OpenFlow features. 1 Paper available in http://netlab.ice.ufjf.br/index.php/OFperformance/ 4 Leonardo C. Costa et al. 2.1 Empirical Evaluation of OpenFlow Performance Bianco et al. [4] analyzed one virtual switch working in bridge mode. The authors compare the performance of this switch in OpenFlow mode with the performance of the same switch operating in legacy L2 switch mode, IP rout- ing, and Ethernet switching use cases. Despite that, no hardware-based switch was used in the experiments. Further, the performance results do not isolate the switch's performance from the controller performance. S¨unnenet al [38] also evaluate the performance of a single OpenFlow hard- ware switch. Some of the evaluation metrics used depend heavily on the con- troller's performance, which is not the focus of our work. In the present work, the data plane performance is independent of the controller employed because all the rules are written before the beginning of the experiment flows. In turn, Pfaff et al. [31] perform a comparison between three different OpenFlow switches, consisting of two hardware switches and a software Open vSwitch version. Again, the evaluation metrics also considered the influence of the controller. Osman et al. [29] evaluated how a lossy control link (e.g. wireless networks) affects the performance of OF switches. They have shown that the performance of OF switches is fairly resilient to a high packet loss by using a hybrid system in which the control can be delegated to local controllers when the link reliabil- ity to the central controller is degraded. Anew, no hardware-based switch was evaluated and authors did not evaluate the data plane performance. Authors only show that data plane can achieve reasonable performance, even under a high error rate control link. Mallon et al. [22] evaluated the capacity of OF controllers to handle a large number of flows, identifying which system design aspects contribute to the performance of the control plane. The authors show that performance bottlenecks arise due to the object-oriented nature of controllers, the use of global data structures that are changed by many threads/cores, as well as the inefficient use of caching and memory copies. Silva et al. [9] used existing SDN switches to evaluate and compare five load balancing policies. Their results showed that the txbytes(transmitted bytes) and cpuq-load(CPU usage and number of open connections) policies outper- formed the others.