OFLOPS-Turbo: Testing the Next-Generation OpenFlow switch Charalampos Rotsos∗{, Gianni Antichi∗, Marc Bruyereyzx, Philippe Owezarskiyz, Andrew W. Moore∗ { School of Computing and Communications, Infolab 21, Lancaster University ∗Computer Laboratory, University of Cambridge yUniversite´ de Toulouse zCNRS, LAAS, France xDELL Inc. Abstract—The heterogeneity barrier breakthrough ment. OpenFlow introduces a reduction in the network achieved by the OpenFlow protocol is currently paced by control timescales which closely approximate flow control the variability in performance semantics among network timescales. As a result, OpenFlow switch implementation devices, which reduces the ability of applications to take complete advantage of programmable control. As a result, influences the control architecture of the network and control applications remain conservative on performance its overall performance. A switch which provides poor requirements in order to be generalizable and trade performance in the support of a protocol functionality performance for explicit state consistency in order to can become a bottleneck for architectures that rely heav- support varying performance behaviours. In this paper ily on the specific functionality; e.g., high latency in we argue that network control must be optimized towards network device capabilities and network managers and flow_stats_reply functionality of a switch can be application developers must perform informed design critical for traffic monitoring control applications. Protocol decision using accurate switch performance profiles. This support variability is equally critical for the consistency becomes highly critical for modern OpenFlow-enabled of the control plane. Most switches do not provide any 10 GbE optical switches which significantly elevate switch guarantees on the installation order and latency of a performance requirements. We present OFLOPS-Turbo, the integration of the OFLOPS switch evaluation platform, sequence of flow table updates; as a result, such updates with the OSNT platform, a hardware-accelerated traffic need to be carefully sequenced to not violate policy [8], generation and capture system supporting lossless 10 GbE effectively increasing the insertion latency. functionality. Using OFLOPS-Turbo, we conduct an The OpenFlow protocol, currently defining its 1:5 ver- evaluation of flow table manipulation capabilities in a sion, has greatly transformed its design over the years, representative collection of 10 GbE production OpenFlow switch devices and interpret the evolution of OpenFlow reflecting the deployment experience and requirements of support by comparison with historical data. a constantly widening range of network environments. As a result, the OpenFlow community requires a performance Keywords-SDN; OpenFlow; Open–Source; High Perfor- mance; Testing; NetFPGA. testing platform capable to co-evolve with the protocol and to support rapid prototyping of experimentation scenarios which highlight the impact of new protocols’ features. I. INTRODUCTION Furthermore, the increase in link capacity augments the Research on SDN technologies and primarily its pre- precision requirement for meaningful packet-level mea- dominant realisation, the OpenFlow protocol, has devel- surements. For example, 10 GbE links have become the oped a wide range of applications to improve network de-facto solution for the aggregation layer of modern functionality. OpenFlow control applications can improve datacenter networks, a first-class citizen of the SDN network management, monitoring and performance, while ecosystem, and require high measurement precision, on being backwards-compatible with data plane protocols the order of sub-µsec, for certain network application and end-host network stacks. As a result, within only a classes. An estimation error of 100µsec in the policy few years since the definition of the first version of the enforcement of a security application may translate to protocol, many vendors have introduced production-level unauthorized transmission of multiple KBs of sensitive support in an effort to transfer innovative research output information. This paper claims that in order to fully exploit to the market. OpenFlow protocol capabilities in production environ- Nonetheless, such continuous network innovation intro- ments, we require a flexible and high–precision open– duces a dilemma for network testing, as relevant evalu- source measurement platform. The openness and flexibility ation platforms remain closed and proprietary and pro- is important in order to establish an evolvable community– vide limited flexibility. To achieve compliant and func- based tool. tional equipment, effort must be put into all parts of the This paper presents an effort to enhance the measure- network-equipment life-cycle, from design to production. ment capabilities of the OFLOPS [15] switch evaluation The problem of network testing is further augmented in framework with support for the emerging protocol require- the OpenFlow protocol context. The protocol philosophy ment. Specifically, we present the OFLOPS integration introduces new performance challenges in network device with the NetFPGA-10G platform1 and the Open Source design, dissimilar to the challenges of traditional network equipment, and sets testing flexibility as a primary require- 1http://www.netfpga.org Network Tester (OSNT)2 platform [1]. OSNT enhances common performance comparison “vocabulary”, the SDN OFLOPS with sub-µsec precision and 20 Gbps full bi- community requires a high-precision performance testing directional traffic generation and capturing (when traffic platform, which will establish a common ground between thinning techniques are being used), providing a highly different implementations through standardised evaluation flexible and open platform for OpenFlow experimentation galleries, similar to the OFTest [12] compatibility testing at high data rates. platform. For the rest of this paper, we initially motivate the OFLOPS-Turbo design with a set of use cases (x II), C. IXP: Internet eXchange Point followed by a detailed presentation of the system design Internet eXchange Points (IXP) are a special network (x III). Furthermore, we present an evaluation of the flow infrastructure, providing inter-AS peering in a single lo- table manipulation capabilities for a representative set of cation. IXPs must support high data plane performance 10 GbE OpenFlow switches (x IV), discuss related work and in parallel provide programmability, advanced traffic (x V) and conclude our work (x VI). monitoring capabilities and scalability for hundreds of peers. The introduction of the SDN paradigm provides II. USE CASES unprecedented opportunities to improve the performance Programmable control is considered the holy grail for a of such network environments using commodity network number of common network research problems. In order devices [9]. IXP service providers, interested to invest to motivate the discussion for switch performance charac- in infrastructural SDN/OpenFlow support, require tools terisation, this section presents a set of SDN applications which allow them to identify the ultimate performer switch and discusses their performance requirements. for their needs. Furthermore, to facilitate the migration A. White-Box dilemma of the IXP to a programmable control paradigm, network device vendors design switches with hybrid capabilities White-box Ethernet switches [7] evolve into a com- and concurrent support for legacy control on a subset of pelling network device solution for a wide range of the Ethernet interfaces and OpenFlow control on the rest. network environments, primarily due to their enhanced Nonetheless, hybrid switches have limitations, primarily flexibility. White-box networking refers to the ability to deriving by the CAM configuration in order to abstract use generic, off-the-shelf switches and routers in the under a common flow abstraction multiple forwarding forwarding plane of a network. Such devices are open tables with variable matching capabilities. For example, to different OSes, while OpenFlow protocol, implemented a hybrid switch must provide matching table support for through OF-agents, is a popular control abstraction among the complete OpenFlow tuple, along with support for FIB them. The majority of white-box switch OSes are Linux- and ACL lookup tables. Research on SDN-based IXP based, primarily because of their support for a wide range control architectures focuses on the network controller of CPU architectures and free tools. Nonetheless, different and demonstrates fast processing of sophisticated routing realisations of the same standard commonly optimise policies and scalability for hundreds of IXP members. different performance aspects. In the case of white-box Nonetheless it remains equally important to understand switches, although vendors use the same chipset (most how an OpenFlow-enabled switching fabric behave during, 48x10Gbps + 4x40Gbps 1RU switch use the Broadcom for example, large bursts of flow table modifications. Trident chipset [3]), they employ a diverse device design approach (CPUs, memory, implementation). Technology III. OFLOPS-TURBO DESIGN adopters require flexible and easily configurable mecha- Measuring OpenFlow switch implementations is a chal- nisms to develop comparison studies between combina- lenging task in terms of characterisation accuracy and pre- tions of white-box switches (i.e., OSes and OF-agents). cision. Predominantly, production-level OpenFlow-switch This is helpful to acquire