Bachelor Informatica Open-source network operat- ing systems: feature evalua- tion of SONiC Erik Puijk, 11017651 June 7, 2018 Informatica | Universiteit van Amsterdam Supervisor(s): Dr. Paola Grosso,Lukasz Makowski MSc Signed: 2 Abstract Open network switches are increasing in popularity and allow the deployment of dif- ferent open-source network operating systems (NOS). In contrast to locked-in switches, open switches with an open-source NOS have been tested less extensively as they are quite new phenomena. This thesis examines whether open switches with an open-source network operating system, namely SONiC, can be deployed successfully to perform fundamental networking operations. Furthermore, it examines for what use cases SONiC is suitable and beneficial. We experiment with open switches in various topologies to examine the deploy- ment of fundamental OSI Layer 2 and Layer 3 networking features. We conclude that all SONiC-supported features we tested can be deployed successfully. Moreover, we examine several use cases of open switches with SONiC and conclude that SONiC is most suitable for use cases in large-scale data centers and enterprise networks. 3 4 Contents 1 Introduction 7 2 Open networking background 9 2.1 Open switches . .9 2.2 Network operating systems . 10 2.2.1 ASIC communication . 10 2.2.2 ASIC control module . 10 2.3 SONiC . 11 2.3.1 Quagga . 12 3 Networking features 15 3.1 Layer 2 features . 15 3.2 Layer 3 features . 17 4 Experiments 19 4.1 Preparatory phase . 19 4.1.1 Mellanox SN2100 and ONIE . 20 4.1.2 Arista 7050QX-32S and Aboot . 20 4.2 Feature tests . 21 4.2.1 Layer 2 features . 21 4.2.2 Layer 3 features . 26 4.2.3 Result overview . 28 5 Discussion 29 5.1 Ease of use . 29 6 Use case scenarios for open switches 31 6.1 Current use cases . 31 6.2 Example use cases . 31 7 Conclusions 35 7.1 Future research . 35 8 Acknowledgements 37 9 Bibliography 39 10 Appendix 41 5 6 CHAPTER 1 Introduction Network switches are essential in computer networks, to connect devices and forward frames between them on the OSI data link layer (Layer 2)[1]. Some switches are also capable of OSI Layer 3 features, such as the routing of segments among networks using IP. In this thesis, we discuss exactly this category of devices. Traditionally, switches are sold with locked-in hardware with a pre-installed network operating system (NOS) on them, without the possibility for a network administrator to install third-party NOS's or other software on them. An open switch, in contrast, does allow the user to install another operating system on the device. Open switches thus give network administrators more possibilities to customize the switch to their own needs, possibly explaining their rising popularity. Another advantage of this category of switches is the reduction of expenses, due to the possibility to install low-cost software. In the past, this cost reduction would be compensated by an increase in operational costs, because using the switches would require hiring external Linux-expertise to configure the switches [2]. However, as Linux expertise has grown over the past few years, this barrier for using open switches has become much thinner. Also, large manufacturers such as Dell and HP have been developing open switches on which NOS's like Cumulus Networks or Pica8 are already installed [3]. This removed another obstacle from companies using the switches because no manual installation is required. The increasing popularity raises questions about the suitability for use in real networks, the ease of use and how their feature sets compare to those of traditional switches. Considering that open switches are relatively new compared to locked-in switches, there is still need for testing and evaluation to assess whether open switches can replace traditional switches without loss of functionality, performance or ease of use, which the reduction in costs might not weigh up to. In this context, we examine the functioning of open switches running an open-source network operating system, namely SONiC, in a network. We study whether open switches with SONiC are able to deploy several fundamental networking features. In addition, we examine which use cases benefit from the open and flexible nature of open switches with SONiC. We therefore set out to answer the following two research questions: 1. Which networking features can be successfully deployed on open switches with SONiC? 2. Which use cases are (more) easily supported by open switches with SONiC? Chapter 2 provides background information about open networking, open switches, network operating systems (SONiC specifically) and routing suites. Chapter 3 will briefly discuss sev- eral networking features that will later be used in our experiments. Chapter 4 contains the experiments we performed to answer our first research question. In chapter 5, we discuss the experimental methods and results and other findings obtained during this research. In chapter 6, we examine several use cases of open switches with SONiC. Lastly, in chapter 7 we return to our research questions, formulate a conclusion and suggest possibilities for future research. 7 8 CHAPTER 2 Open networking background 2.1 Open switches Generally, a switch can be represented as a stack of four layered components. Figure 2.1 shows these components. Control and management plane Network operating system Hardware Silicon/ASIC Figure 2.1: Layered component stack of a switch. The silicon, or ASIC (application-specific integrated circuit), is a specified hardware element designed for a specific task. In the case of switches, this task is to quickly send packets through the network [4]. The hardware-layer includes all other physical components of the switch, like the interfaces, the input/output-ports, the LEDs and the power supply [5]. The network operating system (NOS) controls the hardware and the underlying ASIC for networking purposes and allows control and management plane applications to use the hardware. Control plane and management plane applications provide particular features to the user of the switch, in addition to those of the underlying operating system [6]. To understand the difference between open switches and traditional switches, one needs to consider the manner in which the above components interact with each other. Switches in which the NOS and the underlying hardware are disintegrated, meaning that they can be changed independently of each other, are called open switches. In traditional (locked-in) switches, this is not possible, for the switch is delivered with pre-installed software that cannot be changed. Open switches thus give the user more choice in what NOS to run. Open switches can be separated into subcategories. Bare metal switches provide the hardware required to run, and allow the user to load the NOS of choice. The manufacturers of bare metal switches are original design manufacturers (ODMs) for well-known switch merchants, which means that the ODMs products are re-branded and sold by other companies. A boot-loader allows the user to boot an NOS of choice on the device [7]. 9 2.2 Network operating systems A network operating system is a key component in the aforementioned component stack of open switches. The NOS controls the hardware of the device and provides applications on the switch with the hardware and software resources they need. These resources might for example be memory allocation or input and output resources. 2.2.1 ASIC communication The aforementioned application-specific integrated circuits (ASICs) are designed to handle a specific task. In networking switches, the ASIC is designed and optimized for quickly processing incoming packets according to the routing table. In order for an NOS to be able to program the ASIC, several APIs have been developed to communicate with the ASIC. The Switch Abstraction Interface1 (SAI) is a well-known method. It is an open source framework that aims to abstract away from the ASIC, which differs for each vendor, so software can be programmed for use in multiple different switches without any changes. This allows for more freedom in the use of software independently from the hardware choice [8]. Another, less adopted method for ASIC communication is the use of SDKs developed by the ASIC vendor. In practice, this approach is not included in open-source software for open switches considering changes in the SDKs would need the application to be modified. Examples of these SDKs are SwitchX SDK by Mellanox [9] and OpenNSL by Broadcom [10]. 2.2.2 ASIC control module On top of the ASIC API, the ASIC control module provides an interface for control plane ap- plications to communicate with the hardware. It also presents the current state of the hardware to the user of the switch. Control plane apps can use the ASIC control module to read from or write to data stored in the hardware. These applications therefore are independent from the hardware in the machine they are running on. Figure 2.2 shows the role of the ASIC control module and the ASIC API in the component stack illustrated before. Control and management plane Network operating system ASIC control module ASIC API Hardware Silicon/ASIC Figure 2.2: ASIC control module and ASIC API in the layered component stack. 1https://github.com/opencomputeproject/SAI 10 2.3 SONiC SONiC2 (Software for Open Networking in the Cloud) is an open-source network operating system that claims to include all features to have a fully functional Layer 3 network device. It is under constant development by Microsoft. The latest release, SONiC.201803, supports features including BGP, LLDP, link aggregation/LACP and VLAN (trunking) [11]. SONiC is Linux-based and runs on Debian Jessie. The SONiC architecture is depicted in figure 2.3.