Services Load Balancer for a Software Defined IXP Miguel Oliveira Rodrigues dos Santos Thesis to obtain the Master of Science Degree in Electrical and Computer Engineering Supervisors: Prof. Rui Antonio´ dos Santos Cruz Prof. Paulo Rogerio´ Barreiros d’Almeida Pereira Examination Committee Chairperson: Prof. Antonio´ Manuel Raminhos Cordeiro Grilo Supervisor: Prof. Rui Antonio´ dos Santos Cruz Member of the Committee: Prof. Joao˜ Lu´ıs Da Costa Campos Gonc¸alves Sobrinho November 2017 Acknowledgments I would like to thank my parents for their patience and support. To my whole family - each and every one of you had a unique role in this thesis. To Miguel Dias, Andre´ Pinho and Pedro Sacadura Botte for the after-work hours. A special thank you goes to Shams Valibhai for borrowing his technical knowledge and putting up with endless discussions that helped me improve my thesis. I would also like to acknowledge my supervisor Prof. Rui Santos Cruz for his guidance and expertise. To each and every one of you – Thank you. Abstract Border Gateway Protocol (BGP) has been used and studied for more than two decades. While its ubiquity makes it almost irreplaceable, its lack of flexibility is becoming more and more apparent. The advent of Software Defined Networking (SDN) technology opens new opportunities to scale interdomain routing. Industrial-scale Software Defined Internet Exchange Point (iSDX) is an implementation of an Internet Exchange Point (IXP) using SDN to emulate the usual features of an IXP. For a new technology to be useful, it cannot simply emulate the functionality of an old one; it has to excel at the old tasks and bring in new capabilities. This thesis extends iSDX to provide added value to the customers. In particular, it studies how to implement new services at the IXP in a scalable fashion. The study results in an implementation of a Services Load Balancer provided by the iSDX. Keywords Routing, Load Balancing, Ryu, Software Defined Networking (SDN), Internet Exchange Point (IXP), Industrial-scale Software Defined Internet Exchange Point (iSDX). iii Resumo O protocolo Border Gateway Protocol (BGP) e´ usado e estudado ha´ mais de duas decadas.´ A sua omnipresenc¸a na internet torna-o quase insubstitu´ıvel, mas a sua falta de flexibilidade e´ cada vez mais clara. O advento da tecnologia Software Defined Networking (SDN) abre novas oportunidades para escalar o encaminhamento inter-dom´ınio. Industrial-scale Software Defined Internet Exchange Point (iSDX) e´ uma implementac¸ao˜ de um Internet Exchange Point (IXP) que usa SDN para emular as funcionalidades mais comuns de um IXP. Mas para uma tecnologia ser util,´ nao˜ basta emular uma tecnologia anterior; tem de a superar. Esta tese estende a implementac¸ao˜ do iSDX para que esta possa oferecer novos servic¸os aos clientes do IXP. Em particular, estuda como implementar novos servic¸os de forma escalavel.´ O estudo e´ concretizado com um exemplo: um Balanceador de Servic¸os incorporado no iSDX. Palavras Chave Encaminhamento, Balanceamento de carga, Ryu, Software Defined Networking (SDN), Internet Ex- change Point (IXP), Industrial-scale Software Defined Internet Exchange Point (iSDX). v Contents 1 Introduction 1 1.1 A quick peek at the history of the Internet...........................3 1.2 Problem statement........................................4 1.3 Objective.............................................5 1.4 Contributions...........................................5 1.5 Document Structure.......................................5 2 State of the art 7 2.1AS internetworking........................................9 2.2 Peering.............................................. 11 2.2.1 Why peer?........................................ 11 2.2.2 How peering works.................................... 12 2.3 Internet Exchange Points.................................... 13 2.3.1 IXP operation....................................... 14 2.3.2 IXP’s business model.................................. 15 2.3.2.A Costs for customers.............................. 15 2.3.2.B Is it really cheaper than transit networks?.................. 15 2.3.3 European versus U.S. Internet Exchanges....................... 15 2.3.4 IXP architecture..................................... 16 2.3.5 A deeper look into remote peering........................... 18 2.3.5.A Remote Peering networking principles.................... 18 2.4 Anti-social peering practices................................... 18 2.4.1 Static next-hop misconfiguration............................ 19 2.4.2 Stealing traffic by announcing more specific prefixes................. 19 2.5 Software Defined Networking (SDN).............................. 20 2.5.1 The SDN architecture.................................. 21 2.5.2 Current status of SDN................................. 22 2.5.3 SDN Technologies: OpenFlow............................. 22 vii 2.5.4 SDN Technologies: Ryu SDN Framework....................... 23 2.5.5 Implementation of a learning switch using Ryu.................... 23 2.5.6 IXPs–The killer app for SDN.............................. 27 3 Related Work 29 3.1 SDX: A software defined IXP.................................. 31 3.2 iSDX: An Industrial-Scale Software Defined Internet Exchange Point............ 33 3.2.1 Scaling.......................................... 33 3.2.1.A Decomposing the IXP fabric into four tables................. 33 3.2.1.B Statically encoding BGP routing information in a tag............ 36 3.2.2 Architecture of the iSDX................................. 37 3.2.3 Critique of iSDX design................................. 38 3.2.3.A Security: is offloading responsibilities to participants’ routers a good idea? 38 3.2.3.B Scalability: Limitations concerning the number of participants....... 39 3.2.3.C Routing correctness.............................. 40 3.3 Current status of SDX technology................................ 40 4 Design and Development of a Service Load Balancer in the iSDX 43 4.1 Load Balancing in general–features and modes........................ 45 4.2 SDN-based load balancing................................... 46 4.3 Applying SDN-based load balancing to the iSDX....................... 47 4.3.1 Reasoning about a proactive implementation..................... 48 4.3.2 Reasoning about a reactive implementation...................... 49 4.3.3 Adding reactive load balancing while keeping previous functionality......... 50 4.3.4 Load distribution methods................................ 51 4.4 Design Rationale of iSDX Service Load Balancer....................... 52 4.4.1 Controller Logic...................................... 52 4.4.2 Configuring the iSDX Load Balancer.......................... 54 4.5 Development Process of the iSDX Service Load Balancer.................. 55 4.5.1 Test Environment Setup................................. 55 4.5.2 Development and Test Environment.......................... 56 4.5.3 Implementing load distribution methods........................ 57 4.5.4 Implementing the Controller Logic........................... 60 5 Validation of the implementation 63 5.1 Connectivity checks and bandwidth testing.......................... 65 5.2 Load balancing.......................................... 66 viii 6 Conclusion 71 6.1 Conclusions............................................ 73 6.2 System Limitations and Future Work.............................. 73 ix x List of Figures 2.1 Example of the propagation of routing information...................... 10 2.2 Daily traffic in AMS-IX between 31-Jan-2017 and 02-Feb-2017............... 13 2.3 Direct peering........................................... 17 2.4 Remote peering......................................... 17 2.5 Static next-hop misconfiguration................................ 19 2.6 Stealing traffic by announcing more specific prefixes..................... 20 2.7 SDN Architecture......................................... 21 2.8 A simple Mininet topology with Ryu Controller......................... 24 3.1 Multi-table scheme in the iSDX................................. 35 3.2 Setup of an inbound policy bypass attack........................... 39 3.3 A forwarding loop caused by two seemingly unrelated policies................ 40 4.1 Sample configuration of a load balanced service....................... 47 4.2 Table pipeline for the load balanced service.......................... 51 4.3 Development and Test environment............................... 56 5.1 Network used for testing purposes............................... 65 5.2 Distribution of sessions by the servers using a Round Robin load balancing approach.. 67 5.3 Distribution of sessions by the servers using a Weighted Round Robin load balancing approach............................................. 68 5.4 Distribution of sessions by the servers using a random load balancing approach..... 68 5.5 Distribution of sessions by the servers using a weighted random load balancing approach 69 xi xii List of Tables 2.1 Routing Policies......................................... 10 2.2 Internet Transit Prices...................................... 16 List of Algorithms 4.1 Installing static flow entries in the controller........................... 53 4.2 Installing static flow entries in the hardware to handle traffic targeted at the anycast Internet Protocol (IP) address....................................... 53 4.3 Installing static flow entries in the hardware to handle traffic coming from the anycast IP address............................................... 53 xiii xiv Listings 2.1 IPv6 ping from Cogent’s looking-glass server to Hurricane